We have a textual representation of contextual profiles for test scenarios, mainly. This patch moves that to YAML instead of JSON. YAML is more succinct and readable (some of the .ll tests should be illustrative). In addition, JSON is parse-able by the YAML reader.
A subsequent patch will address deserialization.
(thanks, @kazutakahirata, for showing me how to use the llvm YAML reader/writer APIs, which I incorrectly thought to be more low-level than the JSON ones!)
This reapplies 6d72bf47606, which was reverted in 57447d3ddf to investigate
build failures, e.g. https://lab.llvm.org/buildbot/#/builders/3/builds/10114.
The original patch contained an invalid unused friend declaration of
std::make_shared. This has been removed.
Also adds a new IdleTask type and updates DynamicThreadPoolTaskDispatcher to
schedule IdleTasks whenever the total number of threads running is less than
the maximum number of MaterializationThreads.
A SimpleLazyReexportsSpeculator instance maintains a list of speculation
suggestions ((JITDylib, Function) pairs) and registered lazy reexports. When
speculation opportunities are available (having been added via
addSpeculationSuggestions or when lazy reexports were created) it schedules
an IdleTask that triggers the next speculative lookup as soon as resources
are available. Speculation suggestions are processed first, followed by
lookups for lazy reexport bodies. A callback can be registered at object
construction time to record lazy reexport executions as they happen, and these
executions can be fed back into the speculator as suggestions on subsequent
executions.
The llvm-jitlink tool is updated to support speculation when lazy linking is
used via three new arguments:
-speculate=[none|simple] : When the 'simple' value is specified a
SimpleLazyReexportsSpeculator instances is used
for speculation.
-speculate-order <path> : Specifies a path to a CSV containing
(jit dylib name, function name) triples to use
as speculative suggestions in the current run.
-record-lazy-execs <path> : Specifies a path in which to record lazy function
executions as a CSV of (jit dylib name, function
name) pairs, suitable for use with
-speculate-order.
The same path can be passed to -speculate-order and -record-lazy-execs, in
which case the file will be overwritten at the end of the execution.
No testcase yet: Speculative linking is difficult to test (since by definition
execution behavior should be unaffected by speculation) and this is an new
prototype of the concept*. Tests will be added in the future once the interface
and behavior settle down.
* An earlier implementation of the speculation concept can be found in
llvm/include/llvm/ExecutionEngine/Orc/Speculation.h. Both systems have the
same goal (hiding compilation latency) but different mechanisms. This patch
relies entirely on information available in the controller, where the old
system could receive additional information from the JIT'd runtime via
callbacks. I aim to combine the two in the future, but want to gain more
practical experience with speculation first.
…#121991)"
This reverts commit f8f8598fd886cddfd374fa43eb6d7d37d301b576.
This breaks ARMv7 and s390x buildbot with the following message:
```
llvm-exegesis error: No available targets are compatible with triple "armv8l-unknown-linux-gnueabihf"
FileCheck error: '<stdin>' is empty.
FileCheck command line: /home/tcwg-buildbot/worker/clang-armv7-2stage/stage2/bin/FileCheck /home/tcwg-buildbot/worker/clang-armv7-2stage/llvm/llvm/test/tools/llvm-exegesis/dry-run-measurement.test
```
With the new benchmark phase, `dry-run-measurement`, llvm-exegesis can
run everything except the actual snippet execution. It is useful when we
want to test some parts of the code between the `assemble-measured-code`
and `measure` phase without actually running on native platforms.
ORC and JITLink debugging output write the dbgs() raw_ostream, which isn't
thread-safe. Use -num-threads=0 to force single-threaded linking for tests that
produce debugging output.
The llvm-jitlink tool is updated to suggest -num-threads=0 when debugging
output is enabled.
Adding a new flag(`--csprof-max-unsymbolized-context-depth`) to only
limit unsymbolized context depth. Currently,`--csprof-max-context-depth`
applies to both symbolized and unsymbolized profile context, there are
scenarios where `--csprof-max-context-depth` may not be flexible enough,
e.g. if we want to limit the context but still keep all the inlinings
from the leaf frame, we could set the value
csprof-max-unsymbolized-context-depth >= 1.
Also tested with Ubuntu on SiFive's HiFive Premier P550 board. Curiously
latency is reporting ~1.5 on basic scalar arithmetic, scalar mul is
~3.5, and div is ~36.5. This 0.5 cycles higher than I expect.
Recently https://github.com/llvm/llvm-project/pull/120991 broke a couple
of tests.
Also `macho-merged-funcs-dwarf.yaml` was already flaky due to some
non-determinism issues.
Fixing the previous code to not break tests and modifying
`macho-merged-funcs-dwarf.yaml` to fix the non-determinism (which will
be resolved later).
Adds the ability to lookup and display all merged functions for an
address in llvm-gsymutil.
Now, when `--merged-functions` is used in combination with
`--address/--addresses-from-stdin`, lookup results will contain
information about merged functions, if available.
To support printing merged function information when using the
`--verbose` option, the `LookupResult` data structure also had to be
extended with pointers to the raw function data and raw merged function
data. This is because merged functions share the same address range, so
it's not easy to look up the raw merged function data for a particular
`LookupResult` that is based on a merged function.
Commit edca1d9bad2 enabled threaded linking by default in llvm-jitlink, but we
need to handle the case where LLVM is built with -DLLVM_ENABLE_THREADS=Off.
This patch updates the llvm-jitlink tool to switch back to materialization on
the main thread (equivalent to llvm-jitlink -num-threads=0 ...) when LLVM is
built without thread support.
This reapplies edca1d9bad2 which was reverted in 7ec139ad4bc due to bot
failures.
LocalDependencyPropagation.s is updated to use -num-threads=0 in order to
avoid interleaving debugging output.
ELFNixPlatform.h is updated to protect the deferred runtime function calls
map during bootstrap.
In `llvm-cov show`, this option rounds counters (line, branch) to
`[1,0]` at rendering. This will be useful when the number of counts
doesn't interest but **Covered/uncoverd** does.
This should have been in the original commit, but I somehow forgot to
run git add && git commit --amend --no-edit between making the change in
my editor, saving the file, and pushing the commit.
When originally writing this feature up, I apparently completely forgot
to actually make the test exercise it and left an extra exit in the
function implementing the functionality without the appropriate
preprocessor macros around it, causing things to never work. This patch
should fix that.
Reapplies commit edca1d9bad2 which was reverted in 34531cff638 while I
investigated bot failures, (e.g.
https://lab.llvm.org/buildbot/#/builders/137/builds/10791).
Commit 158a60051d2 should address the -check failures on the bots, which
were caused by checks running earlier under the concurrent linking
scheme before all files referenced by the checks had been fully linked.
This patch also fixes the -threads option failure by renaming the option
to -num-threads to avoid clashing with the ThreadCount cl::opt variable
defined in ThinLTOCodeGenerator.cpp.
ORC dependence tracking is fine-grained (i.e. per-symbol), however when running
-check mode we want to wait for all links triggered by the entry point lookup
to complete, regardless of whether the code / data in them is actually
reachable from the entry point. This simplifies test-cases, since authors don't
need to reason about per-symbol dependencies to know that additional files will
be linked (if referenced transitively in any way from the test-case).
The new Session::waitForFilesLinkedFromEntryPointFile utility does _not_ wait
for lazily linked (-lazy) files.
This will be used to fix buildbot errors caused by edca1d9bad2.
Adds a -threads option to llvm-jitlink. By default llvm-jitlink will now use a
DynamicThreadPoolTaskDispatcher with the number of materialization threads set
to whatever is returned by std::hardware_concurrency(). This brings the default
in-place linking behavior in line with the concurrent linking that is used for
-oop-executor and -oop-executor-connect mode.
In-place linking on the main thread can be forced by passing -threads=0.
This patch also makes following amendments to core exegesis:
* Added distinction between regular registers aliasing check and
registers used as memory address in instruction.
* Added scratch memory space pointer register.
* General exegesis options were amended:
* mattr - new option to pass a list of enabled target features
Llvm-exegesis RISCV port is a result of team effort. Below everyone
involved listed.
Co-authored-by: Konstantin Vladimirov
<konstantin.vladimirov@syntacore.com>
Co-authored-by: Dmitrii Petrov <dmitrii.petrov@syntacore.com>
Co-authored-by: Dmitry Bushev <dmitry.bushev@syntacore.com>
Co-authored-by: Mark Goncharov <mark.goncharov@syntacore.com>
Co-authored-by: Anastasiya Chernikova
<anastasiya.chernikova@syntacore.com>
Original pr: #89047
---------
Co-authored-by: Kazu Hirata <kazu@google.com>
This patch also makes following amendments to core exegesis:
* Added distinction between regular registers aliasing check and
registers used as memory address in instruction.
* Added scratch memory space pointer register.
* General exegesis options were amended:
* mattr - new option to pass a list of enabled target features
Llvm-exegesis RISCV port is a result of team effort. Below everyone
involved listed.
Co-authored-by: Konstantin Vladimirov
<konstantin.vladimirov@syntacore.com>
Co-authored-by: Dmitrii Petrov <dmitrii.petrov@syntacore.com>
Co-authored-by: Dmitry Bushev <dmitry.bushev@syntacore.com>
Co-authored-by: Mark Goncharov <mark.goncharov@syntacore.com>
Co-authored-by: Anastasiya Chernikova
<anastasiya.chernikova@syntacore.com>
---------
Co-authored-by: Anastasiya Chernikova <anastasiya.chernikova@syntacore.com>
This patch also makes following amendments to core exegesis:
* Added distinction between regular registers aliasing check and
registers used as memory address in instruction.
* Added scratch memory space pointer register.
* General exegesis options were amended:
* mattr - new option to pass a list of enabled target features
Llvm-exegesis RISCV port is a result of team effort. Below everyone
involved listed.
Co-authored-by: Konstantin Vladimirov
<konstantin.vladimirov@syntacore.com>
Co-authored-by: Dmitrii Petrov <dmitrii.petrov@syntacore.com>
Co-authored-by: Dmitry Bushev <dmitry.bushev@syntacore.com>
Co-authored-by: Mark Goncharov <mark.goncharov@syntacore.com>
Co-authored-by: Anastasiya Chernikova
<anastasiya.chernikova@syntacore.com>
---------
Co-authored-by: Dmitry Bushev <dmitry.bushev@syntacore.com>
NFC except for calculating `Total`. I've replaced
`(uint64_t)+(uint64_t)` with `(double)+(double)`.
This is still inexact with large numbers `(1LL << 53)` but will be expected to prevent possible overflow.
This change adds support for loading gSYM callsite information from
DWARF. Previously the only support was for loading callsites info from
YAML.
For testing, we add a pass where `macho-gsym-merged-callsites-dsym`
loads callsite info from DWARF rather than YAML.
```
% echo 90 | llvm-mc -triple=x86_64 --disassemble --hex
.text
nop
```
The initial `.text` kludge is due `initSection`, which is actually only
needed by AIX XCOFF for its `getCurrentSectionOnly()` use in
MCAsmStreamer::emitInstruction (https://reviews.llvm.org/D95518). Adjust
MCAsmStreamer::emitInstruction to not trigger failures on
```
echo 7c4303a6 | llvm-mc --cdis --hex --triple=powerpc-aix-ibm-xcoff
```
Pull Request: https://github.com/llvm/llvm-project/pull/120185
`--disassemble`/`--cdis` parses input bytes as decimal, 0bbin, 0ooct, or
0xhex. While the hexadecimal digit form is most commonly used, requiring
a 0x prefix for each byte (`0x48 0x29 0xc3`) is cumbersome.
Tools like xxd -p and rz-asm use a plain hex dump form without the 0x
prefix or space separator. This patch adds --hex to disassemble such hex
bytes with optional whitespace.
```
% rz-asm -a x86 -b 64 -d 4829c34829c4
sub rbx, rax
sub rsp, rax
% llvm-mc -triple=x86_64 --cdis --hex --output-asm-variant=1 <<< 4829c34829c4
.text
sub rbx, rax
sub rsp, rax
```
Pull Request: https://github.com/llvm/llvm-project/pull/119992
This relands commit #115111.
Use traditional way to update post dominator tree, i.e. break critical
edge splitting into insert, insert, delete sequence.
When splitting critical edges, the post dominator tree may change its
root node, and `setNewRoot` only works in normal dominator tree...
See
6c7e5827ed/llvm/include/llvm/Support/GenericDomTree.h (L684-L687)
This patch does two things:
- During deserialization, we accept a function name for Frame as an
alternative to the usual GUID expressed as a hexadecimal number.
- During serialization, we print a GUID of Frame as a 16-digit
hexadecimal number prefixed with 0x in the usual way. (Without this
patch, we print a decimal number, which is not customary.)
The patch uses a machinery called "normalization" in YAML I/O, which
lets us serialize and deserialize into an alternative data structure.
For our use case, we have an alternative Frame data structure, which
is identical to "struct Frame" except that Function is of type
GUIDHex64 instead of GlobalValue::GUID. This alternative type
supports the two bullet points above without modifying "struct Frame"
at all.
