If there is no debug information, we wouldn't call
`DebugObject::collectTargetAlloc` in the post-allocation phase.
Therefore, when it's in the post-fixup phase,
`DebugObject::awaitTargetMem` will fail with _"std::future_error: No
associated state"_ because the std::future was not even populated.
On some systems, backtraces contain addresses with their high bits set*.
These high bits prevent symbolication using the JIT symbol table. Since
this test is for a best-effort debugging / diagnosis tool it seems best
to remove the test until/unless we can get it passing on all systems, or
find some way to identify systems that will fail.
See discussion in https://github.com/llvm/llvm-project/pull/175537.
* Note that the test does not use PAC or pointer tagging -- the high
bits are coming from somewhere else. Possibly libunwind, but that is
just speculation.
Optimize the GOT and Stub relocations if the edge target address is in
range from call site - Indirect jump by plt stub can be replaced with
direct jump to target in post-allocation optimization.
Ensures that the address scraping passes are added to the JIT linker pipeline
if either of the -check or -show-addrs are passed. Prior to this commit we only
considered -check, so passing -show-addrs on its own was printing an empty
(unpopulated) symbol table.
This change exposed bugs in RuntimeDyldChecker's `MemoryRegionInfo` type, aned
llvm-jitlink's symbol printing code:
- `MemoryRegionInfo` was assuming that zero-sized symbols wouldn't be defined
and was using Size == 0 as a proxy for "region info uninitialized". This was
triggering assertions for valid LinkGraphs containing zero-sized symbols.
- llvm-jitlink's symbol printing code was assuming that all defined symbols
contained content, and was not correctly handling zero-fill / absolute
symbols. This triggered assertions about requesting content addresses for
non-content symbols.
This commit fixes the issues above, and adds both a generic testcase for the
-show-addrs option, and an ELF-specific testcase with a zero-sized symbol to
check that the option works for such symbols.
…#175242)
This reapplies 451ca458cf51d553f5c49e67d841280e8166f933, which was
reverted in 25976e83606f1a7615e3725e6038bb53ee96c3d5 due to bot
failures.
The REQUIRES line has been further constrained to try to address the
failures.
This reverts commit 906b48616c03948a4df62a5a144f7108f3c455e8.
The forward fix for this got reverted in
25976e83606f1a7615e3725e6038bb53ee96c3d5, so reverting the original
commit given it is still broken and the forward fix that mitigated most
of the issues is no longer in tree.
Some builds (e.g.
https://lab.llvm.org/buildbot/#/builders/154/builds/26180) failed for
this testcase, for reasons that aren't immediately clear from the logs.
Since the options being tested are for debugging only, this commit
switches to testing Darwin/arm64 only.
Whne similar backtrace symbolication options are added to lli we can
reintroduce a generic testcase and start investigating these failures.
This patch adds tools for capturing symbol information from JIT'd code
and using it to symbolicate backtraces. This is useful for debugging
crashes in JIT-compiled code where traditional symbolication tools may
not have access to the JIT symbol table. These tools are not a general
solution to the JIT symbolication problem (that will require further
integration with system components like libunwind, the dynamic linker,
and/or crash tracing tools), but will aid in JIT debugging and
development until a general solution is available.
APIs Added:
1. SymbolTableDumpPlugin - A LinkGraphLinkingLayer::Plugin that captures
symbol information as code is JIT'd and writes it to a file.
- Create(StringRef Path) ->
Expected<std::shared_ptr<SymbolTableDumpPlugin>> Creates a plugin that
appends symbol information to the specified file.
- Symbol table format: "<link graph name>" <address> <symbol name>
<address> <symbol name> ...
The plugin uses a PostAllocationPass to write symbols after addresses
have
been assigned but before the code is finalized.
2. DumpedSymbolTable - A class for symbolicating backtraces using a
previously dumped symbol table.
- Create(StringRef Path) -> Expected<DumpedSymbolTable> Loads and parses
a symbol table from a file.
- symbolicate(StringRef Backtrace) -> std::string Given text of a
backtrace, for rows ending with a hex address, adds the symbol name,
offset, and defining graph name.
New `llvm-jitlink` Command Line Options:
1. -write-symtab=<path> Enables the SymbolTableDumpPlugin to write
symbol information to the specified file as objects are JIT'd. The
symbol table can then be used to symbolicate backtraces from crashes or
signal handlers.
2. -symbolicate-with=<path> Runs llvm-jitlink in symbolication mode.
Reads the symbol table from <path> and symbolicates backtraces read from
stdin or input files.
Usage Examples:
$ llvm-jitlink -write-symtab=symbols.txt mycode.o
$ llvm-jitlink -symbolicate-with=symbols.txt - < backtrace.txt
So far, the setting enforced only jitlink but not rtdyld. We get better
test coverage now that we honor both cases. We drop EPC-based execution
on the way, because with ORC lli always executes in-process.
Mark ELF_ppc64_relocations.s as unsupported on SystemZ because of cross
build issue related to using dlsym for host symbols.
Test fails to resolve __tls_get_aadr on SystemZ host.
Co-authored-by: anoopkg6 <anoopkg6@github.com>
When scanning an interface source (dylib or TBD file), consider
"fallback" architectures (CPUType / CPUSubType pairs) in addition to the
process's CPUType / CPUSubType.
Background:
When dyld loads a dylib into a process it may load dylib or slice whose
CPU type / subtype isn't an exact match for the process's CPU type /
subtype. E.g. arm64 processes can load arm64e dylibs / slices.
When building an interface we need to follow the same logic, otherwise
we risk generating a spurious "does not contain a compatible slice"
error. E.g. If we're running an arm64 JIT'd program and loading an
interface from a TBD file, and if no arm64 slice is present in that
file, then we should fall back to looking for an arm64e slice.
rdar://164510783
The "MachO_ptrauth_noolloc_sections.yaml" testcase had a typo in the
name, and didn't explicitly specify its endianness. This was causing it
to fail when enabled on big endian platforms.
See conversation in https://github.com/llvm/llvm-project/pull/167902
Set up initial infrastructure for SystemZ architecture support in
JITLink. It includes features like GOT and PLT handling. Relaxation of
GOT/PLT and support for TLS were intentionally left out for the moment.
Support for TLS might require info regarding moduleID. This could
further mean changes to target independent part of JITLink and library
support.
---------
Co-authored-by: anoopkg6 <anoopkg6@github.com>
…fixes.
This reapplies c8c86efbbb5, which was reverted in 13ca8723d1b due to bot
failures. Those failures were compilation errors exposed when LLVM is
built with LLVM_ENABLE_EXPENSIVE_CHECKS=On. They have been fixed in this
commit.
WaitingOnGraph tracks waiting-on relationships between nodes (intended
to represent symbols in an ORC program) in order to identify nodes that
are *Ready* (i.e. are not waiting on any other nodes) or have *Failed*
(are waiting on some node that cannot be produced).
WaitingOnGraph replaces ORC's baked-in data structures that were
tracking the same information (EmissionDepUnit, EmissionDepUnitInfo,
...). Isolating this information in a separate data structure simplifies
the code, allows us to unit test it, and simplifies performance testing.
The WaitingOnGraph uses several techniques to improve performance
relative to the old data structures, including symbol coalescing
("SuperNodes") and symbol keys that don't perform unnecessary reference
counting (NonOwningSymbolStringPtr).
This commit includes unit tests for common dependence-tracking issues
that have led to ORC bugs in the past.
…cation.
Replaces a call to ObjectFile::makeTriple (still used for ELF and COFF)
with a call to MachOObjectFile::getArchTriple. The latter knows how to
build correct triples for different MachO CPU subtypes, e.g. arm64 vs
arm64e, which is important for selecting the right slice from universal
archives.
Adds the name and triple of the graph to LinkGraph::dump output before
the rest of the graph content. Calls from JITLinkGeneric.cpp to dump the
graph are updated to avoid redundantly naming the graph.
Cygwin is like Windows in that it uses COFF, and doesn't emit
.debug_frame on 64-bit architectures.
However, if -elf is appended to the target triple on Cygwin MCJIT remote
tests fail due to `__register_frame` not being defined. Only one test
fails without -elf that succeeds with it, so mark just that test as
XFAIL on Cygwin.
This patch removes all uses of %T from within LLVM tests. %T has been
deprecated for about seven years and use is not advised given it is not
unique per test and can thus lead to races. The goal of this is to
eventually remove support for %T from lit.
When JITLink reports an out-of-range error, the underlying reason could
be hidden from the user if it's due to an excessively large target
addend. Add non-zero target addend to the message for clarity.
…eFromDylib.
When comparing CPU subtypes from slices in a MachO universal binary we
need to apply the MachO::CPU_SUBTYPE_MASK to mask out any flags in the
high bits, otherwise we might fail to correctly match a slice and return
a spurious "does not contain slice" error.
rdar://153913779
Its presence seems to actively hinder the ToolSubst mechanism that was
supposed to fill in the path to the tool, which prevented these tests
from working on Cygwin.
This flag has been superseded by `.option exact`, as the test updates
show.
Given the flag was always hidden, it makes sense to me to remove it, and
move tests that required it to use `.option exact`.
This reapplies b0979b8c65d, which was reverted in 370aecb9572 due to bot
failures. The failures were caused by typos in the testcase that are fixed in
this patch.
This refactor was motivated by two bugs identified in out-of-tree
builds:
1. Some implementations of the VisitMembersFunction type (often used to
implement special loading semantics, e.g. -all_load or -ObjC) were assuming
that buffers for archive members were null-terminated, which they are not in
general. This was triggering occasional assertions.
2. Archives may include multiple members with the same file name, e.g.
when constructed by appending files with the same name:
% llvm-ar crs libfoo.a foo.o
% llvm-ar q libfoo.a foo.o
% llvm-ar t libfoo.a foo.o
foo.o
While confusing, these members may be safe to link (provided that they're
individually valid and don't define duplicate symbols). In ORC however, the
archive member name may be used to construct an ORC initializer symbol,
which must also be unique. In that case the duplicate member names lead to a
duplicate definition error even if the members define unrelated symbols.
In addition to these bugs, StaticLibraryDefinitionGenerator had grown a
collection of all member buffers (ObjectFilesMap), a BumpPtrAllocator
that was redundantly storing synthesized archive member names (these are
copied into the MemoryBuffers created for each Object, but were never
freed in the allocator), and a set of COFF-specific import files.
To fix the bugs above and simplify StaticLibraryDefinitionGenerator this
patch makes the following changes:
1. StaticLibraryDefinitionGenerator::VisitMembersFunction is generalized
to take a reference to the containing archive, and the index of the
member within the archive. It now returns an Expected<bool> indicating
whether the member visited should be treated as loadable, not loadable,
or as invalidating the entire archive.
2. A static StaticLibraryDefinitionGenerator::createMemberBuffer method
is added which creates MemoryBuffers with unique names of the form
`<archive-name>[<index>](<member-name>)`. This defers construction of
member names until they're loaded, allowing the BumpPtrAllocator (with
its redundant name storage) to be removed.
3. The ObjectFilesMap (symbol name -> memory-buffer-ref) is replaced
with a SymbolToMemberIndexMap (symbol name -> index) which should be
smaller and faster to construct.
4. The 'loadability' result from VisitMemberFunctions is now taken into
consideration when building the SymbolToMemberIndexMap so that members
that have already been loaded / filtered out can be skipped, and do not
take up any ongoing space.
5. The COFF ImportedDynamicLibraries member is moved out into the
COFFImportFileScanner utility, which can be used as a
VisitMemberFunction.
This fixes the bugs described above; and should lower memory consumption
slightly, especially for archives with many files and / or symbol where
most files are eventually loaded.
Linux perf_events is not implemented in WSL1, skip the test that
requires it.
There is just a single test that requires perf_events. It fails under
WSL1 with:
```sh
env JITDUMPDIR=/home/meinersbur/build/llvm-project/release/test/ExecutionEngine/JITLink/x86-64/Output/ELF_perf.s.tmp /home/meinersbur/build/llvm-project/release/bin/llvm-jitlink -perf-support /home/meinersbur/build/llvm-project/release/test/ExecutionEngine/JITLink/x86-64/Output/ELF_perf.s.tmp/ELF_x86-64_perf.o
llvm-jitlink error: PerfState not initialized
```
WSL environment detection logic follows
https://github.com/scivision/detect-windows-subsystem-for-linux/blob/main/is_wsl.py
Also see WSL issue: https://github.com/microsoft/WSL/issues/4595
The context string can be added to indicate the source of the duplicate
definition. E.g. if the context is set to "foo2.o", then:
"Duplicate definition of symbol 'foo'"
becomes
"In foo2.o, duplicate definition of symbol 'foo'".
The JITDylib::defineImpl method is updated to use the name of the
MaterializationUnit being added as the context string for duplicate definition
errors. The JITDylib::defineMaterializing method is updated to use
"defineMaterializing operation" as the conext string.
When building on i386, both clang and gcc define a builtin 'i386' macro (see
discussion in https://github.com/llvm/llvm-project/pull/137063). This causes
build errors in the JITLink/i386 backend when attempting to build LLVM on i386.
This commit renames the 'i386' backend (namespaces, APIs and files) to 'x86' to
avoid this issue.
R_386_GOT32 was already handled by lowering to
EdgeKind_i386::RequestGOTAndTransformToDelta32FromGOT. R_386_GOT32X is just an
optionally relaxable version of R_386_GOT32, so we can lower to the same edge
kind.
I've left a TODO to add a relaxable edge kind and update the i386 relaxation
optimization in the future, though I'll probably leave this as an exercise for
any i386 aficionados out there. ;)
As this test is about executing x86_64 code with "lli", we should only
try to do it if we actually execute on an x86_64 target. So instead of
XFAILing individual architectures that can't execute it, instead change
this into requiring an x86_64 target.
Also generalize the target triple form used; don't assume that the
vendor field is set to "pc" - many mingw toolchains use the vendor field
set to "w64".
Ptrauth relocations can only be fixed up in the executing process, but noalloc
sections do not have any memory in the executing process. Failure to skip
ptrauth edges results in signing instructions that operate on invalid addresses,
leading to segfaults or data corruption.
Ignoring noalloc sections for ptrauth lowering purposes allows the ptrauth
edges to persist until they reach the applyFixup method, at which point they
raise a useful error and cleanly terminate linking.
