LLVM is inconsistent about how it converts `errno` to `std::error_code`.
This can cause problems because values outside of `std::errc` compare
differently if one is system and one is generic on POSIX systems.
This is even more of a problem on Windows where use of the system
category is just wrong, as that is for Windows errors, which have a
completely different mapping than POSIX/generic errors. This patch fixes
one instance of this mistake in `JSONTransport.cpp`.
This patch adds `errnoAsErrorCode()` which makes it so people do not
need to think about this issue in the future. It also cleans up a lot of
usage of `errno` in LLVM and Clang.
If notifyEmitted encounters a failure (either because some plugin returned one,
or because the ResourceTracker was defunct) then we need to deallocate the
FinalizedAlloc manually.
No testcase yet: This requires a concurrent setup -- we'll need to build some
infrastructure to coordinate links and deliberately injected failures in order
to reliably test this.
Remove an overly aggressive cantFail: This call to defineMaterializing should
never fail with a duplicate symbols error (since all new symbols shoul be
weak), but may fail if the tracker has become defunct in the mean time. In that
case we need to propagate the error.
[ORC] Re-land https://github.com/llvm/llvm-project/pull/81826
This patch adds two plugins: VTuneSupportPlugin.cpp and
JITLoaderVTune.cpp. The testing is done in a manner similar to
llvm-jitlistener. Currently, we only support the old version of Intel
VTune API.
The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.
This is a breaking change: clients who use to create a ThreadPool must
now create a DefaultThreadPool instead.
API clients can now set a MachO::HeaderOptions::BuildVersionOpts field to have
MachOPlatform add an LC_BUILD_VERSION load command to the Mach header for each
JITDylib.
No testcase yet. In the future we'll try to add a MachO parser to the ORC
runtime and extra test options to llvm-jitlink for this.
This commit also incidentally fixes a bug in the MachOBuilder class that lead to
a delegation cycle.
This patch adds two plugins: VTuneSupportPlugin.cpp and
JITLoaderVTune.cpp. The testing is done in a manner similar to
llvm-jitlistener. Currently, we only support the old version of Intel
VTune API.
This pull request is stacked on top of
https://github.com/llvm/llvm-project/pull/81825
The SectCreateMaterializationUnit creates a LinkGraph with a single named
section containing a single named block whose content is given by a
MemoryBuffer. It is intended to support emulation of ld64's -sectcreate option.
Right now InProcessMemoryManager only releases a standard segment (via
sys::Memory::releaseMappedMemory) in `deallocate` when there is a
DeallocAction associated, leaving residual memory pages in the process
until termination.
Despite being a de facto memory leak, it won't cause a major issue if
users only create a single LLJIT instance per process, which is the most
common use cases. It will, however, drain virtual memory pages if we
create thousands of ephemeral LLJIT instances in the same process.
This patch fixes this issue by releasing every standard segments
regardless of the attached DeallocAction.
Switch the primary implementation of EPC lookupSymbols to be async,
keeping a synchronous wrapper for compatibility. Use the new async
implementation inside EPCDynamicLibrarySearchGenerator to work
working towards a fully async search generator.
Provide an asynchronous lookup API for EPCGenericDylibManager and adopt
that from the SimpleRemoteEPC. This enables an end-to-end async
EPCDynamicLibrarySearchGenerator. Note: currently we keep the current
per-dlhandle lookup model, but a future improvement could do a single
async call for a given lookup operation.
As noted in issues #68594 and #73935, `JITLink/RISCV/ELF_ehframe.s`
fails with libstdc++'s expensive checks because `getRISCVPCRelHi20`
calls `std::equal_range` on the edges which may not be ordered by their
offset. Instead let `ELFJITLinker_riscv` build a hashmap of all edges
with type `R_RISCV_PCREL_HI20` that can be looked up in constant time.
Closes#73935
The error check should be performed after the iterator increment, not before
it. Thanks to @dcb314 for catching this!
Fixes github.com/apple/swift/issues/81119
This code appears to be a hack to set the features to include compressed
instructions if the ELF EFLAGS flags bit is present, but the ELF
attribute for the ISA string is no present or not accurate.
We can't remove the hack because llvm-mc doesn't create ELF attributes
by default so a lot of tests fail to disassembler properly. Using clang
as the assembler does set the attributes.
This patch changes the hack to only set Zca since that is the minimum
implied by the flag. Setting anything else potentially conflicts with
the ISA string containing Zcmp or Zcmt.
JITLink also needs to be updated to recognize Zca in addition to C.
Removes the MaterializationResponsibility::addDependencies and
addDependenciesForAll methods, and transfers dependency registration to
the notifyEmitted operation. The new dependency registration allows
dependencies to be specified for arbitrary subsets of the
MaterializationResponsibility's symbols (rather than just single symbols
or all symbols) via an array of SymbolDependenceGroups (pairs of symbol
sets and corresponding dependencies for that set).
This patch aims to both improve emission performance and simplify
dependence tracking. By eliminating some states (e.g. symbols having
registered dependencies but not yet being resolved or emitted) we make
some errors impossible by construction, and reduce the number of error
cases that we need to check. NonOwningSymbolStringPtrs are used for
dependence tracking under the session lock, which should reduce
ref-counting operations, and intra-emit dependencies are resolved
outside the session lock, which should provide better performance when
JITing concurrently (since some dependence tracking can happen in
parallel).
The Orc C API is updated to account for this change, with the
LLVMOrcMaterializationResponsibilityNotifyEmitted API being modified and
the LLVMOrcMaterializationResponsibilityAddDependencies and
LLVMOrcMaterializationResponsibilityAddDependenciesForAll operations
being removed.
The JITLink AArch32 backend reached feature-parity with RuntimeDyld
on ELF-based systems. This patch changes the default JIT-linker in Orc's
LLJIT for these platforms.
This allows us to run clang-repl with JITLink on ARM and use all the
features we had with RuntimeDyld before. All existing tests for
clang-repl are passing.
This stub type loads an absolute address directly into the PC register.
It's the simplest and most compatible way to implement a branch
indirection across the entire address space (and probably the slowest as
well). It's the ideal fallback for all targets for which we did not
(yet) implement a more performant solution.
`R_ARM_PREL31` is a 31-bits relative data relocation where the
most-significant bit is preserved. It's used primarily in `.ARM.exidx`
sections, which we skipped processing until now, because we didn't
support the relocation type. This was implemented in RuntimeDyld with
https://reviews.llvm.org/D25069 and I implemented it in a similar way in
JITLink in order to reach feature parity.
We want to emit stubs that match the instruction set state of the
relocation site. This is important for branches that have no built-in
switch for the instruction set state. It's the case for Jump24
relocations. Relocations on instructions that support switching on
the fly will be rewritten in a relaxation step in the future. This
affects Call relocations on `BL`/`BLX` instructions.
In this patch, the StubManager gains a second stub symbol slot for each
target and selects which one to use based on the relocation type. For
testing, we select the appropriate slot with a stub-kind filter, i.e.
`arm` or `thumb`. With that we can implement Armv7 stubs and test
that we can have both kinds of stubs for a single external symbol.
We use `jitlink-check` lines in LIT tests as the primary tool for
testing JITLink backends. Parsing and evaluation of the expressions is
implemented in `RuntimeDyldChecker`. The `stub_addr(obj, name)`
expression allows to obtain the linker-generated stub for the external
symbol `name` in object file `obj`.
This patch adds support for a filter parameter to select one out of many
stubs. This is necessary for the AArch32 JITLink backend, which must be
able to emit two different kinds of stubs depending on the instruction
set state (Arm/Thumb) of the relocation site. Since the new parameter is
optional, we don't have to update existing tests.
Filters are regular expressions without brackets that match exactly one
existing stub. Given object file `armv7.o` with two stubs for external
function `ext` of kinds `armv7_abs_le` and `thumbv7_abs_le`, we get the
following filter results e.g.:
```
stub_addr(armv7.o, ext, thumb) thumbv7_abs_le
stub_addr(armv7.o, ext, thumbv7) thumbv7_abs_le
stub_addr(armv7.o, ext, armv7_abs_le) armv7_abs_le
stub_addr(armv7.o, ext, v7_.*_le) Error: "ext" has 2 candidate stubs in file "armv7.o". Please refine stub-kind filter "v7_.*_le" for disambiguation (encountered kinds are "thumbv7_abs_le", "armv7_abs_le").
stub_addr(armv7.o, ext, v8) Error: "ext" has 2 stubs in file "armv7.o", but none of them matches the stub-kind filter "v8" (all encountered kinds are "thumbv7_abs_le", "armv7_abs_le").
```
Add a HeaderOptions struct that can be used to configure commonly-used
load commands LC_ID_DYLIB, LC_LOAD_DYLIB, and LC_RPATH when setupDylib
creates a mach-o header.
Adds a function to create a LinkGraph of absolute symbols, and a
callback in dynamic library search generators to enable using it to
expose its symbols to the platform/orc runtime. This allows e.g. using
__orc_rt_run_program to run a precompiled function that was found via
dlsym. Ideally we would use this in llvm-jitlink's own search generator,
but it will require more work to align with the Process/Platform
JITDylib split, so not handled here.
As part of this change we need to handle LinkGraphs that only have
absolute symbols.
This migrates the dylib manager lookup and related APIs to replace
ExecutorAddress with ExecutorSymbolDef so that in the future we can
model JITSymbolFlags for these symbols. The current change should be NFC
as we are only setting the Exported symbol flag.
Vectors are always bit-packed and don't respect the elements' alignment
requirements. This is different from arrays. This means offsets of
vector GEPs need to be computed differently than offsets of array GEPs.
This PR fixes many places that rely on an incorrect pattern
that always relies on `DL.getTypeAllocSize(GTI.getIndexedType())`.
We replace these by usages of `GTI.getSequentialElementStride(DL)`,
which is a new helper function added in this PR.
This changes behavior for GEPs into vectors with element types for which
the (bit) size and alloc size is different. This includes two cases:
* Types with a bit size that is not a multiple of a byte, e.g. i1.
GEPs into such vectors are questionable to begin with, as some elements
are not even addressable.
* Overaligned types, e.g. i16 with 32-bit alignment.
Existing tests are unaffected, but a miscompilation of a new test is fixed.
---------
Co-authored-by: Nikita Popov <github@npopov.com>
https://github.com/llvm/llvm-project/pull/76236 introduced the forth
copy and it was time to deduplicate. This patch brings it back to 2,
one in OrcTargetProcess and one in legacy ExecutionEngine.
For out-of-process support the DynamicLibrarySearchGenerator must go
through EPC, otherwise we lookup symbols from the host and not the
target process.
Previously the JITLink MachO backends (aarch64 and x86-64) only looked at the
fixup block to determine which symbol was being fixed up. This assumption breaks
if both symbols used in the subtractor are in the same block. The fix is to
check for such cases and use the offsets of each symbol to decide which is being
fixed up.
The issue only resulted in incorrect behavior for negative-delta relocations,
so the testcases use eh-frames with explicit edges for the CIE-pointer field in
FDEs (since these are negative-deltas).
rdar://119351329
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
LLJIT users may want to customize their native platform setup beyond what the
ExecutorNativePlatform helper permits. In this case LLJIT users can construct
the platform instance manually, and then add an ORCPlatformSupport instance
to forward initialize/deinitialize operations to the ORC runtime.
SimpleMachOHeaderMU can be used as a convenient base for classes that create
custom MachO headers. Instances of these custom classes can be used by passing
a MachOHeaderMUBuilder using the MachOPlatform extensions added in ef314d39b92.
Allow C-API users to debug their JITed code via the GDB JIT Interface.
This is currently supported on ELF and MachO based platforms. On
other systems `LLVMOrcLLJITEnableDebugSupport()` returns an error.
This patch adds a new C-API header `LLJITUtils.h`, which can host
further advanced JIT features in the future. Using the header requires
linking against LLVMOrcDebugging.
1. Prevent deadlock by unlocking JDStatesMutex when calling back to the
controller to request a push of new symbols. (If JDStatesMutex is locked
then the push operation can't register the new symbols, and so can't
complete).
2. Record MachOPlatform runtime symbols during bootstrap and attach their
registration to the bootstrap-completion graph, similar to the way that
deferred allocation actions are handled. We can't register the symbols
the normal way during bootstrap since the symbol registration function is
itself in the process of being materialized.
3. Add dlsym testcases to exercise these fixes.
Replaces an llvm::cast that assumed that all Binary instances were either
Archive or MachOUniversalBinary instances with a dyn_cast. The cast was
triggering an assert in StaticLibraryDefinitionGenerator::Load if that method
was given a path or MemoryBuffer containing a relocatable object file.
Switching to dyn_cast causes the operation to error out with a bad-format
error as expected.
Fixes rdar://119262300
HeaderAddr shouldn't be a member variable of MachOPlatformPlugin: there's only
one plugin instance shared between all JITDylibs, so the shared HeaderAddr will
be overwritten in an unpredictable and unsafe way. We haven't seen any issues
due to this yet, but it triggered failures during testing of an upcoming
llvm-jitlink patch (e.g. ORC-RT test Darwin/x86-64/jit-re-dlopen-trivial.S).
This patch pre-fixes the issue in advance of the llvm-jitlink patch landing.
This patch also removes some stale debugging output in MachOPlatform.
Adds symbol tables to the JITDylibState struct in the ORC runtime
MachOPlatformRuntimeState class. This table will hold the addresses of
materialized symbols (registered by a new JITLink pass in MachOPlatform),
allowing these to be looked up in the executor without an IPC request to the
controller.
The old lookup-symbols callback (made by the runtime in response to dlsym
lookups) is replaced with a push-symbols callback that can trigger
materialization of requested symbols.
Holding a symbol table on the executor side should make repeat calls to dlsym
(and other symbol lookup operations) cheaper since the IPC to trigger
materialization happens at most once per symbol. It should also enable us (at
some point in the future) to symbolicate backtraces in JIT'd code even if the
controller process is gone (e.g. detached or crashed). The trade-off for this
is increased memory consumption in the executor and larger JIT'd data transfers
(since symbol names are now transferred to the executor unconditionally, even
though they may never be used).
