This reapplies 427fb5cc5ac, which was reverted in 08c1a6b3e18 due to bot
failures.
The fix was to remove an incorrect assertion: In IL_emit, during the initial
worklist loop, an EDU can have all of its dependencies removed without becoming
ready (because it may still have implicit dependencies that will be added back
during the subsequent propagateExtraEmitDeps operation). The EDU will be marked
Ready at the end of IL_emit if its Dependencies set is empty at that point.
Prior to that we can only assert that it's either Emitted or Ready (which is
already covered by other assertions).
AsynchronousSymbolQuery tracks the symbols that it depends on in order to (1)
detach the query in the event of a failure, and (2) report those dependencies
to clients of the ExecutionSession::lookup method (via the RegisterDependencies
argument). Previously we tracked only dependencies on symbols that didn't meet
the required state (the only symbols that the query needs to be attached to),
but this is insufficient to report all necessary dependencies to lookup clients.
E.g. A lookup requiring SymbolState::Resolved where some matched symbol is
already Resolved but not yet Emitted or Ready would result in the dependency on
that symbol not being reported, which could result in illegal access in
concurrent JIT setups. (This bug was discovered by @mikaoP on discord with a
simple concurrent JIT setup).
This patch tracks and reports all dependencies on symbols that aren't Ready yet,
correcting the under-reporting issue. AsynchronousSymbolQuery::detach is updated
to stop asserting that all depended-upon symbols have a query attached.
ErrorAsOutParameter's Error* constructor supports cases where an Error might not
be passed in (because in the calling context it's known that this call won't
fail). Most clients always have an Error present however, and for them an Error&
overload is more convenient.
Check that we're not reusing any handler tag addresses before installing any
handlers. This ensures that either all of the handlers are installed*, or none
of them are, simplifying error recovery.
* Ignoring handlers whose tags couldn't be resolved at all: these were never
installed.
Dependant lists hold raw pointers back to EDUs that depend on them. We need to
remove these entries before destroying the EDU or we'll be left with a dangling
reference that can result in use-after-free bugs.
No testcase: This has only been observed in multi-threaded setups that
reproduce the issue inconsistently.
rdar://135403614
This reapplies 785d376d123, which was reverted in c49837f5f68 due to bot
failures. The fix was to relax some asserts to allow common symbols to be
resolved with either common or weak flags, rather than requiring one or the
other.
Duplicate common definitions should be coaleseced, rather than being treated as
duplicate definitions. Strong definitions should override common definitions.
rdar://132314264
JITDylib::removeTracker already runs with the session mutex locked (and must do
so), so remove the redundant locking and add an 'IL_' ("inside lock") prefix to
the method name.
Saves several MB of memory in larger applications after linking finishes
by clearing DenseMap storage that is empty. This does not attempt to
shrink partially full materialization infos. The assumption is that
adding more after linking finishes is rare.
rdar://126145336
Same idea as 006aaf32258 -- reduce boilerplate and improve readability. This
time updates will be piecemeal to make it easier to identify errors.
Coding my way home: 2.18555S, 93.78063W
Add an ExecutionSession state verifier, enabled under EXPENSIVE_CHECKS, that can
be used to identify inconsistent session state to assist in tracking down bugs.
This initial version was motivated by investigation of the EDU-update bug that
was fixed in a671ceec334.
rdar://125376708
We were catching a local variable, SymMI, by value instead of by reference
during EDU cleanup and this was leaving the dependence graph in an
inconsistent state that could lead to crashes on subsequent emits. Fixing this
bug required us to also avoid aliasing between SymMI and MI (which would have
caused cleanup to clear the MI.DependantEDUs set that we're iterating over).
No testcase: the crash only triggered in very specific circumstances
(including concurrent linking) in an out-of-tree ORC client. I'm working on a
session state verifier that could be turned on when compiling with
expensive-checks turned on and that should help us catch issues like this in
the future.
rdar://125164262
Coding my way home: 0.89527S, 89.61313W
Removes the MaterializationResponsibility::addDependencies and
addDependenciesForAll methods, and transfers dependency registration to
the notifyEmitted operation. The new dependency registration allows
dependencies to be specified for arbitrary subsets of the
MaterializationResponsibility's symbols (rather than just single symbols
or all symbols) via an array of SymbolDependenceGroups (pairs of symbol
sets and corresponding dependencies for that set).
This patch aims to both improve emission performance and simplify
dependence tracking. By eliminating some states (e.g. symbols having
registered dependencies but not yet being resolved or emitted) we make
some errors impossible by construction, and reduce the number of error
cases that we need to check. NonOwningSymbolStringPtrs are used for
dependence tracking under the session lock, which should reduce
ref-counting operations, and intra-emit dependencies are resolved
outside the session lock, which should provide better performance when
JITing concurrently (since some dependence tracking can happen in
parallel).
The Orc C API is updated to account for this change, with the
LLVMOrcMaterializationResponsibilityNotifyEmitted API being modified and
the LLVMOrcMaterializationResponsibilityAddDependencies and
LLVMOrcMaterializationResponsibilityAddDependenciesForAll operations
being removed.
Access to individual DefinitionGenerators is serialized in order to make
generators easier to implement: serializing access means that tryToGenerate
methods don't have to handle concurrent, potentially overlapping, requests.
Prior to this patch serialization was achieved by having each lookup acquire a
lock on each generator, however this causes the lookup thread to block if the
generator is in use. In the common case where many objects reference some
common library symbol that is provided by a generator this may cause many
threads to block concurrently preventing progress on other work.
This patch changes the model so that lookups are automatically suspended if
they need to use a generator that is already in use, and then automatically
resumed once the generator is free. This is achieved by reusing the lookup
suspension machinery that was introduced in 069919c9ba3 for optionally
asynchronous generators.
The ExecutionSession::removeJITDylibs operation will remove all JITDylibs in
the given list (i.e. first clear them, then remove them from the session).
ExecutionSession::endSession is updated to remove JITDylibs rather than just
clearing them. This prevents new code from being added to any JITDylib once
endSession has been called.
An in-flight materialization may try to claim responsibility for new symbols
(via MaterializationResponsibility::defineMaterializing) after the tracker that
is associated with the materialization is removed, leaving the tracker defunct.
Failure to error out early here could leave the JITDylib in an invalid state,
with defineMaterializing associating new symbols with the already-defunct
tracker. Erroring out early prevents this.
ExecutorAddr was introduced in b8e5f918166 as an eventual replacement for
JITTargetAddress. ExecutorSymbolDef is introduced in this patch as a
replacement for JITEvaluatedSymbol: ExecutorSymbolDef is an (ExecutorAddr,
JITSymbolFlags) pair, where JITEvaluatedSymbol was a (JITTargetAddress,
JITSymbolFlags) pair.
A number of APIs had already migrated from JITTargetAddress to ExecutorAddr,
but many of ORC's internals were still using the older type. This patch aims
to address that.
Some public APIs are affected as well. If you need to migrate your APIs you can
use the following operations:
* ExecutorAddr::toPtr replaces jitTargetAddressToPointer and
jitTargetAddressToFunction.
* ExecutorAddr::fromPtr replace pointerToJITTargetAddress.
* ExecutorAddr(JITTargetAddress) creates an ExecutorAddr value from a
JITTargetAddress.
* ExecutorAddr::getValue() creates a JITTargetAddress value from an
ExecutorAddr.
JITTargetAddress and JITEvaluatedSymbol will remain in JITSymbol.h for now, but
the aim will be to eventually deprecate and remove these types (probably when
MCJIT and RuntimeDyld are deprecated).
Sort symbols before dumping so we get a deterministic order and can check them in tests.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146658
This reverts commit bdf5f9c3228d6ed1d7c6f87b3828a7d573b34c03, which was a work
in progress for https://reviews.llvm.org/D144276 and accidentally committed
early.
Simplifies the process of building an LLJIT instance that supports the native
platform features (initializers, TLV, etc.).
SetUpExecutorNativePlatform can be passed to LLJITBuilder::setPlatformSetUp
method. It takes a reference to the ORC runtime (as a path or an in-memory
archive) and automatically sets the platform for LLJIT's ExecutionSession based
on the executor process's triple.
Differential Revision: https://reviews.llvm.org/D144276
The loop body may add and remove entries in the symbol table so we can't hold
iterators to the entries. This commit updates the method to use the newly added
NonOwningSymbolStringPtr type as keys for removal instead.
Side note: This bug has been present since the introduction of the
defineMaterializing method, but the method is called rarely and DenseMap
resizes are also rare so we didn't see any fallout until a large program was
thrown at it. There's no testcase as I haven't been able to reproduce the
failure with smaller testcases.
In some cases it's helpful to group trackers by JITDylib. E.g. Platform classes
may want to track initializer symbols with a `JITDylib -> Tracker -> [ Symbol ]`
map. This makes it easy to collect all symbols for the JITDylib, while still
allowing efficient removal of a single tracker. Passing the JITDylib as an
argument to ResourceManager::notifyRemovingResources and
ResourceManager::notifyTransferringResources supports such use-cases.
Previously we stripped Weak flags from JITDylib symbol table entries once they
were resolved (there was no particularly good reason for this). Now we want to
retain them and query them when setting the Linkage on external symbols in
LinkGraphs during symbol resolution (this was the motivation for 75404e9ef88).
Making weak linkage of external definitions discoverable in the LinkGraph will
in turn allow future plugins to implement correct handling for them (by
recording locations that depend on exported weak definitions and pointing all
of these at one chosen definition at runtime).
This code previously used cantFail, but both steps (resolution and emission)
can fail if the resource tracker associated with the
AbsoluteSymbolsMaterializationUnit is removed. Checking these errors is
necessary for correct error propagation.
Idiomatic llvm::Error usage can result in a FailedToMaterialize error tearing
down an ExecutionSession instance. Since the FailedToMaterialize error holds
SymbolStringPtrs and JITDylib references this leads to crashes when accessing
or logging the error.
This patch modifies FailedToMaterialize to retain the SymbolStringPool and
JITDylibs involved in the failure so that we can safely report an error message
to the client, even if the error tears down the session.
The contract for JITDylibs allows the getName method to be used even after the
session has been torn down, but no other JITDylib fields should be accessed via
the FailedToMaterialize error if the ssesion has been torn down. Logging the
error is guaranteed to be safe in all cases.
Clients are required to call ExecutionSession::endSession before destroying the
ExecutionSession. Failure to do so can lead to memory leaks and other difficult
to debug issues. Enforcing this requirement by assertion makes it easy to spot
or debug situations where the contract was not followed.
