Saves several MB of memory in larger applications after linking finishes
by clearing DenseMap storage that is empty. This does not attempt to
shrink partially full materialization infos. The assumption is that
adding more after linking finishes is rare.
rdar://126145336
Same idea as 006aaf32258 -- reduce boilerplate and improve readability. This
time updates will be piecemeal to make it easier to identify errors.
Coding my way home: 2.18555S, 93.78063W
Add an ExecutionSession state verifier, enabled under EXPENSIVE_CHECKS, that can
be used to identify inconsistent session state to assist in tracking down bugs.
This initial version was motivated by investigation of the EDU-update bug that
was fixed in a671ceec334.
rdar://125376708
We were catching a local variable, SymMI, by value instead of by reference
during EDU cleanup and this was leaving the dependence graph in an
inconsistent state that could lead to crashes on subsequent emits. Fixing this
bug required us to also avoid aliasing between SymMI and MI (which would have
caused cleanup to clear the MI.DependantEDUs set that we're iterating over).
No testcase: the crash only triggered in very specific circumstances
(including concurrent linking) in an out-of-tree ORC client. I'm working on a
session state verifier that could be turned on when compiling with
expensive-checks turned on and that should help us catch issues like this in
the future.
rdar://125164262
Coding my way home: 0.89527S, 89.61313W
Removes the MaterializationResponsibility::addDependencies and
addDependenciesForAll methods, and transfers dependency registration to
the notifyEmitted operation. The new dependency registration allows
dependencies to be specified for arbitrary subsets of the
MaterializationResponsibility's symbols (rather than just single symbols
or all symbols) via an array of SymbolDependenceGroups (pairs of symbol
sets and corresponding dependencies for that set).
This patch aims to both improve emission performance and simplify
dependence tracking. By eliminating some states (e.g. symbols having
registered dependencies but not yet being resolved or emitted) we make
some errors impossible by construction, and reduce the number of error
cases that we need to check. NonOwningSymbolStringPtrs are used for
dependence tracking under the session lock, which should reduce
ref-counting operations, and intra-emit dependencies are resolved
outside the session lock, which should provide better performance when
JITing concurrently (since some dependence tracking can happen in
parallel).
The Orc C API is updated to account for this change, with the
LLVMOrcMaterializationResponsibilityNotifyEmitted API being modified and
the LLVMOrcMaterializationResponsibilityAddDependencies and
LLVMOrcMaterializationResponsibilityAddDependenciesForAll operations
being removed.
Access to individual DefinitionGenerators is serialized in order to make
generators easier to implement: serializing access means that tryToGenerate
methods don't have to handle concurrent, potentially overlapping, requests.
Prior to this patch serialization was achieved by having each lookup acquire a
lock on each generator, however this causes the lookup thread to block if the
generator is in use. In the common case where many objects reference some
common library symbol that is provided by a generator this may cause many
threads to block concurrently preventing progress on other work.
This patch changes the model so that lookups are automatically suspended if
they need to use a generator that is already in use, and then automatically
resumed once the generator is free. This is achieved by reusing the lookup
suspension machinery that was introduced in 069919c9ba3 for optionally
asynchronous generators.
The ExecutionSession::removeJITDylibs operation will remove all JITDylibs in
the given list (i.e. first clear them, then remove them from the session).
ExecutionSession::endSession is updated to remove JITDylibs rather than just
clearing them. This prevents new code from being added to any JITDylib once
endSession has been called.
An in-flight materialization may try to claim responsibility for new symbols
(via MaterializationResponsibility::defineMaterializing) after the tracker that
is associated with the materialization is removed, leaving the tracker defunct.
Failure to error out early here could leave the JITDylib in an invalid state,
with defineMaterializing associating new symbols with the already-defunct
tracker. Erroring out early prevents this.
ExecutorAddr was introduced in b8e5f918166 as an eventual replacement for
JITTargetAddress. ExecutorSymbolDef is introduced in this patch as a
replacement for JITEvaluatedSymbol: ExecutorSymbolDef is an (ExecutorAddr,
JITSymbolFlags) pair, where JITEvaluatedSymbol was a (JITTargetAddress,
JITSymbolFlags) pair.
A number of APIs had already migrated from JITTargetAddress to ExecutorAddr,
but many of ORC's internals were still using the older type. This patch aims
to address that.
Some public APIs are affected as well. If you need to migrate your APIs you can
use the following operations:
* ExecutorAddr::toPtr replaces jitTargetAddressToPointer and
jitTargetAddressToFunction.
* ExecutorAddr::fromPtr replace pointerToJITTargetAddress.
* ExecutorAddr(JITTargetAddress) creates an ExecutorAddr value from a
JITTargetAddress.
* ExecutorAddr::getValue() creates a JITTargetAddress value from an
ExecutorAddr.
JITTargetAddress and JITEvaluatedSymbol will remain in JITSymbol.h for now, but
the aim will be to eventually deprecate and remove these types (probably when
MCJIT and RuntimeDyld are deprecated).
Sort symbols before dumping so we get a deterministic order and can check them in tests.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146658
This reverts commit bdf5f9c3228d6ed1d7c6f87b3828a7d573b34c03, which was a work
in progress for https://reviews.llvm.org/D144276 and accidentally committed
early.
Simplifies the process of building an LLJIT instance that supports the native
platform features (initializers, TLV, etc.).
SetUpExecutorNativePlatform can be passed to LLJITBuilder::setPlatformSetUp
method. It takes a reference to the ORC runtime (as a path or an in-memory
archive) and automatically sets the platform for LLJIT's ExecutionSession based
on the executor process's triple.
Differential Revision: https://reviews.llvm.org/D144276
The loop body may add and remove entries in the symbol table so we can't hold
iterators to the entries. This commit updates the method to use the newly added
NonOwningSymbolStringPtr type as keys for removal instead.
Side note: This bug has been present since the introduction of the
defineMaterializing method, but the method is called rarely and DenseMap
resizes are also rare so we didn't see any fallout until a large program was
thrown at it. There's no testcase as I haven't been able to reproduce the
failure with smaller testcases.
In some cases it's helpful to group trackers by JITDylib. E.g. Platform classes
may want to track initializer symbols with a `JITDylib -> Tracker -> [ Symbol ]`
map. This makes it easy to collect all symbols for the JITDylib, while still
allowing efficient removal of a single tracker. Passing the JITDylib as an
argument to ResourceManager::notifyRemovingResources and
ResourceManager::notifyTransferringResources supports such use-cases.
Previously we stripped Weak flags from JITDylib symbol table entries once they
were resolved (there was no particularly good reason for this). Now we want to
retain them and query them when setting the Linkage on external symbols in
LinkGraphs during symbol resolution (this was the motivation for 75404e9ef88).
Making weak linkage of external definitions discoverable in the LinkGraph will
in turn allow future plugins to implement correct handling for them (by
recording locations that depend on exported weak definitions and pointing all
of these at one chosen definition at runtime).
This code previously used cantFail, but both steps (resolution and emission)
can fail if the resource tracker associated with the
AbsoluteSymbolsMaterializationUnit is removed. Checking these errors is
necessary for correct error propagation.
Idiomatic llvm::Error usage can result in a FailedToMaterialize error tearing
down an ExecutionSession instance. Since the FailedToMaterialize error holds
SymbolStringPtrs and JITDylib references this leads to crashes when accessing
or logging the error.
This patch modifies FailedToMaterialize to retain the SymbolStringPool and
JITDylibs involved in the failure so that we can safely report an error message
to the client, even if the error tears down the session.
The contract for JITDylibs allows the getName method to be used even after the
session has been torn down, but no other JITDylib fields should be accessed via
the FailedToMaterialize error if the ssesion has been torn down. Logging the
error is guaranteed to be safe in all cases.
Clients are required to call ExecutionSession::endSession before destroying the
ExecutionSession. Failure to do so can lead to memory leaks and other difficult
to debug issues. Enforcing this requirement by assertion makes it easy to spot
or debug situations where the contract was not followed.
This patch updates the MachO platform (both the ORC MachOPlatform class and the
ORC-Runtime macho_platform.* files) to use allocation actions, rather than EPC
calls, to transfer the initializer information scraped from each linked object.
Interactions between the ORC and ORC-Runtime sides of the platform are
substantially redesigned to accomodate the change.
The high-level changes in this patch are:
1. The MachOPlatform::setupJITDylib method now calls into the runtime to set up
a dylib name <-> header mapping, and a dylib state object (JITDylibState).
2. The MachOPlatformPlugin builds an allocation action that calls the
__orc_rt_macho_register_object_platform_sections and
__orc_rt_macho_deregister_object_platform_sections functions in the runtime
to register the address ranges for all "interesting" sections in the object
being allocated (TLS data sections, initializers, language runtime metadata
sections, etc.).
3. The MachOPlatform::rt_getInitializers method (the entry point in the
controller for requests from the runtime for initializer information) is
replaced by MachOPlatform::rt_pushInitializers. The former returned a data
structure containing the "interesting" section address ranges, but these are
now handled by __orc_rt_macho_register_object_platform_sections. The new
rt_pushInitializers method first issues a lookup to trigger materialization
of the "interesting" sections, then returns the dylib dependence tree rooted
at the requested dylib for dlopen to consume. (The dylib dependence tree is
returned by rt_pushInitializers, rather than being handled by some dedicated
call, because rt_pushInitializers can alter the dependence tree).
The advantage of these changes (beyond the performance advantages of using
allocation actions) is that it moves more information about the materialized
portions of the JITDylib into the executor. This tends to make the runtime
easier to reason about, e.g. the implementation of dlopen in the runtime is now
recursive, rather than relying on recursive calls in the controller to build a
linear data structure for consumption by the runtime. This change can also make
some operations more efficient, e.g. JITDylibs can be dlclosed and then
re-dlopened without having to pull all initializers over from the controller
again.
In addition to the high-level changes, there are some low-level changes to ORC
and the runtime:
* In ORC, at ExecutionSession teardown time JITDylibs are now destroyed in
reverse creation order. This is on the assumption that the ORC runtime will be
loaded into an earlier dylib that will be used by later JITDylibs. This is a
short-term solution to crashes that arose during testing when the runtime was
torn down before its users. Longer term we will likely destroy dylibs in
dependence order.
* toSPSSerializable(Expected<T> E) is updated to explicitly initialize the T
value, allowing it to be used by Ts that have explicit constructors.
* The ORC runtime now (1) attempts to track ref-counts, and (2) distinguishes
not-yet-processed "interesting" sections from previously processed ones. (1)
is necessary for standard dlopen/dlclose emulation. (2) is intended as a step
towards better REPL support -- it should enable future runtime calls that
run only newly registered initializers ("dlopen_more", "dlopen_additions",
...?).
Calls to JITDylib's getDFSLinkOrder and getReverseDFSLinkOrder methods (both
static an non-static versions) are now valid to make on defunct JITDylibs, but
will return an error if any JITDylib in the link order is defunct.
This means that platforms can safely lookup link orders by name in response to
jit-dlopen calls from the ORC runtime, even if the call names a defunct
JITDylib -- the call will just fail with an error.
This is a counterpart to Platform::setupJITDylib, and is called when JITDylib
instances are removed (via ExecutionSession::removeJITDylib).
Upcoming MachOPlatform patches will use this to clear per-JITDylib data when
JITDylibs are removed.
If all symbols in a lookup match before we reach the end of the search order
then bail out of the search-order loop early.
This should reduce unnecessary contention on the session lock and improve
readability of the debug logs.
MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.
Having a type for this will make functions that build materializer interfaces
more readable and maintainable.
This allows JITDylibs to be removed from the ExecutionSession. Calling
ExecutionSession::removeJITDylib will disconnect the JITDylib from the
ExecutionSession and clear it (removing all trackers associated with it). The
JITDylib object will then be destroyed as soon as the last JITDylibSP pointing
at it is destroyed.
GeneratorsMutex should prevent lookups from proceeding through the
generators of a single JITDylib concurrently (since this could
result in redundant attempts to generate definitions). Mutation of
the generators list itself should be done under the session lock.
This keeps the tracker alive for the lifetime of the MR. This is needed so that
we can check whether the tracker has become defunct before posting results (or
failure) for the MR.
The callWrapperAsync and callSPSWrapperAsync methods take a handler object
that is run on the return value of the call when it is ready. The new RunPolicy
parameters allow clients to control how these handlers are run. If no policy is
specified then the handler will be packaged as a GenericNamedTask and dispatched
using the ExecutorProcessControl's TaskDispatch member. Callers can use the
ExecutorProcessControl::RunInPlace policy to cause the handler to be run
directly instead, which may be preferrable for simple handlers, or they can
write their own policy object (e.g. to dispatch as some other kind of Task,
rather than GenericNamedTask).
ExecutorProcessControl objects will now have a TaskDispatcher member which
should be used to dispatch work (in particular, handling incoming packets in
the implementation of remote EPC implementations like SimpleRemoteEPC).
The GenericNamedTask template can be used to wrap function objects that are
callable as 'void()' (along with an optional name to describe the task).
The makeGenericNamedTask functions can be used to create GenericNamedTask
instances without having to name the function object type.
In a future patch ExecutionSession will be updated to use the
ExecutorProcessControl's dispatcher, instead of its DispatchTaskFunction.
