This reapplies aba6bb0820b, which was reverted in 28e2a891210 due to bot
failures. It contains fixes to silence warnings for uncovered switches,
and for incorrect initializer-symbol handling on ELF and COFF.
SideEffectsOnly is a new jitlink::Scope value that corresponds to the
JITSymbolFlags::MaterializationSideEffectsOnly flag: Symbols with this scope
can be looked up (and form part of the initial interface of a LinkGraph) but
never actually resolve to an address (so can only be looked up with a
WeaklyReferencedSymbol lookup).
Previously ObjectLinkingLayer implicitly treated JITLink symbols as having this
scope, regardless of a Symbol's actual scope, if the
MaterializationSideEffectsOnly flag was set on the corresponding symbol in the
MaterializationResponsibility object. Using an explicit scope in JITLink for
this (1) allows JITLink plugins to identify and correctly handle
side-effects-only symbols, and (2) allows raw LinkGraphs to define
side-effects-only symbols without clients having to manually modify their
`MaterializationUnit::Interface`.
ErrorAsOutParameter's Error* constructor supports cases where an Error might not
be passed in (because in the calling context it's known that this call won't
fail). Most clients always have an Error present however, and for them an Error&
overload is more convenient.
We can get a reference to the `ExecutionSession` from the
`ObjectLinkingLayer` argument, so there's no need to pass it in
separately.
This mirrors recent changes to `ElfNixPlatform` and `MachOPlatform` by
@lhames in
3dba4ca155
and
cc20dd285a.
Instead, when a MaterializationResponsibility contains an initializer symbol,
the Platform classes (MachO, COFF, ELFNix) will now add a defined symbol with
the same name to an arbitary block within the initializer sections, and then
add keep-alive edges from that symbol to all other init section blocks.
ObjectLinkingLayer is updated to automatically discard symbols where the
corresponding MaterializationResponsibility entry has the
MaterializationSideEffecstsOnly flag. This change simplifies both the
ObjectLinkingLayer::Plugin interface and the dependence tracking algorithm,
which no longer needs a special case for "synthetic"
(MaterializationSideEffectsOnly) symbols.
Now that llvm::support::endianness has been renamed to
llvm::endianness, we can use the shorter form. This patch replaces
support::endianness with llvm::endianness.
Now that llvm::support::endianness has been renamed to
llvm::endianness, we can use the shorter form. This patch replaces
support::endianness::{big,little,native} with
llvm::endianness::{big,little,native}.
This reapplies 371cb1af61d, which was reverted in 0b2240eda01 due to bot
failures.
The clang-repl test failure is fixed by dropping the process symbols definition
generator that was manually attached to the main JITDylib, since LLJIT now
exposes process symbols by default. (The bug was triggered when JIT'd code used
the process atexit provided by the generator, rather than the JIT atexit which
has been moved into the platform JITDylib).
Any LLJIT clients that see crashes in static destructors should likewise remove
any process symbol generators attached to their main JITDylib.
This commit includes several related ergonomic improvements to LLJIT.
(1) Adds a default JITDylibSearchOrder to be appended to the initial link order
of JITDylibs created via LLJIT::createJITDylib (dropping any duplicate entries).
This was introduced to support automatic reflection of process symbols (see
(2) below), but has been made visible to clients as it's generically useful,
e.g. if clients have some extra set of libraries that they want to be visible
to JIT'd code by default.
The default JITDylibSearchOrder is only appended to the link order of JITDylibs
created via LLJIT::createJITDylib, and will not be apply to JITDylibs created by
directly calling the underlying ExecutionSession -- in that case clients can set
up the link order manually.
(2) Makes process symbols visible to JIT'd code by default via the new "Process"
JITDylib, which is added to the default link order.
LLJIT clients usually want symbols in the executor process to be accessible to
JIT'd code. Until now clients have been left to set this up themselves by adding
a DynamicLibrarySearchGenerator to the Main JITDylib. This patch adds a new
process symbols JITDylib that will be created by default (with an
EPCDynamicLibrarySearchGenerator attached) and added to the default link order,
making process symbols available to JIT'd code.
Clients who do not want process symbols to be visible to JIT'd code by default
can call setLinkProcessSymbolsByDefault(false) on their LLJITBuilder to disable
this:
LLJITBuilder()
...
.setLinkProcessSymbolsByDefault(false)
...
.create();
Clients can also call setProcessSymbolsJITDylibSetup to take over responsibility
for configuring the process symbols JITDylib (the callback that the client
supplies will be called on the bare process symbols JITDylib immediately after
it is created).
If setLinkProcessSymbolsByDefault(false) is called and no JITDylib setup
callback has been set then the process symbols JITDylib will not be created and
LLJIT::getProcessSymbolsJITDylib will return null.
(3) Adds an ExecutorNativePlatform utility that makes it easier to enable
native platform features.
Some object format features (e.g. native static initializers and thread locals)
require runtime support in the executing process. Support for these features in
ORC is implemented cooperatively between the ORC runtime and the LLVM Platform
subclasses (COFFPlatform, ELFNixPlatform, and MachOPlatform).
ExecutorNativePlatfrom simplifies the process of loading the ORC runtime and
creating the appropriate platform class for the executor process.
ExecutorNativePlatform takes a path to the ORC runtime (or a MemoryBuffer
containing the runtime) and other required runtimes for the executor platform
(e.g. MSVC on Windows) and then configures LLJIT with an appropriate platform
class based on the executor's target triple:
LLJITBuilder()
.setPlatformSetUp(ExecutorNativePlatform("/path/to/orc-runtime.a"));
(The ORC runtime is built as part of compiler-rt, and the exact name of the
archive is platform dependent).
The ORC runtime and platform symbols will be added to a new "Platform" JITDylib,
which will be added to the *front* of the default link order (so JIT'd code will
prefer symbol definitions in the platform/runtime to definitions in the executor
process).
ExecutorNativePlatform assumes that the Process JITDylib is available, as
the ORC runtime may depend on symbols provided by the executor process.
Differential Revision: https://reviews.llvm.org/D144276
ExecutorAddr was introduced in b8e5f918166 as an eventual replacement for
JITTargetAddress. ExecutorSymbolDef is introduced in this patch as a
replacement for JITEvaluatedSymbol: ExecutorSymbolDef is an (ExecutorAddr,
JITSymbolFlags) pair, where JITEvaluatedSymbol was a (JITTargetAddress,
JITSymbolFlags) pair.
A number of APIs had already migrated from JITTargetAddress to ExecutorAddr,
but many of ORC's internals were still using the older type. This patch aims
to address that.
Some public APIs are affected as well. If you need to migrate your APIs you can
use the following operations:
* ExecutorAddr::toPtr replaces jitTargetAddressToPointer and
jitTargetAddressToFunction.
* ExecutorAddr::fromPtr replace pointerToJITTargetAddress.
* ExecutorAddr(JITTargetAddress) creates an ExecutorAddr value from a
JITTargetAddress.
* ExecutorAddr::getValue() creates a JITTargetAddress value from an
ExecutorAddr.
JITTargetAddress and JITEvaluatedSymbol will remain in JITSymbol.h for now, but
the aim will be to eventually deprecate and remove these types (probably when
MCJIT and RuntimeDyld are deprecated).
This reverts commit bdf5f9c3228d6ed1d7c6f87b3828a7d573b34c03, which was a work
in progress for https://reviews.llvm.org/D144276 and accidentally committed
early.
Simplifies the process of building an LLJIT instance that supports the native
platform features (initializers, TLV, etc.).
SetUpExecutorNativePlatform can be passed to LLJITBuilder::setPlatformSetUp
method. It takes a reference to the ORC runtime (as a path or an in-memory
archive) and automatically sets the platform for LLJIT's ExecutionSession based
on the executor process's triple.
Differential Revision: https://reviews.llvm.org/D144276
Testing the COFFPlatform on MSVC, a name shadowing issue surfaced where `LoadDynLibrary` inside the constructor was actually using the moved-from function argument. This patch simply renames the argument to avoid that shadowing.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D145710
Renames the existing allocateString method to allocateContent and adds a pair of
allocateCString methods.
The previous allocateString method did not include a null-terminator. It behaved
the same as allocateContent except with a Twine input, rather than an
ArrayRef<char>. Renaming allocateString to allocateBuffer (overloading the
existing method) makes this clearer.
The new allocateCString methods allocate the given content plus a
null-terminator character, and return a buffer covering both the string and
null-terminator. This makes them suitable for creating c-string content for
jitlink::Blocks.
Existing users of the old allocateString method have been updated to use the
new allocateContent overload.
Removes public bootstrap method that is not really necessary and not consistent with other platform API.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D132780
Supports dynamic VC runtime. It implements atexits handling which is required to load msvcrt.lib successfully. (the object file containing atexit symbol somehow resolves to static vc runtim symbols) It also default to dynamic vc runtime which tends to be more robust.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D132525
Initial platform support for COFF/x86_64.
Completed features:
* Statically linked orc runtime.
* Full linking/initialization of static/dynamic vc runtimes and microsoft stl libraries.
* SEH exception handling.
* Full static initializers support
* dlfns
* JIT side symbol lookup/dispatch
Things to note:
* It uses vc runtime libraries found in vc toolchain installations.
* Bootstrapping state is separated because when statically linking orc runtime it needs microsoft stl functions to initialize the orc runtime, but static initializers need to be ran in order to fully initialize stl libraries.
* Process symbols can't be used blidnly on msvc platform; otherwise duplicate definition error gets generated. If process symbols are used, it's destined to get out-of-reach error at some point.
* Atexit currently not handled -- will be handled in the follow-up patches.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130479
