2717 Commits

Author SHA1 Message Date
Lang Hames
0f5b70769d [llvm-jitlink] Add -phony-externals option to suppress unresolved externals.
The -phony-externals option adds a generator which explicitly defines any
otherwise unresolved externals as null. This transforms link-time
unresolved-symbol errors into potential runtime null pointer accesses
(if an unresolved external is actually accessed during execution).

This option can be useful in -harness mode to avoid having to mock a
large number of symbols that are not reachable at runtime (e.g. unused
methods referenced by a class vtable).
2020-08-01 18:33:44 -07:00
Hans Wennborg
6a3b07a4bf RuntimeDyldELF: report_fatal_error instead of asserting for unimplemented relocations (PR46816)
This fixes the ExecutionEngine/MCJIT/stubs-sm-pic.ll test in no-asserts
builds which is set to XFAIL on some platforms like 32-bit x86. More
importantly, we probably don't want to silently error in these cases.

Differential revision: https://reviews.llvm.org/D84390
2020-07-31 20:06:47 +02:00
Lang Hames
8ce8cee1e1 [llvm-jitlink] Add -harness option to llvm-jitlink.
The -harness option enables new testing use-cases for llvm-jitlink. It takes a
list of objects to treat as a test harness for any regular objects passed to
llvm-jitlink.

If any files are passed using the -harness option then the following
transformations are applied to all other files:

  (1) Symbols definitions that are referenced by the harness files are promoted
      to default scope. (This enables access to statics from test harness).

  (2) Symbols definitions that clash with definitions in the harness files are
      deleted. (This enables interposition by test harness).

  (3) All other definitions in regular files are demoted to local scope.
      (This causes untested code to be dead stripped, reducing memory cost and
      eliminating spurious unresolved symbol errors from untested code).

These transformations allow the harness files to reference and interpose
symbols in the regular object files, which can be used to support execution
tests (including fuzz tests) of functions in relocatable objects produced by a
build.
2020-07-30 15:26:19 -07:00
Lang Hames
9f1dcdca71 [JITLink] Allow JITLinkContext::notifyResolved to return an Error.
This allows clients to detect invalid transformations applied by JITLink passes
(e.g. inserting or removing symbols in unexpected ways) and terminate linking
with an error.

This change is used to simplify the error propagation logic in
ObjectLinkingLayer.
2020-07-30 15:26:18 -07:00
Lang Hames
47a40eda17 [ORC] Remove a redundant call to getTargetMemory. 2020-07-26 17:34:31 -07:00
Lang Hames
a01c4ee71c [ORC] Rename TargetProcessControl DynamicLibraryHandle and loadLibrary.
The new names, DylibHandle and loadDylib, are more concise and make
clear that these utilities are for loading dynamic libraries, not static
ones.
2020-07-25 15:21:43 -07:00
Lang Hames
11d5316afd [ORC] Don't require PageSize or Triple during TargetProcessControl construction
Subclasses will commonly gather that information from a remote during
construction, in which case they won't have meaningful values to pass to
TargetProcessControl's constructor.
2020-07-25 15:21:43 -07:00
Lang Hames
69091eb1c4 [ORC] Enable use of TargetProcessControl::getMemMgr with ObjectLinkingLayer.
This patch makes ownership of the JITLinkMemoryManager by ObjectLinkingLayer
optional: the layer can still own the memory manager but no longer has to.

Evevntually we want to move to a state where ObjectLinkingLayer never owns its
memory manager. For now allowing optional ownership makes it easier to develop
classes that can dynamically use either RTDyldObjectLinkingLayer, which owns
its memory managers, or ObjectLinkingLayer (e.g. LLJIT).
2020-07-23 16:18:57 -07:00
Simon Pilgrim
f758d72eb8 Speculation.h - remove unnecessary includes. NFC. 2020-07-23 11:58:23 +01:00
Lang Hames
13ad00be98 [ORC] Add a TargetProcessControl-based dynamic library search generator.
TPCDynamicLibrarySearchGenerator uses a TargetProcessControl instance to
load libraries and search for symbol addresses in a target process. It
can be used in place of a DynamicLibrarySearchGenerator to enable
target-process agnostic lookup.
2020-07-22 16:19:24 -07:00
Lang Hames
574713c307 [ExecutionEngine] Initialize near block hint in SectionMemoryManager.
When allocating a new memory block in SectionMemoryManager, initialize
the Near hint for the other memory groups if they have not been set
already.

Patch by Dana Koch. Thanks Dana!
2020-07-20 14:40:54 -07:00
Lang Hames
0d944e00ea [ORC] Refactor TrampolinePool to reduce virtual function calls.
Virtual function calls are now only made when the pool needs to be
grown to accommodate o new request.
2020-07-19 22:38:41 -07:00
Lang Hames
cdd10ca280 [JITLink][MachO] Tidy up debugging output for relocation parsing.
Identify relocations by (section name, offset) pairs, rather than plain
vmaddrs. This makes it easier to cross-reference debugging output for
relocations with output from standard object inspection tools (otool,
readelf, objdump, etc.).
2020-07-19 19:45:50 -07:00
Lang Hames
f7a571537a [JITLink][MachO] Fix handling of non-extern UNSIGNED pair of SUBTRACTOR relocs.
When processing a MachO SUBTRACTOR/UNSIGNED pair, if the UNSIGNED target
is non-extern then check the r_symbolnum field of the relocation to find
the targeted section and use the section's address to find 'ToSymbol'.

Previously 'ToSymbol' was found by loading the initial value stored at
the fixup location and treating this as an address to search for. This
is incorrect, however: the initial value includes the addend and will
point to the wrong block if the addend is less than zero or greater than
the block size.

rdar://65756694
2020-07-19 10:22:55 -07:00
Logan Smith
3ee7fe4cfd [llvm][NFC] Add missing 'override's 2020-07-17 17:35:59 -07:00
David Tenty
8dea7f3202 [z/OS][AIX] Move lambda definition to fix build problem
This is a follow on change to eed19bd8 and contains a fix for a build
failure that occurs on both z/OS and AIX as a result of this commit:

https://reviews.llvm.org/rG670915094462d831e3733e5b01a76471b8cf6dd8.
2020-07-17 10:08:01 -04:00
Benjamin Kramer
9a0689e072 Make helpers static. NFC. 2020-07-17 13:49:11 +02:00
Lang Hames
10056238ac [ORC] Switch from initializer lists to named arguments to work around MSVC.
MSVC doesn't like some of the initializer list uses in 0e940d55f8a.
Switch to named arguments to work around this.
2020-07-16 15:58:31 -07:00
Lang Hames
b0bc77380d [ORC] Add more explicit casts to fix a narrowing conversion errors. 2020-07-16 15:37:18 -07:00
Lang Hames
121302ac62 [ORC] Add explicit cast to fix a narrowing conversion error. 2020-07-16 15:33:02 -07:00
Lang Hames
0e940d55f8 [ORC] Add TargetProcessControl and TPCIndirectionUtils APIs.
TargetProcessControl is a new API for communicating with JIT target processes.
It supports memory allocation and access, and inspection of some process
properties, e.g. the target proces triple and page size.

Centralizing these APIs allows utilities written against TargetProcessControl
to remain independent of the communication procotol with the target process
(which may be direct memory access/allocation for in-process JITing, or may
involve some form of IPC or RPC).

An initial set of TargetProcessControl-based utilities for lazy compilation is
provided by the TPCIndirectionUtils class.

An initial implementation of TargetProcessControl for in-process JITing
is provided by the SelfTargetProcessControl class.

An example program showing how the APIs can be used is provided in
llvm/examples/OrcV2Examples/LLJITWithTargetProcessControl.
2020-07-16 15:09:13 -07:00
Jared Wyles
ef454c5444 [jitlink] Adding support for PCRel32GOTLoad in ELF x86 for the jitlinker
Summary: This adds the basic support for GOT in elf x86.
Was able to just get away using the macho code by generalising the edges.
There will be a follow up patch to turn that into a generic utility for both of the x86 and Mach-O code.

This patch also lands support for relocations relative to symbol.

Reviewers: lhames

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83748
2020-07-16 07:15:08 +10:00
Abhina Sreeskantharajan
eed19bd844 [NFC] [AIX] [z/OS] Fix build failure on AIX and z/OS
Summary: This PR contains a build failure fix that occurs on both AIX and z/OS as a result of this commit https://reviews.llvm.org/rG670915094462d831e3733e5b01a76471b8cf6dd8.

Reviewers: uweigand, Kai, hubert.reinterpretcast, daltenty, lhames

Reviewed By: Kai, hubert.reinterpretcast, daltenty

Subscribers: SeanP, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83889
2020-07-15 16:07:34 -04:00
Richard Smith
71292379d7 Make LazyCallThroughManager destructor virtual rather than arranging for
all clients to delete it via the correct dynamic type.
2020-07-14 18:41:07 -07:00
Richard Smith
099fd37484 Fix undefined behavior due to deleting an object with a non-virtual
destructor via a pointer of the wrong static type.

This caused crashes during deallocation in C++14 builds when using a
deallocator whose sized delete requires the size argument to be correct.

Also make the LazyCallThroughManager destructor protected to catch this
sort of bug in the future.
2020-07-14 17:17:20 -07:00
Lang Hames
3d931e85f1 [ORC] Don't take ownership of the trampoline pool in LazyReexportsManager.
LazyReexportsManager instances use the trampoline pool, but they don't need to
own it. Keeping TrampolinePool ownership separate allows re-use of the
trampoline pool by other clients.
2020-07-14 10:56:45 -07:00
Lang Hames
31b5b12129 [ORC] Remove a spurious reinterpret_cast. 2020-07-13 12:39:24 -07:00
Lang Hames
fb7ef0bb0b [ORC] Generalize emit re-entry, stub, etc. APIs for working addr != link addr.
This patch generalizes the APIs for writing re-entry blocks, trampolines and
stubs to allow their final linked address to differ from the address of
their initial working memory. This will allow these routines to be used with
JITLinkMemoryManagers, which will in turn allow for unification of code paths
for in-process and cross-process lazy JITing.
2020-07-13 10:15:10 -07:00
Lang Hames
6709150944 [ORC] Modify LazyCallThroughManager to support asynchronous resolution.
Asynchronous resolution is a better fit for handling reentry over
IPC/RPC where we want to avoid blocking a communication handler/thread.
2020-07-08 21:13:55 -07:00
Christopher Tetreault
9b500e564a [SVE] Remove calls to VectorType::getNumElements from ExecutionEngine
Reviewers: efriedma, lhames, sdesmalen, fpetrogalli

Reviewed By: lhames, sdesmalen

Subscribers: tschuett, hiraditya, rkruppe, psnobl, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82211
2020-06-30 11:05:38 -07:00
Guillaume Chatelet
368a5e3a66 [Alignment][NFC] migrate DataLayout::getPreferredAlignment
This patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Differential Revision: https://reviews.llvm.org/D82752
2020-06-29 11:24:36 +00:00
Lang Hames
bf783a6aa8 [JITLink] Display host -> target address mapping in debugging output.
This can be helpful for sanity checking JITLink memory manager behavior.
2020-06-19 10:05:02 -07:00
Lang Hames
5682f192bd [RuntimeDyld] Add dependence on Core.
Commit 498dd745f55 introduced a dependence on Core. This patch updates
LLVMbuild.txt to reflect this.
2020-06-15 11:14:27 -07:00
Lang Hames
498dd745f5 [ORC] Honor linker private global prefix on symbol names.
If a symbol name begins with the linker private global prefix (as
described by the DataLayout) then it should be treated as non-exported,
regardless of its LLVM IR visibility value.
2020-06-15 10:28:36 -07:00
Leonard Chan
6adc664b9d [llvm][ELF][AArch64] Handle R_AARCH64_PLT32 relocation
This patch allows for usage of the @PLT modifier in AArch64 assembly which
lowers to an R_AARCH64_PLT32 relocation. See D81184 for handling this
relocation in lld.

Differential Revision: https://reviews.llvm.org/D81446
2020-06-10 11:34:16 -07:00
Lang Hames
e429cffd4f [ORC] Always use ObjectLinkingLayer/JITLink for MachO on x86-64 and arm64.
JITLink supports all code and relocation models, so there's no reason to
conditionalize using JITLink on the code or relocation model settings.

Clients wanting to use RTDyldObjectLinkingLayer/RuntimeDyld will now
need to use a custom object linking layer creator.
2020-06-05 21:02:29 -07:00
Lang Hames
d27cbf03cf [JITLink] Skip debug sections in MachO objects.
Debug sections will not be linked into the final executable and may contain
ambiguous relocations*. Skipping them avoids both some unnecessary processing
cost and the hassle of dealing with the problematic relocations.

* E.g. __debug_ranges contains non-extern relocations to the end of functions
hat begin with named symbols. Under the usual rules for interpreting non-extern
relocations these will be incorrectly associated with the following block, or
no block at all (if there is a gap between one block and the next).
2020-06-03 11:08:14 -07:00
Jared Wyles
3f0841f6d0 [jitlink] R_X86_64_PC32 support for the elf x86 jitlinker
Summary:

Adding in our first relocation type, and all the required plumbing to support the rest in following patches

Differential Revision: https://reviews.llvm.org/D80613

Reviewer: lhames
2020-05-30 10:53:18 +10:00
Lang Hames
ff92d3c672 [ORC] Add debugging output for LLJIT construction.
This can be handy for checking whether the LLJIT instance you're constructing
matches your expectations.
2020-05-28 20:31:50 -07:00
Lang Hames
a6deaeec37 [JITLink] Improve llvm-jitlink regression testing support for ELF.
This patch adds a jitlink pass, 'registerELFGraphInfo', that records section
and symbol information about each LinkGraph in the llvm-jitlink session object.
This allows symbols and sections to be referred to by name in llvm-jitlink
regression tests. This will enable a testcase to be written for
https://reviews.llvm.org/D80613.
2020-05-28 20:31:50 -07:00
David Blaikie
fca76b7945 Roll variables into an LLVM_DEBUG block to address -Wunused-but-set-variable 2020-05-26 12:05:08 -07:00
Simon Pilgrim
c9797cf98b Fix msvc "switch statement contains 'default' but no 'case' labels" warning. NFC.
Stripped out the switch statement, but kept case labels as comments for future reference.
2020-05-22 10:49:10 +01:00
Lang Hames
2e40cf06df [JITLink] Initial implementation of ELF / x86-64 support for JITLink.
This initial implementation supports section and symbol parsing, but no
relocation support. It enables JITLink to link and execute ELF relocatable
objects that do not require relocations.

Patch by Jared Wyles. Thanks Jared!

Differential Revision: https://reviews.llvm.org/D79832
2020-05-21 21:44:00 -07:00
Lang Hames
41379f1ec4 [ORC] Share ownership of JITDylibs between ExecutionSession and
MaterializationResponsibility.

MaterializationResponsibility objects provide a connection between a
materialization process (compiler, jit linker, etc.) and the JIT state held in
the ExecutionSession and JITDylib objects. Switching to shared ownership
extends the lifetime of JITDylibs to ensure they remain accessible until all
materializers targeting them have completed. This will allow (in a follow-up
patch) JITDylibs to be removed from the ExecutionSession and placed in a
pending-destruction state while they are kept alive to communicate errors
to/from any still-runnning materialization processes. The intent is to enable
JITDylibs to be safely removed even if they have running compiles targeting
them.
2020-05-10 16:37:17 -07:00
Lang Hames
c66f89005f [ORC] Rename SearchOrder operations on JITDylib to LinkOrder.
Refering to the link order of a dylib better matches the terminology used in
static compilation. As upcoming patches will increase the number of places where
link order matters (for example when closing JITDylibs) it's better to get this
name change out of the way early.
2020-05-04 16:47:52 -07:00
Lang Hames
990eb94467 [JITLink] Fix missing ADDITIONAL_HEADER_DIRS for the LLVMJITLink component.
Also adds target_link_libraries.

Thanks to @jcmac for spotting this.
2020-05-04 12:58:09 -07:00
Lang Hames
7ac0c3b475 [JITLink] Fix a typo in a comment.
Thanks to @jcmac for pointing this out.
2020-05-04 12:58:09 -07:00
Sam McCall
d10c995b4d std::isspace -> llvm::isSpace (where locale should be ignored)
I've left out some cases where I wasn't totally sure this was right or
whether the include was ok (compiler-rt) or idiomatic (flang).
2020-05-02 15:36:04 +02:00
Xing GUO
ff6a0b6a8e [Object] Change ObjectFile::getSymbolValue() return type to Expected<uint64_t>
Summary:
In D77860, we have changed `getSymbolFlags()` return type to `Expected<uint32_t>`.
This change helps bubble the error further up the stack.

Reviewers: jhenderson, grimar, JDevlieghere, MaskRay

Reviewed By: jhenderson

Subscribers: hiraditya, MaskRay, rupprecht, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79075
2020-05-02 14:04:44 +08:00
Craig Topper
a58b62b4a2 [IR] Replace all uses of CallBase::getCalledValue() with getCalledOperand().
This method has been commented as deprecated for a while. Remove
it and replace all uses with the equivalent getCalledOperand().

I also made a few cleanups in here. For example, to removes use
of getElementType on a pointer when we could just use getFunctionType
from the call.

Differential Revision: https://reviews.llvm.org/D78882
2020-04-27 22:17:03 -07:00