2956 Commits

Author SHA1 Message Date
Kazu Hirata
eb198f4c3c [llvm] Use llvm::any_of (NFC) 2021-01-04 11:42:47 -08:00
Lang Hames
a554cd6ae5 [RuntimeDyld] Fix dangling reference in RuntimeDyldELF.
Patch by Moritz Sichert. Thanks Moritz!

Differential Revision: https://reviews.llvm.org/D89373
2021-01-03 10:20:36 +11:00
Brandon Bergren
8f004471c2 [PowerPC] Add the LLVM triple for powerpcle [1/5]
Add a triple for powerpcle-*-*.

This is a little-endian encoding of the 32-bit PowerPC ABI, useful in certain niche situations:

1) A loader such as the FreeBSD loader which will be loading a little endian kernel. This is required for PowerPC64LE to load properly in pseries VMs.
Such a loader is implemented as a freestanding ELF32 LSB binary.

2) Userspace emulation of a 32-bit LE architecture such as x86 on 64-bit hosts such as PowerPC64LE with tools like box86 requires having a 32-bit LE toolchain and library set, as they operate by translating only the main binary and switching to native code when making library calls.

3) The Void Linux for PowerPC project is experimenting with running an entire powerpcle userland.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D93918
2021-01-02 12:17:22 -06:00
Lang Hames
5efc71e119 [ORC] Move Orc RPC code into Shared, rename some RPC types.
Moves all headers from Orc/RPC to Orc/Shared, and from the llvm::orc::rpc
namespace into llvm::orc::shared. Also renames RPCTypeName to
SerializationTypeName and Function to RPCFunction.

In addition to being a more reasonable home for this code, this will make it
easier for the upcoming Orc runtime to re-use the Serialization system for
creating and parsing wrapper-function binary blobs.
2020-12-30 12:48:20 +11:00
Georgii Rymar
438bc157a4 [libObject] - Add more ELF types to LLVM_ELF_IMPORT_TYPES_ELFT define (ELFTypes.h).
This allows to get rid of lots for typedefs/usings from many places.

Differential revision: https://reviews.llvm.org/D93801
2020-12-25 11:39:05 +03:00
Kazu Hirata
9939cf5a56 [ExecutionEngine, Linker] Use erase_if (NFC) 2020-12-23 21:44:39 -08:00
Kazu Hirata
ce94e7d867 [MCA, ExecutionEngine, Object] Use llvm::is_contained (NFC) 2020-12-18 09:09:04 -08:00
Lang Hames
ec6b71df70 [JITLink][ORC] Enable creation / linking of raw jitlink::LinkGraphs.
Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is updated to support emission
of raw LinkGraphs in addition to object buffers.

Raw LinkGraphs can be created by in-memory compilers to bypass object encoding /
decoding (though this prevents caching, as LinkGraphs have do not have an
on-disk representation), and by utility code to add programatically generated
data structures to the JIT target process.
2020-12-16 14:01:50 +11:00
Lang Hames
04795ab836 Re-apply 8904ee8ac7e with missing header included this time. 2020-12-14 13:39:33 +11:00
Nico Weber
5b112bcc0d Revert "[JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs."
This reverts commit 8904ee8ac7ebcc50a60de0914abc6862e28b6664.
Didn't `git add` llvm/ExecutionEngine/JITLink/JITLinkDylib.h and hence doesn't
build anywhere.
2020-12-13 21:30:38 -05:00
Lang Hames
8904ee8ac7 [JITLink] Add JITLinkDylib type, thread through JITLinkMemoryManager APIs.
JITLinkDylib represents a target dylib for a JITLink link. By representing this
explicitly we can:
  - Enable JITLinkMemoryManagers to manage allocations on a per-dylib basis
    (e.g by maintaining a seperate allocation pool for each JITLinkDylib).
  - Enable new features and diagnostics that require information about the
    target dylib (not implemented in this patch).
2020-12-14 12:29:16 +11:00
Lang Hames
0207de0bfe [ORC] Prefer preincrement on iterator. 2020-12-14 12:00:21 +11:00
Lang Hames
5bb28fa0f5 [JITLink][ELF] Reformat/add debug logging in ELF_x86_64.cpp.
Moves symbol name to the end of the output and makes other columns fixed width
so that they line up.
2020-12-10 18:46:44 +11:00
Alexey Bader
be9b4bbdfc [MCJIT] Add cmake variables to customize ittapi git location and revision.
To support llorg builds this patch provides the following changes:

1)  Added cmake variable ITTAPI_GIT_REPOSITORY to control the location of ITTAPI repository.
     Default value of ITTAPI_GIT_REPOSITORY is github location: https://github.com/intel/ittapi.git
     Also, the separate cmake variable ITTAPI_GIT_TAG was added for repo tag.
2)  Added cmake variable ITTAPI_SOURCE_DIR to control the place where the repo will be cloned.
     Default value of ITTAPI_SOURCE_DIR is build area: PROJECT_BINARY_DIR

Reviewed By: etyurin, bader

Patch by ekovanov.

Differential Revision: https://reviews.llvm.org/D91935
2020-12-09 21:04:24 +03:00
Raphael Isemann
a2c157eb3e [cmake] Make ExecutionEngine/Orc/Shared depend on intrinsics_gen to fix modules build
The LLVM_ENABLE_MODULES builds currently randomly fail due depending on the
headers generated by the intrinsics_gen target, but the current dependency only model
the non-modules dependencies:

```
While building module 'LLVM_ExecutionEngine' imported from llvm-project/llvm/lib/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.cpp:13:
While building module 'LLVM_intrinsic_gen' imported from llvm-project/llvm/include/llvm/ExecutionEngine/Orc/ThreadSafeModule.h:17:
In file included from <module-includes>:1:
In file included from llvm-project/llvm/include/llvm/IR/Argument.h:18:
llvm/include/llvm/IR/Attributes.h:75:14: fatal error: 'llvm/IR/Attributes.inc' file not found
    #include "llvm/IR/Attributes.inc"
             ^~~~~~~~~~~~~~~~~~~~~~~~
```

Depending on whether intrinsics_gen runs before compiling Orc/Shared files we either fail or include an outdated Attributes.inc
in module builds. The Clang modules require these additional dependencies as including/importing one module requires all
includes headers by that module to be parsable.

Differential Revision: https://reviews.llvm.org/D92873
2020-12-08 20:41:35 +01:00
Stefan Gränitz
c54d827fdb [Orc] Two small fixes in TPCDynamicLibrarySearchGenerator
There is one result per lookup symbol, so we have to advance the result iterator no matter whether it's NULL or not.
MissingSymbols variable is unused.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D91707
2020-12-08 10:58:20 +01:00
Stefan Gränitz
83b52b5ba2 [JITLink][ELF] Route objects to their matching linker backends based on header info
Distinguish objects by target properties address size, endian and machine architecture. So far we only
support x86-64 (ELFCLASS64, ELFDATA2LSB, EM_X86_64).

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D90860
2020-12-08 10:56:01 +01:00
Chris Sears
9737c128f1 [llvmbuildectomy] removed vestigial LLVMBuild.txt files
LLVMBuild has been removed from the build system. However, three LLVMBuild.txt
files remain in the tree. This patch simply removes them.

llvm/lib/ExecutionEngine/Orc/TargetProcess/LLVMBuild.txt
llvm/tools/llvm-jitlink/llvm-jitlink-executor/LLVMBuild.txt
llvm/tools/llvm-profgen/LLVMBuild.txt

Differential Revision: https://reviews.llvm.org/D92693
2020-12-05 22:00:22 +01:00
Georgii Rymar
ffbce65f95 [lib/Object, tools] - Make ELFObjectFile::getELFFile return reference.
We always have an object, so we don't have to return a pointer.

Differential revision: https://reviews.llvm.org/D92560
2020-12-04 16:02:29 +03:00
Sylvestre Ledru
fa39b9eab1 TargetProcessControl.cpp - Remove warning: extra ‘;’ 2020-11-27 18:19:58 +01:00
serge-sans-paille
733f7b5084 Revert "[build] normalize components dependencies"
This reverts commit c6ef6e1690d517b3401ea06b1fe46871eb67434d.

Basically, publicly linked libraries have a different semantic than components,
which link libraries privately.

Differential Revision: https://reviews.llvm.org/D91461
2020-11-18 19:23:11 +01:00
serge-sans-paille
c6ef6e1690 [build] normalize components dependencies
Use LINK_COMPONENTS instead of explicit target_link_libraries for components.
This avoids redundancy and potential inconsistencies.

Differential Revision: https://reviews.llvm.org/D91461
2020-11-17 10:42:34 +01:00
Lang Hames
22e44358d3 [ORC] Include config.h in RegisterEHFrames.cpp.
RegisterEHFrames.cpp needs access to the HAVE_REGISTER_FRAME /
HAVE_DEREGISTER_FRAME defines.

rdar://71458921
2020-11-17 14:18:04 +11:00
Lang Hames
f62e5f4569 [MCJIT] Profile the code generated by MCJIT engine using Intel VTune profiler
Patch by Elena Kovanova. Thanks Elena!

Problem:

LLVM already has a feature to profile the JIT-compiled code with VTune. This is
done using Intel JIT Profiling API (https://github.com/intel/ittapi). Function
information is captured by VTune as soon as the function is JIT-compiled. We
tried to use the same approach to report the function information generated by
the MCJIT engine – read parsing the debug information for in-memory ELF module
and report it using JIT API. As the results, we figured out that it did not work
properly for the following cases: inline functions, the functions located in
multiple source files, the functions having several bodies (address ranges).

Solution:

To overcome limitations described above, we have introduced new APIs as a part
of Intel ITT APIs to report the entire in-memory ELF module to be further
processed as regular ELF binaries with debug information.

This patch

1. Switches LLVM to open source version of Intel ITT/JIT APIs
(https://github.com/intel/ittapi) to keep it always up to date.

2. Adds support of profiling the code generated by MCJIT engine using Intel
VTune profiler

Another separate patch will get rid of obsolete Intel ITT APIs stuff, having
LLVM already switched to https://github.com/intel/ittapi.

Differential Revision: https://reviews.llvm.org/D86435
2020-11-16 19:28:14 +11:00
serge-sans-paille
b8ec05a533 llvmbuildectomy - remove conflict file 2020-11-13 15:56:05 +01:00
serge-sans-paille
9218ff50f9 llvmbuildectomy - replace llvm-build by plain cmake
No longer rely on an external tool to build the llvm component layout.

Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.

These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.

Differential Revision: https://reviews.llvm.org/D90848
2020-11-13 10:35:24 +01:00
Lang Hames
98f70e94e0 [ORC] Add dependence of OrcJIT on OrcTargetProcess.
The SelfTargetProcessControl class depends on OrcTargetProcess.
2020-11-13 18:09:41 +11:00
Lang Hames
bdf26d8d19 [ORC] Remove designated initializer. 2020-11-13 17:12:33 +11:00
Lang Hames
1d0676b54c [ORC] Break up OrcJIT library, add Orc-RPC based remote TargetProcessControl
implementation.

This patch aims to improve support for out-of-process JITing using OrcV2. It
introduces two new class templates, OrcRPCTargetProcessControlBase and
OrcRPCTPCServer, which together implement the TargetProcessControl API by
forwarding operations to an execution process via an Orc-RPC Endpoint. These
utilities are used to implement out-of-process JITing from llvm-jitlink to
a new llvm-jitlink-executor tool.

This patch also breaks the OrcJIT library into three parts:
  -- OrcTargetProcess: Contains code needed by the JIT execution process.
  -- OrcShared: Contains code needed by the JIT execution and compiler
     processes
  -- OrcJIT: Everything else.

This break-up allows JIT executor processes to link against OrcTargetProcess
and OrcShared only, without having to link in all of OrcJIT. Clients executing
JIT'd code in-process should start linking against OrcTargetProcess as well as
OrcJIT.

In the near future these changes will enable:
  -- Removal of the OrcRemoteTargetClient/OrcRemoteTargetServer class templates
     which provided similar functionality in OrcV1.
  -- Restoration of Chapter 5 of the Building-A-JIT tutorial series, which will
     serve as a simple usage example for these APIs.
  -- Implementation of lazy, cross-target compilation in lli's -jit-kind=orc-lazy
     mode.
2020-11-13 17:05:13 +11:00
David Tenty
ae032e2714 [CMake][ExecutionEngine] add HAVE_(DE)REGISTER_FRAME as a config.h macros
The macro HAVE_EHTABLE_SUPPORT is used by parts of ExecutionEngine to tell __register_frame/__deregister_frame is available to register the
FDE for a generated (JIT) code. It's currently set by a slowly growing set of macro tests in the respective headers, which is updated now and then when it fails to link on some platform or another due to the symbols being missing (see for example https://bugs.llvm.org/show_bug.cgi?id=5715).

This change converts the macro in two HAVE_(DE)REGISTER_FRAME config.h macros (like most of the other HAVE_* macros) and set's them based on whether CMake can actually find a definition for these symbols to link to at configuration time.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D87114
2020-11-10 13:09:44 -05:00
Valentin Churavy
85f4be0912 [RTDYLD] support absolute relocations where needed
These appear in some sections, such as DWARF tables, since
RuntimeDyldELF explicitly maps to this as a sentinel value:
29d1fba7b5/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp (L1199)

That could then be a source of problems if it tried to examine these
sections (for example, with either setProcessAllSections(true) or ORCv2 on i686).

Replaces https://reviews.llvm.org/D89241

Reviewed By: lhames, vchuravy

Differential Revision: https://reviews.llvm.org/D90722
2020-11-06 14:08:59 -05:00
Stefan Gränitz
b397795f1a [JITLink][ELF] Implement R_X86_64_PLT32 relocations
Basic implementation for call and jmp branches with 32 bit offset. Branches to local targets produce
Branch32 edges that are resolved like a regular PCRel32 relocations. Branches to external (undefined)
targets produce Branch32ToStub edges and go through a PLT entry by default. If the target happens to
get resolved within the 32 bit range from the callsite, the edge is relaxed during post-allocation
optimization. There is a test for each of these cases.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D90331
2020-11-03 12:05:54 +00:00
Simon Pilgrim
d64ea0f189 Remove superfluous whitespace around if(). NFC. 2020-10-25 14:38:16 +00:00
Fangrui Song
f04d92af94 [X86] Produce R_X86_64_GOTPCRELX for test/binop instructions (MOV32rm/TEST32rm/...) when -Wa,-mrelax-relocations=yes is enabled
We have been producing R_X86_64_REX_GOTPCRELX (MOV64rm/TEST64rm/...) and
R_X86_64_GOTPCRELX for CALL64m/JMP64m without the REX prefix since 2016 (to be
consistent with GNU as), but not for MOV32rm/TEST32rm/...
2020-10-24 15:14:17 -07:00
Stefan Gränitz
66abe650ff Reapply "[jitlink][ELF] Add zero-fill blocks for symbols in section SHN_COMMON"
Root cause of the test failure was fixed with:
[JITLink][ELF] PCRel32GOTLoad edge offset can be smaller three

This reverts commit 10b1a61bafba39fd7400a814a7272f41222ad579.
2020-10-24 16:58:06 +02:00
Stefan Gränitz
b6ef40891c [JITLink][ELF] PCRel32GOTLoad edge offset can be smaller three
Offset is 2 for MOVL instruction in test ELF_x86-64_common. This should fix the test failures.

Differential Revision: https://reviews.llvm.org/D89795
2020-10-24 16:57:48 +02:00
Stefan Gränitz
10b1a61baf Revert "[jitlink][ELF] Add zero-fill blocks for symbols in section SHN_COMMON"
This reverts commit e9955b0843cc1e5876430f3f051494d4197419f3. Cannot reproduce the buildbot failures yet. Reverting in the meantime.
2020-10-24 15:43:06 +02:00
Stefan Gränitz
e9955b0843 [jitlink][ELF] Add zero-fill blocks for symbols in section SHN_COMMON
Symbols with special section index SHN_COMMON (0xfff2) haven't been handled so far and caused an invalid section error.

This is a more or less straightforward use of the code commented out at the end of the function. I checked with the ELF spec, that the symbol value gives the alignment.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D89795
2020-10-24 14:54:38 +02:00
Stefan Gränitz
138b9f1928 [JITLink][ELF] PCRel32GOTLoad relocations are resolved like regular PCRel32 ones
The difference is that the former are indirect and go to the GOT while the latter go to the target directly. This info can be used to relax indirect ones that don't need the GOT (because the target is in range). We check for this optimization beforehand. For formal correctness and to avoid confusion, we should only change the relocation kind if we actually apply the relaxation.
2020-10-24 14:54:38 +02:00
Geoffrey Martin-Noble
bfb04aeb85 Unconditionally #include <future>
This unbreaks building with `LLVM_ENABLE_THREADS=0`. Since
https://github.com/llvm/llvm-project/commit/069919c9ba33 usage of
`std::promise` is not guarded by `LLVM_ENABLE_THREADS`, so this header
must be unconditionally included.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D89758
2020-10-23 19:17:37 +00:00
Lang Hames
996a8b42aa Re-apply "[JITLink][ELF] Add support for ELF::R_X86_64_REX_GOTPCRELX relocation"
This re-applies e2fceec2fd1 with fixes. Apparently we already *do* support
relaxation for ELF, so we need to make sure the test case allocates a slab at
a fixed address, and that the R_X86_64_REX_GOTPCRELX test references an external
that is guaranteed to be out of range.
2020-10-23 09:48:05 -07:00
Lang Hames
0f910387e6 Revert "[JITLink][ELF] Add support for ELF::R_X86_64_REX_GOTPCRELX relocation."
This reverts commit e2fceec2fd15b7b74617816ddd87f456c42bbc45.

This commit broke one of the bots. Reverting while I investigate.
2020-10-22 23:21:29 -07:00
Lang Hames
e2fceec2fd [JITLink][ELF] Add support for ELF::R_X86_64_REX_GOTPCRELX relocation.
No support for relaxation yet -- this will always use the GOT entry.
2020-10-22 22:43:16 -07:00
Lang Hames
9898d9d885 [ORC] Fix a missing include. 2020-10-19 12:13:55 -07:00
Lang Hames
c89447b659 [ORC] Fix unused variable warning. 2020-10-19 09:06:33 -07:00
Lang Hames
f35707047e [ORC] Break up C-API header Orc.h, and add JITEventListener support.
This patch breaks Orc.h up into Orc.h, LLJIT.h and OrcEE.h.

Orc.h contain core Orc utilities.
LLJIT.h contains LLJIT specific types and functions.
OrcEE.h contains types and functions that depend on ExecutionEngine.

The intent is that these headers should match future library divisions: Clients
who only use Orc.h should only need to link againt the Orc core libraries,
clients using LLJIT.h will also need to link against LLVM core, and clients
using OrcEE.h will also have to link against ExecutionEngine.

In addition to breaking up the Orc.h header this patch introduces functions to:
(1) Set the object linking layer creation function on LLJITBuilder.
(2) Create an RTDyldObjectLinkingLayer instance (particularly for use in (1)).
(3) Register JITEventListeners with an RTDyldObjectLinkingLayer.

Together (1), (2) and (3) can be used to force use of RTDyldObjectLinkingLayer
as the underlying JIT linker for LLJIT, rather than the platform default, and
to register event listeners with the RTDyldObjectLinkingLayer.
2020-10-19 01:59:04 -07:00
Lang Hames
00369849e1 [ORC] Add function to get pool entry string.
Patch by Andres Freund. Thanks Andres!
2020-10-19 01:59:04 -07:00
Lang Hames
24afffe63a [ORC] Add C API support for defining absolute symbols.
Also tweaks the definition of TryToGenerate to make it dovetail more neatly
with the new function.
2020-10-19 01:59:04 -07:00
Lang Hames
b6ca0c7dd5 [ORC] Add support for custom generators to the C bindings.
C API clients can now define a custom definition generator by providing a
callback function (to implement DefinitionGenerator::tryToGenerate) and context
object. All arguments for the DefinitionGenerator::tryToGenerate method have
been given C API counterparts, and the API allows for optionally asynchronous
generation.
2020-10-19 01:59:04 -07:00
Lang Hames
91d1f417fd [ORC] Add basic ResourceTracker support to the OrcV2 C Bindings.
Based on a patch by Andres Freund. Thanks Andres!
2020-10-19 01:59:04 -07:00