If you build LLVM with `-DCMAKE_CXX_VISIBILITY_PRESET=hidden` to help
reduce binary size, these symbols end up becoming local, and getting
stripped. This forces default visibility to override the global setting
in that case.
Relevant:
https://github.com/llvm/llvm-project/issues/62815#issuecomment-1560078260
LLVM is inconsistent about how it converts `errno` to `std::error_code`.
This can cause problems because values outside of `std::errc` compare
differently if one is system and one is generic on POSIX systems.
This is even more of a problem on Windows where use of the system
category is just wrong, as that is for Windows errors, which have a
completely different mapping than POSIX/generic errors. This patch fixes
one instance of this mistake in `JSONTransport.cpp`.
This patch adds `errnoAsErrorCode()` which makes it so people do not
need to think about this issue in the future. It also cleans up a lot of
usage of `errno` in LLVM and Clang.
[ORC] Re-land https://github.com/llvm/llvm-project/pull/81826
This patch adds two plugins: VTuneSupportPlugin.cpp and
JITLoaderVTune.cpp. The testing is done in a manner similar to
llvm-jitlistener. Currently, we only support the old version of Intel
VTune API.
This patch adds two plugins: VTuneSupportPlugin.cpp and
JITLoaderVTune.cpp. The testing is done in a manner similar to
llvm-jitlistener. Currently, we only support the old version of Intel
VTune API.
This pull request is stacked on top of
https://github.com/llvm/llvm-project/pull/81825
This migrates the dylib manager lookup and related APIs to replace
ExecutorAddress with ExecutorSymbolDef so that in the future we can
model JITSymbolFlags for these symbols. The current change should be NFC
as we are only setting the Exported symbol flag.
https://github.com/llvm/llvm-project/pull/76236 introduced the forth
copy and it was time to deduplicate. This patch brings it back to 2,
one in OrcTargetProcess and one in legacy ExecutionEngine.
/data/llvm-project/llvm/lib/ExecutionEngine/Orc/TargetProcess/JITLoaderPerf.cpp:118:10: error: variable 'Written' set but not used [-Werror,-Wunused-but-set-variable]
size_t Written = 0;
^
1 error generated.
This patch ports PerfJITEventListener to a JITLink plugin, but adds unwind
record support and drops debuginfo support temporarily. Debuginfo can be
enabled in the future by providing a way to obtain a DWARFContext from a
LinkGraph.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146169
By using bootstrap symbols to communicate these addresseses, rather than dlsym
lookups, we no longer need them to be exported from the main executable. On ELF,
where symbols aren't exported from the main executable by default, this
eliminates a common source of missing symbol errors and allows for smaller
executables (if exports from the main executable aren't otherwise needed and
can be removed).
This patch ports PerfJITEventListener to a JITLink plugin, but adds unwind record support and drops debuginfo support temporarily. Debuginfo can be enabled in the future by providing a way to obtain a DWARFContext from a LinkGraph.
See D146060 for an experimental implementation that adds debuginfo parsing.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146169
Configure the plugin to automatically call the debugger rendezvous breakpoint `__jit_debug_register_code()` for every translation unit (enabled) or never at all (disabled). Default API and behavior remain unchanged.
If AutoRegisterCode is turned off, it's the client's own responsibility to call the rendezvous breakpoint function at an appropriate time.
Depending on the complexity of the debugger's rendezvous breakpoint implementation, this can provide significant performance improvements in cases where many debug objects are added in sequence.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D147310
SimpleRemoteEPC already included a "bootstrap symbols map" that could be used
to communicate the addresses of symbols needed for JIT bootstrap. The new
bootstrap map can be used to communicate arbitrary bootstrap values (encoded as
SPS buffers).
The bootstrap symbols map is kept as distinct becasue bootstrap symbols are
significant, and having a known value type for them allows for better debug
logging (we know how to render the values) and tooling (e.g. utils for adding
all bootstrap symbols to a JITDylib).
This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b4 due to bot failures.
The new patch contains fixes to ELFLinkGraphBuilder.h to better handle
non-SHT_ALLOC sections (these were being accidentally skipped in the previous
patch), and to skip SHT_NULL sections.
The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* Finalize: allocated memory lives until all finalize actions have been run,
then is destroyed.
This patch introduces a new 'NoAlloc' option. NoAlloc indicates that the
section should be ignored by the JITLinkMemoryManager -- the memory manager
should allocate neither working memory nor executor address space to blocks in
NoAlloc sections. The NoAlloc option is intended to support metadata sections
(e.g. debug info) that we want to keep in the graph and have fixed up if
necessary, but don't want allocated or transmitted to the executor (or we want
that allocation and transmission to be managed manually by plugins).
Since NoAlloc blocks are ignored by the JITLinkMemoryManager they will not have
working memory allocated to them by default post-allocation. Clients wishing to
modify the content of a block in a NoAlloc section should call
`Block::getMutableMemory(LinkGraph&)` to get writable memory allocated on the
LinkGraph's allocator (this memory will exist for the lifetime of the graph).
If no client requests mutable memory prior to the fixup phase then the generic
link algorithm will do so when it encounters the first edge in any given block.
Addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and should not be modified by the JITLinkMemoryManager. Plugins are responsible
for updating addresses if they add/remove content from these sections. The
meaning of addresses in NoAlloc-sections is backend/plugin defined, but for
fixup purposes they will be treated the same as addresses in Standard/Finalize
sections. References from Standard/Finalize sections to NoAlloc sections are
expected to be common (these represent metadata tracking executor addresses).
References from NoAlloc sections to Standard/Finalize sections are expected to
be rare/non-existent (they would represent JIT'd code / data tracking metadata
in the controller, which would be surprising). LinkGraphBuilders and specific
backends may impose additional constraints on edges between Standard/Finalize
and NoAlloc sections where required for correctness.
Differential Revision: https://reviews.llvm.org/D146183
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
Updates tpctypes::DylibHandle to be an ExecutorAddr (rather than a uint64_t),
and SimpleExecutorDylibManager to hold and return raw OS handle values (as
ExecutorAddrs) rather than index values into a map of DynamicLibrary instances.
This will allow clients to use EPCGenericDylibManager in contexts where the
existing DynamicLibrary interface is too limited to be used. (e.g. to look up
JIT symbols in a dylib that was loaded with RTLD_LOCAL).
This updates the ExecutorSharedMemoryMapperService::deinitialize and
InProcessMemoryMapper::deinitialize methods to deinitialize in reverse order,
bringing them into alignment with the behavior of
InProcessMemoryManager::deallocate and SimpleExecutorMemoryManager::deallocate.
Reverse deinitialization is required because later allocations can depend on
earlier ones.
This fixes failures in the ORC runtime test suite.
When deinitializing, the allocation needs to be removed from the
allocation list of its associated reservation so that remaining
allocations can be deinitialized when releasing the reservation.
Differential Revision: https://reviews.llvm.org/D132313
Introduces COFFVCRuntimeBootstrapper that loads/initialize vc runtime libraries. In COFF, we *must* jit-link vc runtime libraries as COFF relocation types have no proper way to deal with out-of-reach data symbols ragardless of linking mode. (even dynamic version msvcrt.lib have tons of static data symbols that must be jit-linked) This class tries to load vc runtime library files from msvc installations with an option to override the path.
There are some complications when dealing with static version of vc runtimes. First, they need static initializers to be ran that requires COFFPlatform support but orc runtime will not be usable before vc runtimes are fully initialized. (as orc runtime will use msvc stl libraries) COFFPlatform that will be introduced in a following up patch will collect static initializers and run them manually in host before boostrapping itself. So, the user will have to do the following.
1. Create COFFPlatform that addes static initializer collecting passes.
2. LoadVCRuntime
3. InitializeVCRuntime
4. COFFPlatform.bootstrap()
Second, the internal crt initialization function had to be reimplemented in orc side. There are other ways of doing this, but this is the simplest implementation that makes platform fully responsible for static initializer. The complication comes from the fact that crt initialization functions (such as acrt_initialize or dllmain_crt_process_attach) actually run all static initializers by traversing from `__xi_a` symbol to `__xi_z`. This requires symbols to be contiguously allocated in sections alphabetically sorted in memory, which is not possible right now and not practical in jit setting. We might ignore emission of `__xi_a` and `__xi_z` symbol and allocate them ourselves, but we have to take extra care after orc runtime boostrap has been done -- as that point orc runtime should be the one running the static initializers.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130456
Since 67220c2ad72e empty SPSSequence<char>s deserialize to default-constructed
ArrayRef<char>s, which have a null data field. We need to check for this to
avoid memcpy'ing from a nullptr.
This should fix the bot failure in
https://lab.llvm.org/buildbot/#/builders/85/builds/9323
...with more fixes.
The original patch was reverted in 3e9cc543f22 due to bot failures caused by
a missing dependence on librt. That issue was fixed in 32d8d23cd0, but that
commit also broke sanitizer bots due to a bug in SimplePackedSerialization:
empty ArrayRef<char>s triggered a zero-byte memcpy from a null source. The
ArrayRef<char> serialization issue was fixed in 67220c2ad7, and this patch has
also been updated with a new custom SharedMemorySegFinalizeRequest message that
should avoid serializing empty ArrayRefs in the first place.
https://reviews.llvm.org/D128544
This reverts commit 32d8d23cd0b2d4d010eb112dfe5216f11b2681f9.
Reason: Broke the UBSan buildbots. See more details on Phabricator:
https://reviews.llvm.org/D128544
The original commit was reverted in 3e9cc543f223 due to buildbot failures, which
should be fixed by the addition of dependencies on librt.
Differential Revision: https://reviews.llvm.org/D128544
This is an implementation of orc::MemoryMapper that maps shared memory
pages in both executor and controller process and writes directly to
them avoiding transferring content over EPC. All allocations are properly
deinitialized automatically on the executor side at shutdown by the
ExecutorSharedMemoryMapperService.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D128544
This change originally landed as part of
e6f1f062457c928c18a88c612f39d9e168f65a85 (D129120), which caused a
Fuchsia buildbot regression in ExecutionEngine tests.
I am resubmitting the backed out parts in smaller pieces after a careful
review.
Bulk remove many of the more trivial uses of ManagedStatic in the llvm
directory, either by defining a new getter function or, in many cases,
moving the static variable directly into the only function that uses it.
Differential Revision: https://reviews.llvm.org/D129120
