This patch replaces SmallSet<T *, N> with SmallPtrSet<T *, N>. Note
that SmallSet.h "redirects" SmallSet to SmallPtrSet for pointer
element types:
template <typename PointeeType, unsigned N>
class SmallSet<PointeeType*, N> : public SmallPtrSet<PointeeType*, N>
{};
We only have 140 instances that rely on this "redirection", with the
vast majority of them under llvm/. Since relying on the redirection
doesn't improve readability, this patch replaces SmallSet with
SmallPtrSet for pointer element types.
cloneExternalModuleToContext can be used to clone an LLVM module onto a given
ThreadSafeContext. Callers of this function are responsible for ensuring
exclusive access to the source module and its LLVMContext.
Previously, `result` was checked before handling the `Error` returned by
`callSPSWrapper`. If `Error` was `success` but `result` indicated
failure, the original `Error` was silently dropped, triggering an
assertion failure for unhandled `Error`.
This patch ensures the `Error` is checked and returned before inspecting
`result`, preventing assertion failures.
Co-authored-by: Anutosh Bhat <andersonbhat491@gmail.com>
This is a generalization of the existing cloneToNewContext operation:
rather than cloning the given module into a new ThreadSafeContext it
clones it into any given ThreadSafeContext. The given ThreadSafeContext
is locked to ensure that the cloning operation is safe.
This removes ThreadSafeContext::Lock, ThreadSafeContext::getLock, and
ThreadSafeContext::getContext, and replaces them with a
ThreadSafeContext::withContextDo method (and const override).
The new method can be used to access an existing
ThreadSafeContext-wrapped LLVMContext in a safe way:
ThreadSafeContext TSCtx = ... ;
TSCtx.withContextDo([](LLVMContext *Ctx) {
// this closure has exclusive access to Ctx.
});
The new API enforces correct locking, whereas the old APIs relied on
manual locking (which almost no in-tree code preformed, relying instead
on incidental exclusive access to the ThreadSafeContext).
ArrayRef now has a new constructor that takes a parameter whose type
has data() and size(). This patch migrates:
ArrayRef<T>(X.data(), X.size()
to:
ArrayRef<T>(X)
This commit adds operations to orc::MemoryAccess for reading basic types
(uint8_t, uint16_t, uint32_t, uint64_t, pointers, buffers, and strings)
from executor memory.
The InProcessMemoryAccess and EPCGenericMemoryAccess implementations are
updated to support the new operations.
This moves the MemoryAccess interface out of the ExecutorProcessControl
class and splits implementation classes InProcessMemoryManager and
SelfExecutorProcessControl out of ExecutorProcessControl.h and into
their own headers.
LookupKind::DLSym should only be used for lookups issued on behalf of the ORC
runtime's emulated dlsym operation.
This should fix a bug where ORC runtime clients are unable to access functions
exported by the runtime.
https://github.com/llvm/llvm-project/issues/145296
We can't exit early when on error here as some threads may still be holding
references to LookupMutex.
Since we don't need high performance in the error case the easy solution is to
drop the early-exit in the error case and wait for all tasks to complete before
returning the error.
Thanks to Jameson Nash for spotting this bug!
…eFromDylib.
When comparing CPU subtypes from slices in a MachO universal binary we
need to apply the MachO::CPU_SUBTYPE_MASK to mask out any flags in the
high bits, otherwise we might fail to correctly match a slice and return
a spurious "does not contain slice" error.
rdar://153913779
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
`RTDyldObjectLinkingLayer` is currently creating a memory manager
without any parameters.
In this PR I am passing the MemoryBuffer that will be emitted to the
MemoryManager so that the user can use it to configure the behaviour of
the MemoryManager.
## Purpose
This patch is one in a series of code-mods that annotate LLVM’s public
interface for export. This patch annotates the `llvm/ExecutionEngine`
library. These annotations currently have no meaningful impact on the
LLVM build; however, they are a prerequisite to support an LLVM Windows
DLL (shared library) build.
## Background
This effort is tracked in #109483. Additional context is provided in
[this
discourse](https://discourse.llvm.org/t/psa-annotating-llvm-public-interface/85307),
and documentation for `LLVM_ABI` and related annotations is found in the
LLVM repo
[here](https://github.com/llvm/llvm-project/blob/main/llvm/docs/InterfaceExportAnnotations.rst).
The bulk of these changes were generated automatically using the
[Interface Definition Scanner (IDS)](https://github.com/compnerd/ids)
tool, followed formatting with `git clang-format`.
The following manual adjustments were also applied after running IDS on
Linux:
- Add `LLVM_ABI_FRIEND` to friend member functions declared with
`LLVM_ABI`
- Add `LLVM_ABI` to a subset of private class methods and fields that
require export
- Add `LLVM_ABI` to a small number of symbols that require export but
are not declared in headers
- Add `LLVM_ABI` to a number of `extern "C"` methods that IDS missed
because they're implicitly exported
## Validation
Local builds and tests to validate cross-platform compatibility. This
included llvm, clang, and lldb on the following configurations:
- Windows with MSVC
- Windows with Clang
- Linux with GCC
- Linux with Clang
- Darwin with Clang
MachOPlatform::BootstrapInfo::Mutex was meant to be used to synchronize access
to the MachOPlatform::BootstrapInfo::ActiveGraphs member, but the latter was
also modified under the MachOPlatform::PlatformMutex (in
MachOPlatform::MachOPlatformPlugin::modifyPassConfig), leading to a data race.
There have been external reports (rdar://151041549) of deadlocks on the
MachOPlatform::BootstrapInfo::CV condition variable that are consistent with
corruption of the ActiveGraphs member (though alternative explanations for
the reported behavior exist, and it has been too rare in practice to verify).
This patch removes the misused MachOPlatform::BootstrapInfo::Mutex member and
synchronizes all accesses to ActiveGraphs using MachOPlatform::PlatformMutex
instead. Since ActiveGraphs is only used during bootstrap the performance
impact of this should be negligible.
rdar://151041549 - possible fix.
This reapplies b0979b8c65d, which was reverted in 370aecb9572 due to bot
failures. The failures were caused by typos in the testcase that are fixed in
this patch.
This refactor was motivated by two bugs identified in out-of-tree
builds:
1. Some implementations of the VisitMembersFunction type (often used to
implement special loading semantics, e.g. -all_load or -ObjC) were assuming
that buffers for archive members were null-terminated, which they are not in
general. This was triggering occasional assertions.
2. Archives may include multiple members with the same file name, e.g.
when constructed by appending files with the same name:
% llvm-ar crs libfoo.a foo.o
% llvm-ar q libfoo.a foo.o
% llvm-ar t libfoo.a foo.o
foo.o
While confusing, these members may be safe to link (provided that they're
individually valid and don't define duplicate symbols). In ORC however, the
archive member name may be used to construct an ORC initializer symbol,
which must also be unique. In that case the duplicate member names lead to a
duplicate definition error even if the members define unrelated symbols.
In addition to these bugs, StaticLibraryDefinitionGenerator had grown a
collection of all member buffers (ObjectFilesMap), a BumpPtrAllocator
that was redundantly storing synthesized archive member names (these are
copied into the MemoryBuffers created for each Object, but were never
freed in the allocator), and a set of COFF-specific import files.
To fix the bugs above and simplify StaticLibraryDefinitionGenerator this
patch makes the following changes:
1. StaticLibraryDefinitionGenerator::VisitMembersFunction is generalized
to take a reference to the containing archive, and the index of the
member within the archive. It now returns an Expected<bool> indicating
whether the member visited should be treated as loadable, not loadable,
or as invalidating the entire archive.
2. A static StaticLibraryDefinitionGenerator::createMemberBuffer method
is added which creates MemoryBuffers with unique names of the form
`<archive-name>[<index>](<member-name>)`. This defers construction of
member names until they're loaded, allowing the BumpPtrAllocator (with
its redundant name storage) to be removed.
3. The ObjectFilesMap (symbol name -> memory-buffer-ref) is replaced
with a SymbolToMemberIndexMap (symbol name -> index) which should be
smaller and faster to construct.
4. The 'loadability' result from VisitMemberFunctions is now taken into
consideration when building the SymbolToMemberIndexMap so that members
that have already been loaded / filtered out can be skipped, and do not
take up any ongoing space.
5. The COFF ImportedDynamicLibraries member is moved out into the
COFFImportFileScanner utility, which can be used as a
VisitMemberFunction.
This fixes the bugs described above; and should lower memory consumption
slightly, especially for archives with many files and / or symbol where
most files are eventually loaded.
The context string can be added to indicate the source of the duplicate
definition. E.g. if the context is set to "foo2.o", then:
"Duplicate definition of symbol 'foo'"
becomes
"In foo2.o, duplicate definition of symbol 'foo'".
The JITDylib::defineImpl method is updated to use the name of the
MaterializationUnit being added as the context string for duplicate definition
errors. The JITDylib::defineMaterializing method is updated to use
"defineMaterializing operation" as the conext string.
The JITLinkMemoryManager::InFlightAlloc::abandon method should only abandon
memory for the current allocation, not any other allocations. In
MapperJITLinkMemoryManager this corresponds to the deinitialize operation, not
the deallocate operation (which releases whole slabs of memory that may be
shared by many allocations).
No testcase: This was spotted by inspection. The failing program was linking
concurrently when one linker instance raised an error. Through the call to
abandon an entire underlying slab was deallocated, resulting in segfaults in
other concurrent links that were sharing that slab.
We can use *Set::insert_range to collapse:
for (auto Elem : Range)
Set.insert(E);
down to:
Set.insert_range(Range);
In some cases, we can further fold that into the set declaration.
Currently, `DIContext::getLineInfoForAddress` and
`DIContext::getLineInfoForDataAddress` returns empty DILineInfo when the
debug info is missing for the given address. This is not differentiable
with the case when debug info is found for the given address but the
debug info is default value (filename:linenum is <invalid>:0).
This change wraps the return types of `DIContext::getLineInfoForAddress`
and `DIContext::getLineInfoForDataAddress` with `std::optional`.
Part of ongoing work to make core ORC operations asynchronous.
This only affects the runner utilities, not the AllocationActions themselves.
The AllocationActions will be made asynchronous in a future patch.
This PR will solve the issue with leaking shared memory we have after running llvm lit test on z/OS.
In particular llvm/unittests/ExecutionEngine/Orc/SharedMemoryMapperTest.cpp was causing the leak.
The module currently stores the target triple as a string. This means
that any code that wants to actually use the triple first has to
instantiate a Triple, which is somewhat expensive. The change in #121652
caused a moderate compile-time regression due to this. While it would be
easy enough to work around, I think that architecturally, it makes more
sense to store the parsed Triple in the module, so that it can always be
directly queried.
For this change, I've opted not to add any magic conversions between
std::string and Triple for backwards-compatibilty purses, and instead
write out needed Triple()s or str()s explicitly. This is because I think
a decent number of them should be changed to work on Triple as well, to
avoid unnecessary conversions back and forth.
The only interesting part in this patch is that the default triple is
Triple("") instead of Triple() to preserve existing behavior. The former
defaults to using the ELF object format instead of unknown object
format. We should fix that as well.
