Call dangerous code on main thread as well.
And use _exit() to avoid any issues during regular process cleanup.
And disable TSAN as it does not lock allocator yet.
almost NFC, just that now we accept INT_MIN and INT_MAX
as discussed in https://r.android.com/2831100, but I didn't add the
*ValueEnd != Value check because I want to keep this change
behaviour-keeping.
Embedded Darwin platforms have generalized COMPILER_RT_ENABLE_<PLATFORM>
configuration settings, but currently 'osx' is always eabled on Darwin.
Add ORC_ENABLE_OSX to allow explicitly *disabling* the orc runtime for
macOS platform. This can be useful if you only want to build a specific
embedded platform. It would be nice to generalize this to handle other
compiler-rt projects (i.e. add COMPILER_RT_ENABLE_OSX), but would
require additional attention from each compiler-rt project.
Note: some tests currently only are configured for osx, so these are
disabled when osx is disabled.
## Motivation
Since we don't need the metadata sections at runtime, we can somehow
offload them from memory at runtime. Initially, I explored [debug info
correlation](https://discourse.llvm.org/t/instrprofiling-lightweight-instrumentation/59113),
which is used for PGO with value profiling disabled. However, it
currently only works with DWARF and it's be hard to add such artificial
debug info for every function in to CodeView which is used on Windows.
So, offloading profile metadata sections at runtime seems to be a
platform independent option.
## Design
The idea is to use new section names for profile name and data sections
and mark them as metadata sections. Under this mode, the new sections
are non-SHF_ALLOC in ELF. So, they are not loaded into memory at runtime
and can be stripped away as a post-linking step. After the process
exits, the generated raw profiles will contains only headers + counters.
llvm-profdata can be used correlate raw profiles with the unstripped
binary to generate indexed profile.
## Data
For chromium base_unittests with code coverage on linux, the binary size
overhead due to instrumentation reduced from 64M to 38.8M (39.4%) and
the raw profile files size reduce from 128M to 68M (46.9%)
```
$ bloaty out/cov/base_unittests.stripped -- out/no-cov/base_unittests.stripped
FILE SIZE VM SIZE
-------------- --------------
+121% +30.4Mi +121% +30.4Mi .text
[NEW] +14.6Mi [NEW] +14.6Mi __llvm_prf_data
[NEW] +10.6Mi [NEW] +10.6Mi __llvm_prf_names
[NEW] +5.86Mi [NEW] +5.86Mi __llvm_prf_cnts
+95% +1.75Mi +95% +1.75Mi .eh_frame
+108% +400Ki +108% +400Ki .eh_frame_hdr
+9.5% +211Ki +9.5% +211Ki .rela.dyn
+9.2% +95.0Ki +9.2% +95.0Ki .data.rel.ro
+5.0% +87.3Ki +5.0% +87.3Ki .rodata
[ = ] 0 +13% +47.0Ki .bss
+40% +1.78Ki +40% +1.78Ki .got
+12% +1.49Ki +12% +1.49Ki .gcc_except_table
[ = ] 0 +65% +1.23Ki .relro_padding
+62% +1.20Ki [ = ] 0 [Unmapped]
+13% +448 +19% +448 .init_array
+8.8% +192 [ = ] 0 [ELF Section Headers]
+0.0% +136 +0.0% +80 [7 Others]
+0.1% +96 +0.1% +96 .dynsym
+1.2% +96 +1.2% +96 .rela.plt
+1.5% +80 +1.2% +64 .plt
[ = ] 0 -99.2% -3.68Ki [LOAD #5 [RW]]
+195% +64.0Mi +194% +64.0Mi TOTAL
$ bloaty out/cov-cor/base_unittests.stripped -- out/no-cov/base_unittests.stripped
FILE SIZE VM SIZE
-------------- --------------
+121% +30.4Mi +121% +30.4Mi .text
[NEW] +5.86Mi [NEW] +5.86Mi __llvm_prf_cnts
+95% +1.75Mi +95% +1.75Mi .eh_frame
+108% +400Ki +108% +400Ki .eh_frame_hdr
+9.5% +211Ki +9.5% +211Ki .rela.dyn
+9.2% +95.0Ki +9.2% +95.0Ki .data.rel.ro
+5.0% +87.3Ki +5.0% +87.3Ki .rodata
[ = ] 0 +13% +47.0Ki .bss
+40% +1.78Ki +40% +1.78Ki .got
+12% +1.49Ki +12% +1.49Ki .gcc_except_table
+13% +448 +19% +448 .init_array
+0.1% +96 +0.1% +96 .dynsym
+1.2% +96 +1.2% +96 .rela.plt
+1.2% +64 +1.2% +64 .plt
+2.9% +64 [ = ] 0 [ELF Section Headers]
+0.0% +40 +0.0% +40 .data
+1.2% +32 +1.2% +32 .got.plt
+0.0% +24 +0.0% +8 [5 Others]
[ = ] 0 -22.9% -872 [LOAD #5 [RW]]
-74.5% -1.44Ki [ = ] 0 [Unmapped]
[ = ] 0 -76.5% -1.45Ki .relro_padding
+118% +38.8Mi +117% +38.8Mi TOTAL
```
A few things to note:
1. llvm-profdata doesn't support filter raw profiles by binary id yet,
so when a raw profile doesn't belongs to the binary being digested by
llvm-profdata, merging will fail. Once this is implemented,
llvm-profdata should be able to only merge raw profiles with the same
binary id as the binary and discard the rest (with mismatched/missing
binary id). The workflow I have in mind is to have scripts invoke
llvm-profdata to get all binary ids for all raw profiles, and
selectively choose the raw pnrofiles with matching binary id and the
binary to llvm-profdata for merging.
2. Note: In COFF, currently they are still loaded into memory but not
used. I didn't do it in this patch because I noticed that `.lcovmap` and
`.lcovfunc` are loaded into memory. A separate patch will address it.
3. This should works with PGO when value profiling is disabled as debug
info correlation currently doing, though I haven't tested this yet.
Lock Lsan and Thread related date at_fork.
Clean shadow before thread starts, forked process may reuse already
mapped stack of 'lost' parent thread for new threads.
When LLVM_PROFILE_FILE is set incorrectly (e.g. multiple %c) and it
falls back to use `default.profraw` name, but continuous mode is still
set. This might cause signal bus in the following scenario.
LLVM_PROFILE_FILE is set incorrectly (with "%c%c") for process A and B.
Suppose A starts first and falls back to use `default.profraw` and
mmaped its file content to memory. Later B starts and also falls back to
use `default.profraw`, but it will truncate the file because online
merging is disable when reseting to `default.profraw`. When A tries to
update counter via mmaped memory, signal bus will occur.
This fixes it by disabling continuous mode when reset to
default.profraw.
MachOPlatformRuntimeState::lookupSymbols encapsulates the approach used in
dlsym in bb41fc682ee, but generalizes it to multiple symbols:
1. try to resolve symbols locally
2. issue a push-request for any unresolved symbols
3. re-try local resolution
In the future lookupSymbols can serve as the basis for a public bulk lookup
API in the ORC runtime.
1. Prevent deadlock by unlocking JDStatesMutex when calling back to the
controller to request a push of new symbols. (If JDStatesMutex is locked
then the push operation can't register the new symbols, and so can't
complete).
2. Record MachOPlatform runtime symbols during bootstrap and attach their
registration to the bootstrap-completion graph, similar to the way that
deferred allocation actions are handled. We can't register the symbols
the normal way during bootstrap since the symbol registration function is
itself in the process of being materialized.
3. Add dlsym testcases to exercise these fixes.
`realloc` may involve both allocation and deallocation. Given that the
reporting the events is not atomic and which may lead the hook user to a
false case that the double-use pattern happens. In general, this can be
resolved on the hook side. To alleviate the task of handling it, we add
two new hooks to mark the range so that the hook user can combine those
calls together.
When set to non-zero, the HWASan runtime will map the shadow base at the
specified constant address.
This is particularly useful in conjunction with the existing compiler
option
'hwasan-mapping-offset', which bakes a hardcoded constant address into
the instrumentation.
---------
Co-authored-by: Thurston Dang <thurston@google.com>
Whenever linking with -nodefaultlibs for a MinGW target, we manually
need to specify a bunch of libraries - listed in ${MINGW_LIBRARIES}; the
same is already done for sanitizers and libunwind/libcxxabi/libcxx.
Practically speaking, linking with -nodefaultlibs but manually passing
the libraries in ${MINGW_LIBRARIES} restores most of the libraries that
are linked by default, except for the potential compiler builtins and
unwind library; i.e. it has essentially the same effect as linking with
"--unwindlib=none -rtlib=none", except that -rtlib doesn't accept such a
value.
When building only compiler-rt/lib/builtins, not all of compiler-rt,
${MINGW_LIBRARIES} is unset - set it manually here for that case. This
matches what is set in
compiler-rt/cmake/config-ix.cmake, except that the builtins (libgcc or
compiler-rt builtins) is omitted; the only use within lib/buitlins is
for the standalone libatomic, which explicitly already links against the
just-built builtins.
Issue #72639
The commit at 020cdaf broke build of asan on macOS with GCC. GCC does
not support the Apple blocks extension (yet). Uses of blocks in other
parts of the sanitisers are protected by MISSING_BLOCKS_SUPPORT. But the
type definition is not.
_This applies FX's patch from the issue._
Co-authored-by: FX Coudert <fxcoudert@gmail.com>
This reapplies 3d0dd1a7d6, which was reverted in df2485b215a due to bot
failures. This patch addresses the issues seen on the bots by disabling two
Linux atexit tests in the ORC runtime whose behavior could not be maintained
now that the ORC runtime is being loaded into a separate Platform JITDylib.
https://github.com/llvm/llvm-project/issues/74641 has been filed to fix the
issue with atexit.
The test_only_replace_dlopen_main_program flag
(introduced in
0be4c6b948)
will cause internal_strcmp to dereference NULL if DlAddrSelfFName()
returns NULL (which happens in very rare cases). This patch adds a
null pointer check.
Co-authored-by: Thurston Dang <thurston@google.com>
These will be used in follow-up CLs, committing this separately because
it needs a matching change in AOSP. This way we can avoid complicated
multi-repo rollbacks if something is wrong with the follow up CLs.
This change fixes a build break introduced by aafad2d(#70353) on
the clang-ppc64le-rhel build bot. If the third-party Google Test suite
is built using gcc-toolset-12, the implementation of std::stable_sort in
the toolchain will use a get_temporary_buffer declaration that is marked
_GLIBCXX17_DEPRECATED. This change adds -Wno-deprecated-declarations to
the GTest flags if the toolchain is detected in the build compiler on linux.
The Process JITDylib holds reflected process symbols. The Platform JITDylib
holds ORC runtime symbols if the ORC runtime is loaded. The Platform and
Process JITDylibs are appended to the link order of all other JITDylibs,
including the main JITDylib, after any explicitly specified libraries. This
scheme is similar to the one introduced in LLJIT in 371cb1af61d, and makes
it easier to introduce aliases for process and platform symbols in a way that
affects all JITDylibs uniformly.
Since the Process and Platform JITDylibs are created implicitly the -alias
option is generalized to allow source and destination JITDylibs to be explicitly
specified, i.e. the -alias option now supports general re-exports.
Testcases are updated to account for the change.
