Currently, WebAssembly/WASI target does not provide direct support for
code coverage.
This patch set fixes several issues to unlock the feature. The main
changes are:
1. Port `compiler-rt/lib/profile` to WebAssembly/WASI.
2. Adjust profile metadata sections for Wasm object file format.
- [CodeGen] Emit `__llvm_covmap` and `__llvm_covfun` as custom sections
instead of data segments.
- [lld] Align the interval space of custom sections at link time.
- [llvm-cov] Copy misaligned custom section data if the start address is
not aligned.
- [llvm-cov] Read `__llvm_prf_names` from data segments
3. [clang] Link with profile runtime libraries if requested
See each commit message for more details and rationale.
This is part of the effort to add code coverage support in Wasm target
of Swift toolchain.
The profiling code requires GNU extensions as it uses functions such as getpagesize(), fdopen(), etc.
The problem manifests when the compiler is built to not default to the extensions mode, e.g. custom config with -std=c2x. CMake didn't support this scenario very well, but it's been fixed by CMP0128. Set the policy to NEW as we now conform to it.
This PR introduces shared library (DSO) support for XRay based on a
revised version of the implementation outlined in [this
RFC](https://discourse.llvm.org/t/rfc-upstreaming-dso-instrumentation-support-for-xray/73000).
The feature enables the patching and handling of events from DSOs,
supporting both libraries linked at startup or explicitly loaded, e.g.
via `dlopen`.
This patch adds the following:
- The `-fxray-shared` flag to enable the feature (turned off by default)
- A small runtime library that is linked into every instrumented DSO,
providing position-independent trampolines and code to register with the
main XRay runtime
- Changes to the XRay runtime to support management and patching of
multiple objects
These changes are fully backward compatible, i.e. running without
instrumented DSOs will produce identical traces (in terms of recorded
function IDs) to the previous implementation.
Due to my limited ability to test on other architectures, this feature
is only implemented and tested with x86_64. Extending support to other
architectures is fairly straightforward, requiring only a
position-independent implementation of the architecture-specific
trampoline implementation (see
`compiler-rt/lib/xray/xray_trampoline_x86_64.S` for reference).
This patch does not include any functionality to resolve function IDs
from DSOs for the provided logging/tracing modes. These modes still work
and will record calls from DSOs, but symbol resolution for these
functions in not available. Getting this to work properly requires
recording information about the loaded DSOs and should IMO be discussed
in a separate RFC, as there are mulitple feasible approaches.
@petrhosek @jplehr
The logic was simply switched. Fixes building tests for me.
Not sure how I haven't stumbled upon this before considering this code hasn't changed lately and my setup's been the same.
Fixes: 07317bbc66d1f2d7663af3c9f04d0f6c0487ac03
This is only for "real" msvc and is to support running sanitizer tests
with "real" MSVC. Most compiler-rt tests do not yet pass with msvc,
which will be addressed in future PRs for asan, sanitizer_common, and
the fuzzer.
We've been using these changes to lit to run CI for the version of
address sanitizer that ships with MSVC, by upstreaming them we can avoid
being broken by future lit changes and allow contributors (importantly
_us_) to run the MSVC tests on changes developed in the open.
In these environments, the architecture name is armv7; recognize that
and enable the relevant runtimes.
Fix building the sanitizer_common library for this target, by using the
right registers for the architecture - this is similar to what
0c391133c9201ef29273554a1505ef855ce17668 did for aarch64.
(Still, address sanitizer doesn't support hooking functions at runtime
on armv7 or aarch64 - but other runtimes such as ubsan do work.)
This syncs the list of supported sanitizers with the matching code
in clang (`FreeBSD::getSupportedSanitizers()`),
Reviewed By: emaste, DimitryAndric
Pull Request: https://github.com/llvm/llvm-project/pull/84280
In a `runtimes` build on Solaris/amd64, there are two failues:
```
AddressSanitizer-Unit :: ./Asan-i386-calls-Dynamic-Test/failed_to_discover_tests_from_gtest
AddressSanitizer-Unit :: ./Asan-i386-inline-Dynamic-Test/failed_to_discover_tests_from_gtest
```
This happens when `lit` enumerates the tests with `--gtest_list_tests
--gtest_filter=-*DISABLED_*`. The error is twofold:
- The `LD_LIBRARY_PATH*` variables point at the 64-bit directory
(`lib/clang/19/lib/x86_64-pc-solaris2.11`) for a 32-bit test:
```
ld.so.1: Asan-i386-calls-Dynamic-Test: fatal:
/var/llvm/local-amd64-release-stage2-A-flang-clang18-runtimes/tools/clang/stage2-bins/./lib/../lib/clang/19/lib/x86_64-pc-solaris2.11/libclang_rt.asan.so:
wrong ELF class: ELFCLASS64
```
- While the tests are linked with `-Wl,-rpath`, that path always is the
64-bit directory again.
Accordingly, the fix consists of two parts:
- The code in `compiler-rt/test/asan/Unit/lit.site.cfg.py.in` to adjust
the `LD_LIBRARY_PATH*` variables is guarded by a `config.target_arch !=
config.host_arch` condition. This is wrong in two ways:
- The adjustment is always needed independent of the host arch. This is
what `compiler-rt/test/lit.common.cfg.py` already does.
- Besides, `config.host_arch` is ultimately set from
`CMAKE_HOST_SYSTEM_PROCESSOR`. On Linux/x86_64, this is `x86_64` (`uname
-m`) while on Solaris/amd64 it's `i386` (`uname -p`), explaining why the
transformation is skipped on Solaris, but not on Linux.
- Besides, `RPATH` needs to be set to the correct subdirectory, so
instead of using the default arch in `compiler-rt/CMakeLists.txt`, this
patch moves the code to a function which takes the test's arch into
account.
Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.
Follows https://github.com/llvm/llvm-project/pull/98268 with a fix for a
segfault during preinit on `ubuntu:20.04` environments. Previously,
`rtsan` was not handling the situation where `dlsym` calls `calloc`
during the interceptors initialization, resulting in a call to a
function at a null address.
@cjappl and I took inspiration from the solution in `nsan`, but we
re-used the sanitizer internal allocator instead of our own static
buffer. This PR also re-enables the existing non-instrumented `rtsan`
tests for `x86_64` and `arm64` architectures.
---------
Co-authored-by: Chris Apple <cja-private@pm.me>
1. Move checks into parent test/CMakeLists.txt
2. COMPILER_RT_INCLUDE_TESTS disable both lit and
gtests. Before it was very inconsistent between
sanitizers.
Even with the `-u __safestack_init` link order fixed on Solaris, there
are still several safestack test issues left:
- While 540fd42c755f20f7b79c6c79493ec36d8cb9b3d3 enabled safestack on
Solaris in the driver unconditionally, it ignored that Solaris also
exists on SPARC and forgot to enable SPARC support for the runtime lib.
This patch fixes that.
- The tests fail to link with undefined references to
`__sanitizer_internal_memset` etc in `safestack.cpp.o` and
`interception_linux.cpp.o`. These are from indirectly including
`sanitizer_redefine_builtins.h`. Instead of using the implementations
from `sanitizer_common` as was done in [[safestack] Various Solaris
fixes](https://github.com/llvm/llvm-project/pull/98469), this patch
disables the interception as discussed in [Revert "[safestack] Various
Solaris fixes"](https://github.com/llvm/llvm-project/pull/98541). A
similar issue affects 32-bit Linux/sparc where compiling `safestack.cpp`
with `-ftrivial-auto-var-init=pattern` causes the compiler to generate
calls to `memset` to initialize a `pthread_attr_t` which is larger than
can be handled inline. This is avoided by defining
`SANITIZER_COMMON_NO_REDEFINE_BUILTINS` in `safestack.cpp` and also
adding definitions of the interceptors that just forward to `libc` for
the benefit of `interception_linux.cpp`.
- The `pthread*.c` tests `FAIL` with
``` safestack CHECK failed:
/vol/llvm/src/llvm-project/local/compiler-rt/lib/safestack/safestack.cpp:227
size ```
The problem is that `pthread_attr_init` initializes the `stacksize`
attribute to 0, signifying the default. Unless explicitly overridded, it
stays that way. I think this is allowed by XPG7. Since safestack cannot
deal with this, I set `size` to the defaults documented in
`pthread_create(3C)`. Unfortunately, there's no macro for those values
outside of private `libc` headers.
- The Solaris `syscall` interface isn't stable. This is not just a
theoretical concern, but the syscalls have changed incompatibly several
times in the past. Therefore this patch switches the implementations of
`TgKill` (where `SYS_lwp_kill` doesn't exist on Solaris 11.4 anyway),
`Mmap`, `Munmap`, and `Mprotect` to the same `_REAL*` solution already
used in `sanitizer_solaris.cpp`.
With those changes, safestack compiles and all tests `PASS`, so the
tests are re-enabled for good.
Tested on `amd64-pc-solaris2.11`, `sparcv9-sun-solaris2.11`,
`x86_64-pc-linux-gnu`, and `sparc64-unknown-linux-gnu`.
Reverts llvm/llvm-project#98469
We can't add this dependency
```
OBJECT_LIBS RTSanitizerCommon
RTSanitizerCommonLibc
```
safestack is security hardening, and RTSanitizerCommon is too fat for
that.
Even with the `-u __safestack_init` link order fixed on Solaris, there
are still several safestack test issues left:
* While 540fd42c755f20f7b79c6c79493ec36d8cb9b3d3 enabled safestack on
Solaris in the driver unconditionally, it ignored that Solaris also
exists on SPARC and forgot to enable SPARC support for the runtime lib.
This patch fixes that.
- The tests fail to link with undefined references to
`__sanitizer_internal_memset` etc. These are from
`sanitizer_redefine_builtins.h`. Definitions live in
`sanitizer_libc.cpp.o`. This patch adds them to the safestack runtime
lib as is already the case e.g. for asan and ubsan. Why GNU ld allows
the link to complete with those references undefined is beyond me.
- The `pthread*.c` tests `FAIL` with
``` safestack CHECK failed:
/vol/llvm/src/llvm-project/local/compiler-rt/lib/safestack/safestack.cpp:227
size ```
The problem is that `pthread_attr_init` initializes the `stacksize`
attribute to 0, signifying the default. Unless explicitly overridded, it
stays that way. I think this is allowed by XPG7. Since safestack cannot
deal with this, I set `size` to the defaults documented in
`pthread_create(3C)`. Unfortunately, there's no macro for those values
outside of private `libc` headers.
- The Solaris `syscall` interface isn't stable. This is not just a
theoretical concern, but the syscalls have changed incompatibly several
times in the past. Therefore this patch switches the implementations of
`TgKill` (where `SYS_lwp_kill` doesn't exist on Solaris 11.4 anyway),
`Mmap`, `Munmap`, and `Mprotect` to the same `_REAL*` solution already
used in `sanitizer_solaris.cpp`. Instead of duplicating what's already
in `sanitizer_common`, it seems way better to me to just reuse those
implementations, though. A subsequent patch does just that.
With those changes, safestack compiles and all tests `PASS`, so the
tests are re-enabled for good.
Tested on `amd64-pc-solaris2.11`, `sparcv9-sun-solaris2.11`,
`x86_64-pc-linux-gnu`, and `sparc64-unknown-linux-gnu`.
Some failures in Rtsan-aarch64-NoInstTest have been reported.
This reverts commit ed17431bf7489c87ea81a6a67af5c7c206da0080.
This reverts commit b81fcd01bde51eb8976b81a2c0c19fc0645cd2ff.
Summary:
This patch adds initial support to build the `builtins` library for GPU
targets. Primarily this requires adding a few new architectures for
`amdgcn` and `nvptx64`. I built this using the following invocations.
```console
$ cmake ../compiler-rt -DCMAKE_C_COMPILER=clang
-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release -GNinja
-DCMAKE_C_COMPILER_TARGET=<nvptx64-nvidia-cuda|amdgcn-amd-amdhsa>
-DCMAKE_CXX_COMPILER_TARGET=<nvptx64-nvidia-cuda|amdgcn-amd-amdhsa>
-DCMAKE_C_COMPILER_WORKS=1 -DCMAKE_CXX_COMPILER_WORKS=1
-DLLVM_CMAKE_DIR=../cmake/Modules -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
-C ../compiler-rt/cmake/caches/GPU.cmake
```
Some pointers would be appreciated for how to test this using a standard
(non-default target only) build.
GPU builds are somewhat finnicky. We only expect this to be built with a
sufficiently new clang, as it's the only compiler that supports the
target and output we distribute. Distribution is done as LLVM-IR blobs
for now.
GPUs have little backward compatibility, so linking object files is
left to a future patch.
More work is necessary to build correctly for all targets and ship into
the correct clang resource directory. Additionally we need to use the
`libc` project's support for running unit tests.
Follow on to #92460 (reporting old android builds failing) and #98264
(powerpc failing to discover gtests).
Getting back to stability by getting back down to a very basic set of
supported arches and OS's. In the future if there is demand we can build
it back up. Especially when I understand how to test these systems, or
have people who want to do the work on them.
---------
Co-authored-by: David Trevelyan <david.trevelyan@gmail.com>
Follow up to #98219
This reverts commit
[14f7450](14f745074d)
Ensure that -pthread is explicitly linked when running the rtsan tests.
Issue this fixes:
```
FAILED: compiler-rt/lib/rtsan/tests/Rtsan-powerpc64le-NoInstTest /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/Rtsan-powerpc64le-NoInstTest
cd /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests && /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./bin/clang++ RtsanNoInstTestObjects.rtsan_preinit.cpp.powerpc64le.o RtsanNoInstTestObjects.rtsan_test_context.cpp.powerpc64le.o RtsanNoInstTestObjects.rtsan_test_main.cpp.powerpc64le.o RtsanNoInstTestObjects.gtest-all.cc.powerpc64le.o RtsanNoInstTestObjects.gmock-all.cc.powerpc64le.o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/libRTRtsanTest.powerpc64le.a -o /home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/runtimes/runtimes-bins/compiler-rt/lib/rtsan/tests/./Rtsan-powerpc64le-NoInstTest -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -resource-dir=/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./lib/../lib/clang/19 -Wl,-rpath,/home/buildbots/llvm-external-buildbots/workers/ppc64le-clang-test-suite/clang-ppc64le-test-suite/build/./lib/../lib/clang/19/lib/powerpc64le-unknown-linux-gnu -lstdc++ -no-pie -latomic -m64 -fno-function-sections
/usr/bin/ld: RtsanNoInstTestObjects.gtest-all.cc.powerpc64le.o: undefined reference to symbol 'pthread_getspecific@@GLIBC_2.17'
//usr/lib64/libpthread.so.0: error adding symbols: DSO missing from command line
```
Follow up to #92460
DEPS llvm_gtest is not used by compiler-rt,
compiler-rt compiles them with COMPILER_RT_GOOGLETEST_SOURCES.
This reverts commit e217f98ac3ee55755ee6ea3b97551cf8283528c9.
Introducing the main runtime of realtime sanitizer. For more
information, please see the [discourse
thread](https://discourse.llvm.org/t/rfc-nolock-and-noalloc-attributes/76837).
We have also put together a [reviewer support
document](https://github.com/realtime-sanitizer/radsan/blob/doc/review-support/doc/review.md)
to show what our intention is.
This review introduces the sanitizer backend. This includes:
* CMake build files (largely adapted from asan).
* Main RTSan architecture (the external API, thread local context,
stack).
* Interceptors.
* Many unit tests.
Please see the [reviewer support
document](https://github.com/realtime-sanitizer/radsan/blob/doc/review-support/doc/review.md)
for what our next steps are. We are moving in lockstep with this PR
#84983 for the codegen coming up next.
Note to reviewers: If you see support documentation mention "RADSan",
this was the "old acronym" for the realtime sanitizer, they refer to the
same thing. If you see it let us know and we can correct it (especially
in the llvm codebase)
---------
Co-authored-by: David Trevelyan <david.trevelyan@gmail.com>
When using Clang as a compiler, use Clang to normalize the triple that's
used to construct path for runtime library build and install paths. This
ensures that paths are consistent and avoids the issue where the build
uses a different triple spelling.
Differential Revision: https://reviews.llvm.org/D140925
Currently we will not build float16/bfloat16 builtins on apple
platforms, LLVM will include calls to those builtins and fail to link.
I copied the test code we perform for non apple platforms because oddly
we branch on it being apple for building builtins
This diff contains the compiler-rt changes / preparations for nsan.
Test plan:
1. cd build/runtimes/runtimes-bins && ninja check-nsan
2. ninja check-all
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
This reverts commit fe82a3da36196157c0caa1ef2505186782f750d1.
This broke LLDB on MacOS due to a missing symbol during linking.
The fix has been applied in c6c08eee37bada190bd1aa4593c88a5e2c8cdaac.
Original commit message:
The terminfo dependency introduces a significant nonhermeticity into the
build. It doesn't respect `--no-undefined-version` meaning that it's not
a dependency that can be built with Clang 17+. This forces maintainers
of source-based distributions to implement patches or ignore linker
errors.
Remove it to reduce the closure size and improve portability of
LLVM-based tools. Users can still use command line arguments to toggle
color support expliticly.
Fixes#75490Closes#53294#23355
In https://github.com/llvm/llvm-project/pull/88323, I changed the logic
within `add_compiler_rt_runtime` to only explicitly code sign the
resulting library if an older version of Apple's ld64 was in use. This
was based on the assumption that newer versions of ld64 and the new
Apple linker always ad-hoc sign their output binaries. This is true in
most cases, but not when using Apple's new linker with the
`-darwin-target-variant` flag to build Mac binaries that are compatible
with Catalyst.
Rather than adding increasingly complicated logic to detect the exact
scenarios that require explicit code signing, I've opted to always
explicitly code sign when using any Apple linker. We instead detect and
use the 'linker-signed' codesigning option when possible to match the
signatures that the linker would otherwise create. This avoids having
non-'linker-signed' ad-hoc signatures which was the underlying problem
that https://github.com/llvm/llvm-project/pull/88323 was intended to
address.
Co-authored-by: Mark Rowe <markrowe@chromium.org>
The terminfo dependency introduces a significant nonhermeticity into the
build. It doesn't respect `--no-undefined-version` meaning that it's not
a dependency that can be built with Clang 17+. This forces maintainers
of source-based distributions to implement patches or ignore linker
errors.
Remove it to reduce the closure size and improve portability of
LLVM-based tools. Users can still use command line arguments to toggle
color support expliticly.
Fixes#75490Closes#53294#23355
`__sanitizer_siginfo` has been introduced in D142117.
(llvmorg-16-init-17950-ged9ef9b4f248)
It is incompatible to -pedantic.
`clang_rt.ctx_profile` has been introduced in #92456.
CMake configure compiler-rt got broken as a result of following commit:
d3925e65a7ab88eb0ba68d3ab79cd95db5629951
This patch fixes the break by porting the above commit for clang-cl.
This problem was not caught on Windows buildbots beacuase it appeared
when compiler-rt was included via LLVM_ENABLE_PROJECTS while buildbots
include compiler-rt project using LLVM_ENABLE_RUNTIMES flag.
This was added in def0823f1d2db78c4a18b15084407734a45e96c5 to fix a bot
failure, specifically for the standalone build. This does not seem to be
an issue anymore, and setting the policy explicitly to OLD causes
warnings with newer CMake versions.
This patch removes setting the policy to OLD to get rid of the warning.
If LLVM is configured with -DLLVM_DEFAULT_TARGET_TRIPLE, or compiler_rt
is configured with -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE, while the
argument is not normalized, such as Debian-style vendor-less triple,
clang will try to find libclang_rt in lib/<normalized_triple>, while
libclang_rt is placed into lib/<triple_arg>.
Let's also place libclang_rt into lib/<normalized_triple>.
`libcxx/utils/ci/run-buildbot` is also updated to use
`armv7m-none-unknown-eabi` as normalized triple instead of
current `armv7m-none-eabi`.
ChangeLog of this PR:
This patch has been applied and revert twice:
1. The first try is https://github.com/llvm/llvm-project/pull/88407, and
then it is found that it causes some CI failures.
https://lab.llvm.org/buildbot/#/builders/98/builds/36366
It is then resolved by another commit:
1693009679https://lab.llvm.org/buildbot/#/builders/77/builds/36372
It is caused that `COMPILER_RT_DEFAULT_TARGET_TRIPLE` is overwrite
without taking care about `CACHE`.
2. The second try https://github.com/llvm/llvm-project/pull/88835,
resolves https://lab.llvm.org/buildbot/#/builders/77/builds/36372
and in fact only one `execute_process` is needed.
Then we find some other CI failures.
https://github.com/mstorsjo/llvm-mingw/actions/runs/8730621159https://buildkite.com/llvm-project/libcxx-ci/builds/34897#018eec06-612c-47f1-9931-d3bd88bf7ced
It is due to misunderstanding `-print-effective-triple`: which will
output `thumbv7-w64-windows-gnu` for `armv7-w64-windows-gnu` or some
other thumb-enabled arm triples.
In fact we should use `-print-target-triple`.
For armv7m-picolibc, `armv7m-none-eabi` is hardcoded in
libcxx/utils/ci/run-buildbot, while in fact `armv7m-none-unknown-eabi`
is the real normalized triple.
This reverts commit 16f188761da1df6ba5e6627b8742aacfec8e9ec5.
This broke Libcxx Picolib testing at the install step, and builds
for Windows builtins. Revert while we figure out the cause.
If LLVM is configured with -DLLVM_DEFAULT_TARGET_TRIPLE, or compiler_rt
is configured with -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE, while the
argument is not normalized, such as Debian-style vendor-less triple,
clang will try to find libclang_rt in lib/<normalized_triple>, while
libclang_rt is placed into lib/<triple_arg>.
Let's also place libclang_rt into lib/<normalized_triple>.
