This adds a new option to do a faster of a purge.
When doing a release to OS due to a purge call, if another thread is
also doing a release, the call can be blocked while that operation
concludes. In some cases, code wants a fast version that releases as
fast as possible and the call will not block.
For example, on Android, when destroying a Bitmap a purge occurs to save
memory. But this can cause some jank if the purge takes too long.
In the future, I envision that this option will also do a calculation to
stop purging after some cutoff value to avoid being blocked in this call
for too long.
The hook passed to __sanitizer_thread_exit_hook will be null if we ever
call thrd/pthread_exit from the initial thread since
__sanitizer_before_thread_create_hook is never called on the initial
thread. Rather than using the hook, we can use the normal hwasan
machinery for getting the current pointer rather than the hook.
On AIX, when both the `__VEC__` and `_ALL_SOURCE` macros are defined
(they are defined by default), the malloc and calloc system calls are
mapped to vec_malloc and vec_calloc respectively, so we need the
following vec_malloc and vec_calloc interceptors.
Issue: #138916
Fixes missing `-I` path that broke standalone builds in #179011. Matches
`include_directories()` in other compiler-rt libraries.
Signed-off-by: Michał Górny <mgorny@gentoo.org>
This patch adds initial support for AMD Zen 6 architecture (znver6):
- Added znver6 CPU target recognition in Clang and LLVM
- Updated compiler-rt CPU model detection for znver6
- Added znver6 to target parser and host CPU detection
- Added znver6 to various optimizer tests
znver6 features: FP16, AVXVNNIINT8, AVXNECONVERT, AVXIFMA (without BMM).
The `__cxa_atexit` interceptor is disabled for `SANITIZER_AIX` because
Clang on AIX neither uses `__cxa_atexit` nor links against a library
with such. This interceptor calls `StopInitOrderChecking()`, which is
needed to prevent false positives for the `initialization-order-fiasco`
error observed in the asan test `init-order-atexit.cpp` that uses the
`strict_init_order` flag. For now, we'll disable the `strict_init_order`
flag, but we'll look to support it in the future by implementing an
`exit` interceptor or some other alternative. With the flag disabled, we
won't update `init-order-atexit.cpp` to ensure it continues to pass and
the false positive doesn't show up.
Currently, the AIX linker and loader do not provide a mechanism to
implement ifuncs similar to GNU_ifunc on ELF Linux.
On AIX, we will lower `__attribute__((ifunc("resolver"))` to the llvm
`ifunc` as other platforms do. The llvm `ifunc` in turn will get lowered
at late stages of the optimization pipeline to an AIX-specific
implementation. No special linkage or relocations are needed when
generating assembly/object output.
On AIX, a function `foo` has two symbols associated with it: a function
descriptor (`foo`) residing in the `.data` section, and an entry point
(`.foo`) residing in the `.text` section. The first field of the
descriptor is the address of the entry point. Typically, the address
field in the descriptor is initialized once: statically, at load time
(?), or at runtime if runtime linking is enabled.
Here we would like to use the address field in the descriptor to
implement the `ifunc` semantics. Specifically, the ifunc function will
become a stub that jumps to the entry point in the address field. A
constructor function is linked into every linkage module. The
constructor walks an array of `{descriptor, resolver}` pairs, calling
the resolver and saving the result in the address field in the
descriptor (thus setting `foo`'s descriptor to point to the resolved
version early during program runtime).
Known limitations:
- Due to bug #161576, which affects object generation path, you will
need either `-ffunction-sections` or `-fno-integrated-as` to generate a
correct/linkable object file.
- aliases to ifuncs are not supported, a testcase has been added and
marked XFAIL. I'm planning to address in a follow-up PR because it's not
important enough, IMHO, for this PR
- dead ifuncs in a CU that contains at least one live ifunc, will result
in all ifuncs being kept by the linker. The fix for this is common with
a similar problem we have with PGO. PR #159435 is trying to provide a
mechanism that will allow the ifunc and PGO implementations to avoid the
dead code retention at the link step.
- the resolver must return a function that is in the same DSO as the
ifunc; the compiler will try to detect if this condition is violated and
report it, but it cannot detect it in general. To be safe, all candidate
functions (returned by a particular resolver) must either be static or
have hidden/protected visibility. This is so that the ifunc stub doesn't
have to save and restore the TOC register r2. In future work, this case
will be supported and the requirement will be lifted.
---------
Co-authored-by: Wael Yehia <wyehia@ca.ibm.com>
This patch adds implementations for the __aeabi_uread and __aeabi_uwrite
helper functions to compiler-rt.
Without these helpers, LLVM would need to inline byte wise sequences ,
which can increases code size, especially at -Os/-Oz. Using the helper
functions allows to retain correctness while avoiding the code-size
growth.
GCC-based toolchains already provide these AEABI helpers, so supporting
them in compiler-rt ensures parity and avoids accidental dependencies on
libgcc when LLVM begins emitting these calls.
Some tools pass external_symbolizer_path="", which means it is not
expected to have any symbolizers in the list at the point we inserted a
(Darwin-only) warning.
This moves the symbolizer warning into ChooseExternalSymbolizer, after
the external_symbolizer_path="" case has been handled.
rdar://169137614
UnpoisonDefaultStack estimates the stack bottom as:
```
bottom = ((uptr)&local_stack - page_size) & ~(page_size - 1);
```
However, this can try to poision memory that doesn't have a shadow (i.e.
AddrIsInMem(x) is false) if the local_stack variable is within two pages
of the actual bottom of the space available for stack. This causes the
unpoison-alternate-stack.cpp test (which allocates a very small stack)
to fail.
This PR changes this computation to check AddrIsInMem before subtracting
the additional page.
rdar://82645815
When SVE is available use the `-sve` variant of memcpy from AOR for
`__arm_sc_memcpy`. From:
71e3640385/string/aarch64/memcpy-sve.S
This implementation uses FPR/ZPR load/store instructions to do the copy,
so should not cause memory hazards if called in streaming mode (with the
memory later being accessed in the streaming mode with SVE/SME
instructions).
The implementation has been slightly modified from AOR to use local
labels (matching other compiler-rt functions) but still passes the
memcpy and memmove tests from AOR.
Instead of setting a cache entry back to read-write based on whether MTE
is enabled, add a flag that indicates if this entry was made
inaccessible. On Android, we have seen strange crashes were MTE is
disabled for a process, but the cache entry is inaccessible, causing an
access error. This indicates MTE was enabled at some point, then
disabled again. There is likely some race, or method to disable without
making inaccessible entries read-writable again. Adding the flags should
avoid this happening in the future.
Remove the resetting of the secondary cache entries when disabling MTE
since it can be done as necessary.
This change will also allow us to always make the secondary cache
entries inaccessible in the future to increase security measures even
when MTE is not enabled.
Current unit tests already cover this change.
This PR fixes
1. issue #175509 about missing support of deinitialize on ELF platform.
2. missing support of execution order by proirity at both initialize and
deinitialize stage.
cc: @tqchen @joker-eph
Enabling pointer tagging in the userspace ABI for RISC-V kernels differs
to that of Aarch64. It requires requesting a particular number of masked
pointer bits, an error is returned if the platform could not accommodate
the request:
https://docs.kernel.org/arch/riscv/uabi.html#pointer-masking
While experimenting with enabling RISC-V HWASAN on GCC I was hitting the
error
> HWAddressSanitizer failed to enable tagged address syscall ABI
when attempting to run instrumented programs in the spike simulator
running kernel release 6.18. This patch successfully allows the tagged
address syscall ABI to be enabled by the support runtime.
Manually eliminate tail calls, because the compiler didn't.
This fixes some rare crashes (stack overflows) that can occur during
ASan's error reporting, if there is a deep nesting structure to thread
creation. Besides that, it has no change to ASan's output.
Currently giving an invalid value for this flag logs the error message
"Invalid value for signal handler option" - which is misleading.
This patch fixes that error message.
rdar://157565672
When building compiler-rt against an in-tree libc++, intermittent build
failures could occur: some rtsan sources referenced c++ stdlib includes
(i.e. `#include <new>`), but there was no CMake dependency enforcing
that the libc++ headers were copied into place before the rtsan sources
were built.
https://github.com/llvm/llvm-project/issues/127764#issue-2862550745
proposed adding the dependency on the `cxx-headers` target to resolve
the issue. But my understanding is that compiler-rt sanitizer runtimes
aren't expected to bring in C++ stdlib includes in the first place --
`SANITIZER_COMMON_CFLAGS`, used by the other sanitizer runtimes,
contains `-nostdinc++`.
Rtsan only included `<new>` to get support for placement new.
`sanitizer_common/sanitizer_placement_new.h` can provide this without
the need for C++ stdlib headers.
Move from `COMPILER_RT_COMMON_CFLAGS`/`COMPILER_RT_CXX_CFLAGS` to
`SANITIZER_COMMON_CFLAGS`, and from `COMPILER_RT_COMMON_LINK_FLAGS` to
`SANITIZER_COMMON_LINK_FLAGS` to be consistent with the other
sanitizers.
Fixes#127764.
Corrected various spelling mistakes such as 'occurred', 'receiver',
'initialized', 'length', and others in comments, variable names,
function names, and documentation throughout the project. These
changes improve code readability and maintain consistency in naming
and documentation.
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
On 64-bit AIX, set allocator size to 256G and set beginning to
0x0a00000000000000.
Issue: #138916
---------
Co-authored-by: Hubert Tong <hubert.reinterpretcast@gmail.com>
Because `__arm_za_disable` is a private-ZA function, it's only ever
entered with ZA state `off` or `dormant`. If the state is `off` then we
can safely return and there is no need to call `__arm_tpidr2_save` or to
explicitly set PSTATE.ZA or TPIDR2_EL0 to zero.
After #171941, there are two issues:
First, some buildbots that use the old-style build are failing at:
```
Target ${BUILTIN_LIB_TARGET_NAME} does not exist"
```
Example failure:
https://lab.llvm.org/buildbot/#/builders/139/builds/25097
...during CMake configure. This appears to be caused by mismatch between
the builtin library's _target_ name and the output name from
set_output_name. This reverts the change to BUILTIN_LIB_TARGET_NAME made
by #171941, but still use the output name for naming the .sources.txt
file used for configuring builtins tests.
Second, this speculatively fixes an issue caused by two builtins
libraries that are produced with the same name in different directories
because of `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` (e.g.
`lib/clang/22/lib/i386-unknown-linux-gnu/libclang_rt.builtins.a` and
`lib/clang/22/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a`), and
thus the `.sources.txt` paths alias. This causes us to run the wrong
tests against one of the builtins libraries. The chosen fix is to store
the .sources.txt files in `get_compiler_rt_output_dir` (which takes
`LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` into account) rather than
`CMAKE_BINARY_DIR`.
As a side-effect, this allows for the replacement of
`COMPILER_RT_TEST_BUILTINS_DIR` with a simpler boolean option
`COMPILER_RT_TEST_EXTERNAL_BUILTINS`.
Example failure:
https://lab.llvm.org/buildbot/#/builders/66/builds/24433
Adjust asan interceptor compatbility for AIX. AIX uses dlsym to retrieve
addresses of exported functions. However, some functions in libc.a, such
as memcpy, are not exported, so we currently have a limitation in
retrieving these addresses.
Issue: https://github.com/llvm/llvm-project/issues/138916
Some Go frames are wrapper functions that we don't want to report to
users. Add a mechanism for the Go runtime to tell the tsan runtime not
to report those frames.
For https://github.com/golang/go/issues/73915
This solves a stack-use-after-scope reported by AddressSanitizer within
the unsetenv call at end of the test, due to the "Options" buffer, that
we allocate on the stack, having already gone out of scope.
Unlike putenv, which stores the pointer to the passed string directly in
the environment, setenv creates an internal copy.
ReOptimizeLayer was building LLVM IR to define a precomputed,
SPS-serialized argument buffer, then inserting calls directly to
__orc_rt_jit_dispatch, passing the address of the precomputed buffer and
an __orc_rt_reoptimize_tag defined by the ORC runtime. This design is
non-canonical, requiring the ORC runtime to be loaded (or an extra
definition for __orc_rt_reoptimize_tag to be inserted) while not using
the runtime to perform the serialization.
This commit updates ReOptimizeLayer to instead insert calls to an
__orc_rt_reoptimize function implemented in the ORC runtime. This
function will perform serialization and call __orc_rt_jit_dispatch,
similar to other functions in the ORC runtime.
To maintain support for in-process JITs that don't use the ORC runtime,
this commit adds a ReOptimizeLayer::addOrcRTLiteSupport method which
injects IR to define __orc_rt_reoptimize (calling through to an
orc_rt_lite_reoptimize_helper function defined in LLVM) and
__orc_rt_reoptimize_tag. The ReOptimizeLayerTest is updated to use
addOrcRTLiteSupport.
When all the blocks of a page are unused, the page will be full of zero
and decommitted on operating systems that scan the memory.
Change-Id: I278055d82057090b0a04d812b49cf93fdf467478
Co-authored-by: Christopher Ferris <cferris1000@users.noreply.github.com>
When the `symbolize_inline_frames` option is set, we should use the `-i`
atos option and show inline frames.
The implementation is modeled after `ParseSymbolizePCOutput` and is now
quite close to that, but there are some subtle differences which I think
make it difficult to merge the implementations.
rdar://165894291
This PR is the first step towards introducing LFI into LLVM as a new
sub-architecture backend of AArch64. For details, please see the
[RFC](https://discourse.llvm.org/t/rfc-lightweight-fault-isolation-lfi-efficient-native-code-sandboxing-upstream-lfi-target-and-compiler-changes/88380),
which has been approved for AArch64.
This patch creates the `aarch64_lfi` architecture, and marks the
appropriate registers as reserved when it is targeted (`x25`, `x26`,
`x27`, `x28`). It also adds a Clang driver toolchain for targeting LFI,
and updates the compiler-rt CMake to allow builds for the `aarch64_lfi`
target. The patch also includes documentation for LFI and the rewrites
that will be implemented in future patches.
I am planning to split the relevant modifications for LFI into a series
of patches, organized as described below (after this one). Please let me
know if you'd like me to split the changes in a different way, or
provide one big patch.
1. The next patch will introduce the `MCLFIExpander` mechanism for
applying the MC-level rewrites needed by LFI, along with the
`.lfi_expand` and `.lfi_no_expand` assembly directives when targeting
LFI. A preview can be seen on the `lfi-project`
[fork](https://github.com/llvm/llvm-project/compare/main...lfi-project:llvm-project:lfi-patchset/aarch64-pr-2).
2. The following patch will create an `MCLFIExpander` for the AArch64
backend that performs LFI expansions. This patch will contain the
majority of the LFI-specific logic.
3. The final patch will add an optimization to the rewriter that can
eliminate redundant guard instructions that occur within the same basic
block.
We plan to introduce x86-64 support after further discussion and once
the `MCLFIExpander` infrastructure is in place.
Please let me know your feedback, and thank you very much for your help
and guidance in the review process.
This corrects a wrong condition for avx10 (AVX10Ver is always set to
0/1) and corrects how CPUID for avx10 is queried: per ISE table 1-3 we
should query with EAX = 0x24 and ECX = 0x0 -- previously we omitted the
latter.
Issue reported by user Seraphimt here
https://discourse.llvm.org/t/test-for-sys-gethostcpufeatures/89130
Fix#134853
Since the Windows path separator is typically `\\`, Here, we use `pathlib` to
ensure that the separator is in Unix format.
Additionally, since the file itself specifies the use of python3 (pathlib
requires python3), some python2 code has been removed.
