Without this patch compiler-rt ubsan library has a bug displaying
incorrect values for variables of the _BitInt (previously called
_ExtInt) type. This patch affects affects both: generation of metadata
inside code generator and runtime part. The runtime part provided only
for i386 and x86_64 runtimes. Other runtimes should be updated to take
full benefit of this patch.
The patch is constructed the way to be backward compatible and int and
float type runtime diagnostics should be unaffected for not yet updated
runtimes.
This patch fixes issue
https://github.com/llvm/llvm-project/issues/64100.
Co-authored-by: Eänolituri Lómitaurë <vladislav.aranov@ericsson.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: Paul Kirth <paulkirth@google.com>
Fix profile/darwin-proof-of-concept.c on AppleSilicon Mac where there is
a different page alignment. The previous fix to drop alignment is
actually breaking the tests on Apple Silicon Mac. Revert to the original
section alignment and requires an ARM64 target for this test to run.
The nullness check is unreachable.
* For the main thead and pthread_create created threads, the `*Allocate` functions must be called after `*_current_thread` is set.
set.
* For threads created by Linux's `clone`, static TLS is either reused or
set to a new value (CLONE_SETTLS).
Make this change for asan/msan and possibly extend the change to other
sanitizers. (asan supports many platforms and I am not 100% certain that
all platforms have the property.)
Pull Request: https://github.com/llvm/llvm-project/pull/102828
…ImplID
This patch
1. remove the vendorId from `__riscv_vendor_feature_bits`
2. Define a new structure for vendorID, ArchID and ImplID
3. Update the relate init code
This patch add `void* PlatformArgs` parameter to
`__init_riscv_feature_bits`. `PlatformArgs` allows the platform to
provide pre-computed data and access it without extra effort. For
example, Linux could pass the vDSO object to avoid an extra system call.
```
__init_riscv_feature_bits()
->
__init_riscv_feature_bits(void *PlatformArgs)
```
In C, it is UB to call `memcmp(NULL, NULL, 0)`, `memcpy(NULL, NULL, 0)`,
etc. Unfortunately, `(NULL, 0)` is the natural representation of an
empty sequence of objects and extremely common in real world code. As a
result, all C code, and C++ code which calls into C functions, must
carefully guard all calls to `memcpy`.
This is a serious, real world usability issue in C and should be fixed
in the language (see #49459). In the meantime, pay the cost of the extra
branch to avoid tripping UBSan in libFuzzer. Once the usability problem
in C has been fixed, these checks can be removed.
Fixes#96772
When running tests in compiler-rt using the lit internal shell with the
following command:
```
LIT_USE_INTERNAL_SHELL=1 ninja check-compiler-rt
```
one common error that occurs is:
```
'XRAY_OPTIONS=patch_premain=false:verbosity=1': command not found
```
This error, along with over 50 similar "environment variable not found"
errors, appears across 35 files in complier-rt. These errors happen
because the environment variables are not being set correctly when the
internal shell is used, leading to commands failing due to unrecognized
environment variables.
This patch addresses the issue by using the `env` command to properly
set the environment variables before running the tests. By explicitly
setting the environment variables through `env`, the internal shell can
correctly interpret and apply them, allowing the tests to pass.
fixes: #102395
[link to
RFC](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179)
This patch adds the `REQUIRES: shell` directive to compiler-rt's fuzzer
tests that require full shell functionality when using the lit internal
shell. These tests depend on features such as background processes,
signal handling, and session management, which are not supported by
lit's internal shell. The addition of this directive ensures that these
tests are only executed in environments that provide the necessary shell
capabilities.
**Details of the Change:**
The following considerations were addressed:
- **Background Processes (`&`):** The tests run commands in the
background using the `&` operator, which allows the shell to execute
commands concurrently without waiting for each one to finish. In a
standard Unix-like shell, this is a common feature that facilitates
multitasking. However, lit's internal shell does not fully support
background job control, which can lead to unpredictable behavior or test
failures. Without proper handling of background processes, subsequent
commands that depend on the status of these processes may not function
correctly.
- **Signal Handling (`kill -SIGUSR1`, `kill -SIGUSR2`, `kill
-SIGINT`):** These tests involve sending specific signals like
`SIGUSR1`, `SIGUSR2`, and `SIGINT` to running processes. These signals
are used to trigger certain behaviors in the processes, such as pausing,
resuming, or terminating. However, lit's internal shell may not handle
these signals properly—it might not send the signal correctly, or the
process might not respond as it should. This could lead to the test
failing, not because the process is incorrect, but because the shell
didn't manage the signals as required.
- **Session Management (`setsid`):** The tests use `setsid` to create
new sessions, detaching processes from their controlling terminal and
isolating them into their own process groups. However, the internal
shell in lit does not support session management features like `setsid`,
this can't correctly isolate and manage processes as required by these
tests.
This change is relevant for enabling the lit internal shell by default,
as outlined in [[RFC] Enabling the Lit Internal Shell by
Default](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179)
* The performance is better than the glibc allocator.
* Allocator interface functions, sanitizer allocator options, and
MallocHooks/FreeHooks are supported.
* Shadow memory has specific memory layout requirement. Using libc
allocator could lead to conflicts.
* When we add a mmap interceptor for reliability (the VMA could reuse a
previously released VMA that is poisoned): glibc may invoke an
internal system call to call unmmap, which cannot be intercepted. We
will not be able to return the shadow memory to the OS.
Similar to dfsan https://reviews.llvm.org/D101204 . Also intercept
operator new/delete to be similar to other sanitizers using the
sanitizer allocator. The align_val_t overload of operator new has
slightly less overhead.
Pull Request: https://github.com/llvm/llvm-project/pull/102764
The nodes of list may be managed in an array. Instead of managing them
with pointers, using the array index will save some memory in some
cases.
When using the list linked with index, remember to call init() to set up
the base address of the array and a `EndOfListVal` which is nil of the
list.
This fixes a few of these warnings, when building with Clang ToT on
Windows:
```
[622/7618] Building CXX object
projects\compiler-rt\lib\sanitizer_common\CMakeFiles\RTSanitizerCommonSymbolizer.x86_64.dir\sanitizer_symbolizer_win.cpp.obj
C:\src\git\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_symbolizer_win.cpp(74,3):
warning: cast from 'FARPROC' (aka 'long long (*)()') to
'decltype(::StackWalk64) *' (aka 'int (*)(unsigned long, void *, void *,
_tagSTACKFRAME64 *, void *, int (*)(void *, unsigned long long, void *,
unsigned long, unsigned long *), void *(*)(void *, unsigned long long),
unsigned long long (*)(void *, unsigned long long), unsigned long long
(*)(void *, void *, _tagADDRESS64 *))') converts to incompatible
function type [-Wcast-function-type-mismatch]
```
This is similar to https://github.com/llvm/llvm-project/pull/97905
Intermittently on my mac I was getting the same nullptr crash in dlsym.
We need to make sure rtsan gets initialized on mac between when the
binary starts running, and the first intercepted function is called.
Until that point we should use the DlsymAllocator.
On thread creation, asan/hwasan/msan/tsan unpoison the thread stack and
static TLS blocks in case the blocks reuse previously freed memory that
is possibly poisoned. glibc nptl/allocatestack.c allocates thread stack
using a hidden, non-interceptable function.
nsan is similar: the shadow types for the thread stack and static TLS
blocks should be set to unknown, otherwise if the static TLS blocks
reuse previous shadow memory, and `*p += x` instead of `*p = x` is used
for the first assignment, the mismatching user and shadow memory could
lead to false positives.
NsanThread is also needed by the next patch to use the sanitizer
allocator.
Pull Request: https://github.com/llvm/llvm-project/pull/102718
Certain tests within the compiler-rt subproject encountered "command not
found" errors when using lit's internal shell, particularly when trying
to use the `DIR` environment variable. When checking with the command
`LIT_USE_INTERNAL_SHELL=1 ninja check-compiler-rt`, I encountered the
following error:
```
********************
Testing:
FAIL: SanitizerCommon-ubsan-i386-Linux :: sanitizer_coverage_trace_pc_guard-init.cpp (146 of 9570)
******************** TEST 'SanitizerCommon-ubsan-i386-Linux :: sanitizer_coverage_trace_pc_guard-init.cpp' FAILED ********************
Exit Code: 127
Command Output (stdout):
--
# RUN: at line 5
DIR=/usr/local/google/home/harinidonthula/llvm-project/build/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/ubsan-i386-Linux/Output/sanitizer_coverage_trace_pc_guard-init.cpp.tmp_workdir
# executed command: DIR=/usr/local/google/home/harinidonthula/llvm-project/build/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/ubsan-i386-Linux/Output/sanitizer_coverage_trace_pc_guard-init.cpp.tmp_workdir
# .---command stderr------------
# | 'DIR=/usr/local/google/home/harinidonthula/llvm-project/build/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/ubsan-i386-Linux/Output/sanitizer_coverage_trace_pc_guard-init.cpp.tmp_workdir': command not found
# `-----------------------------
# error: command failed with exit status: 127
```
In this patch, I resolved these issues by removing the use of the `DIR`
environment variable. Instead, the tests now directly utilize
`%t_workdir` for managing temporary directories. Additionally, I
simplified the tests by embedding the clang command arguments directly
into the test scripts, which avoids complications with environment
variable expansion under lit's internal shell.
This fix ensures that the tests run smoothly with lit's internal shell
and prevents the "command not found" errors, improving the reliability
of the test suite when executed in this environment.
fixes: #102395
[link to
RFC](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179)
Initially, the LRU list stored all mapped entries with no distinction
between the committed (non-madvise()'d) entries and decommitted
(madvise()'d) entries. Now these two types of entries re separated into
two lists, allowing future cache logic to branch depending on whether or
not entries are committed or decommitted. Furthermore, the retrieval
algorithm will prioritize committed entries over decommitted entries.
Specifically, committed entries that satisfy the MaxUnusedCachePages
requirement are retrieved before optimal-fit, decommitted entries.
This commit addresses the compiler errors raised
[here](https://github.com/llvm/llvm-project/pull/100818#issuecomment-2261043261).
This ports a fix from memprof (https://github.com/llvm/llvm-project/pull/98510), which has a shadow mapping that is similar to ASan (8 bytes of shadow memory per 64 bytes of app memory). This patch changes the allocator to dynamically choose a base address, as suggested by Vitaly for memprof. This simplifies ASan's #ifdef's and avoids potential conflict in the event that ASan were to switch to a dynamic shadow offset in the future [1].
[1] Since shadow memory is mapped before the allocator is mapped:
- dynamic shadow and fixed allocator (old memprof): could fail if
"unlucky" (e.g., https://lab.llvm.org/buildbot/#/builders/66/builds/1361/steps/17/logs/stdio)
- dynamic shadow and dynamic allocator (HWASan; current memprof): always works
- fixed shadow and fixed allocator (current ASan): always works, if
constants are carefully chosen
- fixed shadow and dynamic allocator (ASan with this patch): always works
The test fixture simplifies some of the logic for allocations and
mmap-based allocations
are separated from the cache to allow for more direct cache tests.
Additionally, a couple
of end to end tests for the cache and the LRU algorithm are added.
In these environments, the architecture name is armv7; recognize that
and enable the relevant runtimes.
Fix building the sanitizer_common library for this target, by using the
right registers for the architecture - this is similar to what
0c391133c9201ef29273554a1505ef855ce17668 did for aarch64.
(Still, address sanitizer doesn't support hooking functions at runtime
on armv7 or aarch64 - but other runtimes such as ubsan do work.)
The spec can be found at
https://github.com/riscv-non-isa/riscv-c-api-doc/pull/74.
1. Add the new extension GroupID/Bitmask with latest hwprobe key.
2. Update the `initRISCVFeature `
3. Update `EmitRISCVCpuSupports` due to not only group0 now.
The pair `__asan_before_dynamic_init` and `__asan_after_dynamic_init` is
executed for each TU. `__asan_after_dynamic_init` unpoisons all globals,
which
makes the time complexity O(N^2), where N is the maximum of the global
count and
the TU count. This is expensive for large binaries.
This patch decreases the time complexity to O(N), when lld and static
runtime is
used on SANITIZER_CAN_USE_PREINIT_ARRAY platforms. This requires:
Enabling incremental poisoning (`__asan_before_dynamic_init` since
https://github.com/llvm/llvm-project/pull/101597). Making most
`__asan_after_dynamic_init` calls do nothing.
