When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.
This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.
Differential Revision: https://reviews.llvm.org/D96762
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.
Reviewed-in: https://reviews.llvm.org/D96697
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi lsan msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Note: we need to place `_FILE_OFFSET_BITS` above `#include "sanitizer_platform.h"` to avoid `#define __USE_FILE_OFFSET64 1` in 32-bit ARM `features.h`
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
Suppress the warning:
```
'fake_shared_weak_count' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
```
The warning has been recently enabled [1], but the associated cleanup
missed this instance in Darwin code [2].
[1] 9c31e12609e1935eb84a2497ac08a49e3139859a
[2] d48f2d7c02743571075bb7812bb4c9e634e51ed1
Differential Revision: https://reviews.llvm.org/D94139
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
...and follow-ups. It still doesn't build on Android, see https://reviews.llvm.org/D93848#2476310
This reverts commit a92d01534f1c4fb79210975573e774d0393f2533.
This reverts commit 52d7e183bf25ea38e1149e39e19d21e6212e701f.
This reverts commit 34489da81b39972b40d2ff5581fe48911339406e.
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify no unneeded interceptors.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
rL254966 added `--sysroot=.` to prevent accidental including system headers.
It caused hassle to FreeBSD (D17383)/NetBSD. The next problem is that
we want to include `features.h` (usually `/usr/include/features.h`) to detect `__GLIBC__`.
At this point it seems that `--sysroot=.` adds lots of inconvenience so we disable it for now.
If there is a better way preventing accidental system header inclusion we can consider it again.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D93921
Disable the test on old systems.
pthread_cond_clockwait is supported by glibc-2.30.
It also supported by Android api 30 even though we
do not run tsan on Android.
Fixes https://github.com/google/sanitizers/issues/1259
Reviewed By: dvyukov
As discussed in the review for D87120 (specifically at
https://reviews.llvm.org/D87120#inline-831939), clean up PrintModuleMap
and DumpProcessMap usage differences. The former is only implemented for
Mac OSX, whereas the latter is implemented for all OSes. The former is
called by asan and tsan, and the latter by hwasan and now memprof, under
the same option. Simply rename the PrintModuleMap implementation for Mac
to DumpProcessMap, remove other empty PrintModuleMap implementations,
and convert asan/tsan to new name. The existing posix DumpProcessMap is
disabled for SANITIZER_MAC.
Differential Revision: https://reviews.llvm.org/D89630
- Fixing VS compiler and other cases settings this time.
Reviewers: dmajor, hans
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D89759
Revert "Fix compiler-rt build on Windows after D89640"
This reverts commit a7acee89d68473183cc5021d952a56cdf0ae27d3.
This reverts commit d09b08919ca6e206cd981fdea8b19b1d1188e325.
Reason: breaks Linux / x86_64 build.
Adds a check to avoid symbolization when printing stack traces if the
stack_trace_format flag does not need it. While there is a symbolize
flag that can be turned off to skip some of the symbolization,
SymbolizePC() still unconditionally looks up the module name and offset.
Avoid invoking SymbolizePC() at all if not needed.
This is an efficiency improvement when dumping all stack traces as part
of the memory profiler in D87120, for large stripped apps where we want
to symbolize as a post pass.
Differential Revision: https://reviews.llvm.org/D88361
TSan relies on C++ headers, so when libc++ is being built as part of
the runtimes build, include an explicit dependency on cxx-headers which
is the same approach that's already used for other sanitizers.
Differential Revision: https://reviews.llvm.org/D88912
D28596 added SANITIZER_INTERFACE_WEAK_DEF which can guarantee `*_default_options` are always defined.
The weak attributes on the `__{asan,lsan,msan,ubsan}_default_options` declarations can thus be removed.
`MaybeCall*DefaultOptions` no longer need nullptr checks, so their call sites can just be replaced by `__*_default_options`.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D87175
The interceptor for the block variants of the API references the
function versions (via `REAL(name##_f)`). On Linux, this accesses the
underlying "real pointer", defined by the interceptor macro. So we need
to declare interceptors in the right order to avoid undefined symbol
compiler error:
```
error: no member named 'real_dispatch_async_and_wait_f' in namespace '__tsan::__interception'
```
rdar://68181542
Handle NULL address argument in the `mach_vm_[de]allocate()`
interceptors and fix test: `Assignment 2` is not valid if we weren't
able to re-allocate memory.
rdar://67680613
`dispatch_async_and_wait()` was introduced in macOS 10.14, which is
greater than our minimal deployment target. We need to forward declare
it as a "weak import" to ensure we generate a weak reference so the TSan
dylib continues to work on older systems. We cannot simply `#include
<dispatch.h>` or use the Darwin availability macros since this file is
multi-platform.
In addition, we want to prevent building these interceptors at all when
building with older SDKs because linking always fails.
Before:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __interpose 0x000F5E68 pointer 0 libSystem _dispatch_async_and_wait_f
```
After:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __got 0x000EC0A8 pointer 0 libSystem _dispatch_async_and_wait (weak import)
__DATA __interpose 0x000F5E78 pointer 0 libSystem _dispatch_async_and_wait (weak import)
```
This is a follow-up to D85854 and should fix:
https://reviews.llvm.org/D85854#2221529
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D86103
The linker errors caused by this revision have been addressed.
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14. This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient). For TSan, we
can apply the same semantics as for `dispatch_sync()`.
From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.
Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057
rdar://35757961
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D85854
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14. This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient). For TSan, we
can apply the same semantics as for `dispatch_sync()`.
From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.
Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057
rdar://35757961
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D85854
These were no longer needed following
96855125e77044b1a5d3c7f0ae90ea3a5cb035c0.
Reviewed By: rupprecht
Differential Revision: https://reviews.llvm.org/D85939
Disable huge pages in the TSan shadow regions when no_huge_pages_for_shadow == true (default).
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D85841
Not matching the (real) variadic declaration makes the interceptor take garbage inputs on Darwin/AArch64.
Differential Revision: https://reviews.llvm.org/D84570
