13607 Commits

Author SHA1 Message Date
Fangrui Song
58ecfccd0d [profile] Add __attribute__((used)) to zero size dummy sections
D14468 added these dummy sections. This patch adds `__attribute__((used))` so
that when compiled by GCC>=11 or (expected, D96838) Clang>=13 on some ELF platforms,
these sections will get SHF_GNU_RETAIN to make sure they will not be discarded
by ld --gc-sections.

We are trying to get rid of LLD's "__start_/__stop_ references retain C identifier name sections" rule.
If LLD drops the rule in the future (we will retain compatibility for `__llvm_prf_*` for a while),
`__llvm_prf_*` will need to have the SHF_GNU_RETAIN flag, otherwise:

```
// __llvm_prf_cnts/__llvm_prf_data usually exist, but {names,vnds} may not exist.
// Such diagnostics will happen with {cnts,data} as well if no input object file is instrumented.
% clang++ -fprofile-generate a.cc -fuse-ld=lld -Wl,--gc-sections
ld.lld: error: undefined hidden symbol: __start___llvm_prf_names
>>> referenced by InstrProfilingPlatformLinux.c
>>>               InstrProfilingPlatformLinux.c.o:(__llvm_profile_begin_names) in archive /tmp/RelA/lib/clang/13.0.0/lib/linux/libclang_rt.profile-x86_64.a
...
```

Differential Revision: https://reviews.llvm.org/D96902
2021-02-17 19:22:25 -08:00
Aaron Green
10993bf072 Bugfix for collecting features from very small DSOs.
During unit tests, it was observed that crafting an artificially small DSO could cause OOB memory to be accessed. This change fixes that (but again, the affected DSOs are unlikely to ever occur outside unit tests).

Reviewed By: morehouse, charco

Differential Revision: https://reviews.llvm.org/D94507
2021-02-17 13:04:49 -08:00
Dmitry Vyukov
fb19400d4e tsan: fix mmap_lots test
If tsan runtime will try to allocate something during exit handling,
the allocation will fail because there is no VA whatsoever.
It's observed to fail with the following error in some cases:
failed to allocate 0x1000 (4096) bytes of DTLS_NextBlock.
So terminate the process immediately.

Reviewed-in: https://reviews.llvm.org/D96874
2021-02-17 19:03:17 +01:00
Jonny Farley
f8ed31cd99 [Fuzzer][Test] Use %python substitution for trace-malloc-unbalanced.test
This test was found to fail for some of our downstream builds, on
computers where python was not on the default $PATH. Therefore
add a %python substitution to use sys.executable, based on similar
solutions for python calls in tests elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D96799
2021-02-17 15:00:46 +00:00
Alex Richardson
ca9815fc24 [sanitizers] Define SANITIZER_INTERCEPTOR_HOOKS on FreeBSD
This fixes the weak_hooks.cpp test on FreeBSD. Since this feature appears
to be supported on almost all platforms, it might also make sense to turn
it into an opt-out list instead of being opt-in.

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D96255
2021-02-17 10:36:47 +00:00
Joachim Protze
ac6c13bfc4 [sanitizers] Pass CMAKE_C_FLAGS into TSan buildgo script
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.

This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.

Differential Revision: https://reviews.llvm.org/D96762
2021-02-17 09:14:23 +01:00
Dmitry Vyukov
0984b8de0b tsan: don't leave unmapped hole in non-app memory
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.

Reviewed-in: https://reviews.llvm.org/D96697
2021-02-17 08:37:04 +01:00
Adhemerval Zanella
b9d3234c70 [sanitizer] [arm] Disable some LSAN tests for arm-linux-gnueabihf
Reinstate D90628 since the fix done by D96337 does not change the outcome
of the https://bugs.llvm.org/show_bug.cgi?id=48052
2021-02-15 09:35:02 -03:00
Amy Huang
60a55337e0 Fix test in external_symbolizer_path.cpp, by adding a REQUIRES: static-lib.
Follow-up to https://reviews.llvm.org/D94563.
2021-02-12 14:04:43 -08:00
Amy Huang
1e92b1730f Fix test failure for external_symbolizer_path.cpp 2021-02-12 12:49:41 -08:00
Amy Huang
5815b71eac Disable test in external_symbolizer_path.cpp temporarily to debug test failures. 2021-02-12 11:51:13 -08:00
Marco Vanotti
0fe4701e51 Expand unit tests for fuzzer::Merger
This change adds additional unit tests for fuzzer::Merger::Parse and fuzzer::Merger::Merge in anticipation of additional changes to the merge control file format to support cross-process fuzzing.

It modifies the parameter handling of Merge slightly in order to make NewFeatures and NewCov consistent with NewFiles; namely, Merge *replaces* the contents of these output parameters rather than accumulating them (thereby fixing a buggy return value).

This is change 1 of (at least) 18 for cross-process fuzzing support.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D94506
2021-02-12 11:18:34 -08:00
Amy Huang
394913fdb9 Try to fix external_symbolizer_path.cpp test to stop breaking on buildbots.
Not sure what the issue is, but it might be because the test copies
llvm-symbolizer to a different directory, and it can't find libc++.
Try to add some REQUIRES that we use in other tests where we copy
llvm tools out of their original directories.
2021-02-12 10:16:49 -08:00
Jianzhou Zhao
a7538fee3a [dfsan] Comment out ChainOrigin temporarily
It was added by D96160, will be used by D96564.
Some OS got errors if it is not used.
Comment it out for the time being.
2021-02-12 18:13:24 +00:00
Matthew G McGovern
81b1d3da09 [sanitizers][Windows] Implement __sanitizer_purge_allocator for Win64
Windows' memory unmapping has to be explicit, there is no madvise.
Similarly, re-mapping memory has to be explicit as well. This patch
implements a basic method for remapping memory which was previously
returned to the OS on Windows.

Patch by Matthew G. McGovern and Jordyn Puryear
2021-02-12 09:49:04 -08:00
Jianzhou Zhao
7590c0078d [dfsan] Turn off THP at dfsan_flush
https://reviews.llvm.org/D89662 turned this off at dfsan_init.
dfsan_flush also needs to turn it off.
W/o this a program may get more and more memory usage after hours.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96569
2021-02-12 17:10:09 +00:00
Vitaly Buka
0b3d31222d [ASAN][NFC] Improve language 2021-02-12 02:55:58 -08:00
Vitaly Buka
fc05b2d9e5 [NFC][ProfileData] Improve language 2021-02-12 02:55:58 -08:00
Jianzhou Zhao
083d45b21c [dfsan] Fix building OriginAddr at non-linux OS
Fix the broken build by D96545
2021-02-12 05:02:14 +00:00
Vitaly Buka
f2133f2e31 [NFC,memprof] Update test after D96319 2021-02-11 16:36:16 -08:00
Julian Lettner
9360f1a191 [Sanitizer] Fix sanitizer tests without reducing optimization levels
As discussed, these tests are compiled with optimization to mimic real
sanitizer usage [1].

Let's mark relevant functions with `noinline` so we can continue to
check against the stack traces in the report.

[1] https://reviews.llvm.org/D96198

This reverts commit 04af72c5423eb5ff7c0deba2d08cb46d583bb9d4.

Differential Revision: https://reviews.llvm.org/D96357
2021-02-11 15:22:20 -08:00
Jianzhou Zhao
5ebbc5802f [dfsan] Introduce memory mapping for origin tracking
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96545
2021-02-11 22:33:16 +00:00
Jianzhou Zhao
2d9c6e10e9 [dfsan] Add origin chain utils
This is a part of https://reviews.llvm.org/D95835.

The design is based on MSan origin chains.

An 4-byte origin is a hash of an origin chain. An origin chain is a
pair of a stack hash id and a hash to its previous origin chain. 0 means
no previous origin chains exist. We limit the length of a chain to be
16. With origin_history_size = 0, the limit is removed.

The change does not have any test cases yet. The following change
will be adding test cases when the APIs are used.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96160
2021-02-11 19:10:11 +00:00
Fangrui Song
0dc6122dd3 [asan][test] Fix Linux/odr-violation.cpp on gcc 2021-02-11 09:53:48 -08:00
Adhemerval Zanella
88d1724d9b [sanitizer] Fix suffix-log-path_test.c on arm-linux-gnu
The recent suffix-log-path_test.c checks for a full stacktrace and
since on some arm-linux-gnu configuration the slow unwinder is used
on default (when the compiler emits thumb code as default), it
requires -funwind-tables on tests.

It also seems to fix the issues disable by d025df3c1de.

Reviewed By: ostannard

Differential Revision: https://reviews.llvm.org/D96337
2021-02-11 14:26:04 -03:00
Jianzhou Zhao
b4993cf54d [sanitizer] Move MSan's chained_origin_depot to sanitizer_common
https://reviews.llvm.org/D95835 implements origin tracking for DFSan.
It reuses the chained origin depot of MSan.

This change moves the utility to sanitizer_common to share between
MSan and DFSan.

Reviewed-by: eugenis, morehouse

Differential Revision: https://reviews.llvm.org/D96319
2021-02-11 01:25:56 +00:00
Vedant Kumar
897bec937e [test/ubsan] Allow unused FileCheck prefixes in TypeCheck/vptr.cpp
This test started failing after https://reviews.llvm.org/D95849
defaulted --allow-unused-prefixes to false.

Taking a look at the test, I didn't see an obvious need to add
OS-specific check lines for each supported value of %os.

rdar://74207657
2021-02-10 16:06:51 -08:00
Fangrui Song
096f547208 [asan][test] Fix Linux/odr-violation.cpp after symbolizer change D95927 2021-02-10 14:27:12 -08:00
Mitch Phillips
b93786907c [GWP-ASan] Add back some headers removed by IWYU.
These headers are required for Android.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D96374
2021-02-10 09:13:45 -08:00
Todd Lipcon
5dd29d9922
Fix xray fdr mode to allow multiple flushes
Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D96382
2021-02-10 12:57:24 +11:00
Nico Weber
23cd8d51ad Try to fix compiler-rt tests after 87dbdd2e3bb63b on mac and win
These tests use `--check-prefix=CHECK-%os` but then didn't have
a CHECK line for every os.

In most tests, the linux expectations were sufficient (they match
the "wrap_" prefix with .*), so just remove the check-prefix there.
In the places where this didn't easily work, make sure there are
at least CHECK-Windows and CHECK-Darwin lines.
2021-02-09 09:28:52 -05:00
Roland McGrath
4c9adbb287 [scudo/standalone] Use .arch_extension memtag, not mte
GNU binutils accepts only `.arch_extension memtag` while Clang
accepts either that or `.arch_extension mte` to mean the same thing.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D95996
2021-02-08 12:24:47 -08:00
Mitch Phillips
3d8823b8e4 [GWP-ASan] Add aligned allocations.
Adds a new allocation API to GWP-ASan that handles size+alignment
restrictions.

Reviewed By: cryptoad, eugenis

Differential Revision: https://reviews.llvm.org/D94830
2021-02-08 11:22:29 -08:00
Julian Lettner
04af72c542 [Sanitizer] Fix failing sanitizer tests
The new pass manager was enabled by default [1].

The commit message states the following relevant differences:
  * The inliner works slightly differently
  * -O1 does some amount of inlining

These tests are affected because they specify `-O1` and then check the
reported stack trace.

[1] https://reviews.llvm.org/D95380

Differential Revision: https://reviews.llvm.org/D96198
2021-02-08 09:56:32 -08:00
Jianzhou Zhao
0f3fd3b281 [dfsan] Add thread registration
This is a part of https://reviews.llvm.org/D95835.

This change is to address two problems
1) When recording stacks in origin tracking, libunwind is not async signal safe. Inside signal callbacks, we need
to use fast unwind. Fast unwind needs threads
2) StackDepot used by origin tracking is not async signal safe, we set a flag per thread inside
a signal callback to prevent from using it.

The thread registration is similar to ASan and MSan.

Related MSan changes are
* 98f5ea0dba
* f653cda269
* 5a7c364343

Some changes in the diff are used in the next diffs
1) The test case pthread.c is not very interesting for now. It will be
  extended to test origin tracking later.
2) DFsanThread::InSignalHandler will be used by origin tracking later.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95963
2021-02-05 17:38:59 +00:00
Emily Shi
039567b664 [Darwin] Switch to new logging api for sanitizers
Switch to new logging api added in [[ https://developer.apple.com/documentation/os/os_log_error | macOS 10.12 ]] that is more memory safe and enables us to label the log messages in the future. Falls back to old API if ran on older OS versions.

Commited by Dan Liew on behalf of Emily Shi.

rdar://25181524

Reviewed By: delcypher, yln

Differential Revision: https://reviews.llvm.org/D95977
2021-02-04 21:04:51 -08:00
Vitaly Buka
758928e3b8 [sanitizer] Fix the test on android 2021-02-04 18:02:02 -08:00
Amy Huang
8d7d2deb37 Fix test case from D94563.
The added test case failed on ppc, android, and other buildbots,
so require x86 targets.
2021-02-04 16:34:39 -08:00
Amy Huang
4737aab53d Fix a test case after committing D94563.
D94563 implemented `ReadBinaryName` on Windows, which causes a test case
to now pass, so remove the `XFAIL: windows-msvc` line.
2021-02-04 16:06:51 -08:00
Amy Huang
9ba623c655 [asan] Add %d variable to external_symbolizer_path option, so that user can specify paths relative to the location of the binary.
We want way to set a path to llvm-symbolizer that isn't relative
to the current working directory; this change adds a variable that
expands to the path relative to the current binary.
This approach came from comments in https://reviews.llvm.org/D93070

Differential Revision: https://reviews.llvm.org/D94563
2021-02-04 15:43:02 -08:00
Bill Torpey
dd5c2b8de9 [sanitizer] Add suffix to report file name
For those using a GUI, it can be very helpful to have a
particular suffix appended to the report file name, so
it can be opened with a double-click.

(see also: https://github.com/google/sanitizers/issues/951)

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46546
2021-02-04 13:50:11 -08:00
Nico Weber
b995314143 Revert "[InstrProfiling] Use !associated metadata for counters, data and values"
This reverts commit 97ba5cde52664200819446c1a18de28faf2ed1c6.
Still breaks tests: https://reviews.llvm.org/D76802#2540647
2021-02-03 19:14:34 -05:00
Vitaly Buka
9da05cf6ed [asan] Fix pthread_create interceptor
AsanThread::Destroy implementation expected to be called on
child thread.

I missed authors concern regarding this reviewing D95184.

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D95731
2021-02-03 12:57:56 -08:00
Jianzhou Zhao
15f26c5f51 [dfsan] Wrap strcat
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95923
2021-02-03 18:50:29 +00:00
Jianzhou Zhao
eb5c0a90e7 [dfsan] Test IGN and DFL for sigaction
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95957
2021-02-03 18:46:49 +00:00
Roland McGrath
15aa78abb6 [sanitizer_common] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: phosek, charco

Differential Revision: https://reviews.llvm.org/D95919
2021-02-03 10:45:46 -08:00
Roland McGrath
09fe23a61c [gwp_asan] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D95920
2021-02-03 10:35:07 -08:00
Roland McGrath
d81069e796 [scudo/standalone] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D95921
2021-02-03 10:28:05 -08:00
Jianzhou Zhao
93afc3452c [dfsan] Clean TLS after signal callbacks
Similar to https://reviews.llvm.org/D95642, this diff fixes signal.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95896
2021-02-03 17:21:28 +00:00
Petr Hosek
97ba5cde52 [InstrProfiling] Use !associated metadata for counters, data and values
C identifier name input sections such as __llvm_prf_* are GC roots so
they cannot be discarded. In LLD, the SHF_LINK_ORDER flag overrides the
C identifier name semantics.

The !associated metadata may be attached to a global object declaration
with a single argument that references another global object, and it
gets lowered to SHF_LINK_ORDER flag. When a function symbol is discarded
by the linker, setting up !associated metadata allows linker to discard
counters, data and values associated with that function symbol.

Note that !associated metadata is only supported by ELF, it does not have
any effect on non-ELF targets.

Differential Revision: https://reviews.llvm.org/D76802
2021-02-02 23:19:51 -08:00