Summary:
1. Do not create DFSan labels for the bytes which we do not trace. This is where we run out of labels at the first place.
2. When dumping the traces on the disk, make sure to offset the label identifiers by the number of the first byte in the trace range.
3. For the last label, make sure to write it at the last position of the trace bit string, as that label represents the input size, not any particular byte.
Also fixed the bug with division in python which I've introduced when migrated the scripts to Python3 (`//` is required for integral division).
Otherwise, the scripts are wasting too much time unsuccessfully trying to
collect and process traces from the long inputs. For more context, see
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481761789
Reviewers: kcc
Reviewed By: kcc
Subscribers: delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60538
llvm-svn: 358311
Summary:
Before this change, MutateWithMask used to assert that Mask should be
of sufficient length (>= Size of the input). However, in real cases we may have
inputs that are longer than the Mask they have inherited from the based inputs.
Reviewers: kcc, morehouse
Reviewed By: kcc
Subscribers: delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60571
llvm-svn: 358207
Summary:
In case the current corpus input doesn't have bytes going into the
focus function, MutateWithMask is useless and may fail gently, allowing the
default mutation routine happen, rather than crashing on an assertion.
For more context and the initial fix suggestion, see:
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481862879
Reviewers: kcc, morehouse
Reviewed By: kcc
Subscribers: delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60567
llvm-svn: 358190
Summary:
As with other Sanitizers, and the current version of Scudo, we can
provide flags in differents way: at compile time, through a weak
function, through an environment variable.
This change adds support for the configuration flags, and the string
parsers. Those are fairly similar to the sanitizer_common way of doing
things.
Reviewers: morehouse, hctim, vitalybuka
Reviewed By: morehouse, vitalybuka
Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59597
llvm-svn: 358011
Summary:
This will make it easier to expand on the documentation in the future
that avoids cluttering the code.
rdar://problem/49476995
Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60398
llvm-svn: 357978
Summary:
The previous logging infrastructure had several problems:
* Debugging output was emitted to standard output which is also where
the symbolized output would go. Interleaving these two separate
bits of information makes inspecting the output difficult and could
potentially break tests.
* Enabling debugging output requires modifying the script which is
not very conveninent.
* When debugging it isn't immediately obvious where the output is
coming from.
This patch uses the Python standard library logging infrastructure
which fixes all of the above problems. Logging is controlled using
two new options.
* `--log-level` - Sets the logging level, default is
`info`.
* `--log-dest` - Set the logging destination, default
is standard error.
Some simple test cases for the feature are included.
rdar://problem/49476995
Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60343
llvm-svn: 357951
Summary:
r357240 added an interceptor for wctomb, which uses a temporary local
buffer to make sure we don't write to unallocated memory. This patch
applies the same technique to wcrtomb, and adds some additional tests
for this function.
Reviewers: vitalybuka, eugenis
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59984
llvm-svn: 357889
This is support for building compiler-rt builtins, The library build
should be complete for a subset of supported platforms, but not all
CMake options have been replicated in GN.
We always use the just built compiler to build all the runtimes, which
is equivalent to the CMake runtimes build. This simplifies the build
configuration because we don't need to support arbitrary host compiler
and can always assume the latest Clang. With GN's toolchain support,
this is significantly more efficient than the CMake runtimes build.
Differential Revision: https://reviews.llvm.org/D60331
llvm-svn: 357821
Summary:
When using libfuzzer on Windows, in the contents of a crash sample, bytes that can be mistaken for a \n are replaced by a \r\n sequence. As a consequence, crashes are not reproducible. This patch will open files in binary mode to fix this issue. The patch does not affect POSIX systems.
Patch by tuktuk
Reviewers: kcc, vitalybuka
Reviewed By: vitalybuka
Subscribers: dexonsmith, jdoerfert, llvm-commits, #sanitizers
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D60008
llvm-svn: 357807
Both LLVM 8.0.0 and current trunk fail to link libclang_rt.ubsan.standalone-*.so
on Solaris 11/x86 with GCC 8.1.0:
Undefined first referenced
symbol in file
Scanning dependencies of target clang_rt.asan_cxx-x86_64
__sanitizer::internal_dup(int) ../sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_posix.cc.o
ld: fatal: symbol referencing errors
collect2: error: ld returned 1 exit status
make[2]: *** [projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_standalone-dynamic-i386.dir/build.make:233: lib/clang/9.0.0/lib/sunos/libclang_rt.ubsan_standalone-i386.so] Error 1
Fixed by providing an implementation, which allowed the i386-pc-solaris2.11 build
to finish.
Differential Revision: https://reviews.llvm.org/D60045
llvm-svn: 357753
Both LLVM 8.0.0 and current trunk fail to compile on Solaris 11/x86 with
GCC 8.1.0:
/vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc: In function ‘void __sanitizer::ReExec()’:
/vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: error: ‘getexecname’ was not declared in this scope
pathname = getexecname();
^~~~~~~~~~~
/vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: note: suggested alternative: ‘gethostname’
pathname = getexecname();
^~~~~~~~~~~
gethostname
This is easily fixed by including <stdlib.h> which declares that function.
With that patch, compilation continues.
Differential Revision: https://reviews.llvm.org/D60044
llvm-svn: 357751
This avoids a conflict between stdbool.h, which defines bool to _Bool in
xkeycheck.h. From what I can tell, ymath.h is an internal header, and
the intention is that users should include math.h directly instead. It
doesn't appear to provide declarations of anything required for our
builtins. This include was added back in r249513 from 2015, and it's
possible that ymath.h provided something this code needed at the time,
but today it does not.
llvm-svn: 357728
llvm-ar is a crunchgen-style executable dispatching to dlltool,ranlib,lib,ar based on argv[0].
In our content-addressable storage, readlink -f resolves paths to some
digest and thus lost the original "llvm-ar" filename.
Replace it with a custom path resolution to fix the problem.
llvm-svn: 357450
Summary:
This is required to avoid msan false positives for code using this
function (although generally one should avoid using this function in
favor of wcrtomb).
Reviewers: eugenis, EricWF, vitalybuka
Subscribers: srhines, kubamracek, fedor.sergeev, delcypher, llvm-commits, #sanitizers
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59548
llvm-svn: 357240
Implement rounding mode support for addxf3/subxf3.
On architectures that implemented the support, this will access the
corresponding floating point environment register to apply the
correct rounding. For other architectures, it will keep the current
behaviour and use IEEE-754 default rounding mode (to nearest, ties
to even).
ARM32/AArch64 support implemented in this change. i386 and AMD64
will be added in a follow up change.
Differential Revision: https://reviews.llvm.org/D57143
llvm-svn: 357035
Also hide __cpu_inicator_init and __cpu_features2
for similar reasons.
Summary: Make __cpu_model a hidden symbol, to match libgcc.
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59561
llvm-svn: 356581
Summary:
This change adds fatal error messages for various error conditions that
will be added later in the code.
This also addresses a `TODO` now that we have `reportCheckFailed` (which
lead me to notice a few variables that were not cased properly so I
changed that as well).
Reviewers: morehouse, hctim, vitalybuka
Reviewed By: morehouse, hctim, vitalybuka
Subscribers: mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59551
llvm-svn: 356556
We were treating certain edge cases that are actually normal as denormal
results, and flushing them to zero; we shouldn't do that. Not sure this
is the cleanest way to implement this edge case, but I wanted to avoid
adding any code on the common path.
Differential Revision: https://reviews.llvm.org/D59070
llvm-svn: 356529
In libdispatch, dispatch_apply is implemented in terms of
dispatch_apply_f. Unfortunately, this means that we can't implement the
interceptor for dispatch_apply_f by forwarding to the dispatch_apply
interceptor.
In the interceptor dispatch_apply_f, we can't use WRAP(dispatch_apply).
WRAP(dispatch_apply) -> REAL(dispatch_apply_f). Requires duplication of
some setup code.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D59526
llvm-svn: 356467
Summary:
Add some string utility functions, notably to format strings, get
lengths, convert a string to a number. Those functions will be
used in reports and flags (coming up next). They were mostly borrowed
from sanitizer_common.
Make use of the string length function in a couple places in the
platform code that was checked in with inlined version of it.
Add some tests.
Reviewers: morehouse, eugenis, vitalybuka, hctim
Reviewed By: morehouse, vitalybuka
Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59262
llvm-svn: 356457
When COMPILER_RT_INTERCEPT_LIBDISPATCH is ON the TSan runtime library
now has a dependency on the blocks runtime and libdispatch. Make sure we
set all the required linking options.
Also add cmake options for specifying additional library paths to
instruct the linker where to search for libdispatch and the blocks
runtime. This allows us to build TSan runtime with libdispatch support
without installing those libraries into default linker library paths.
`CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY` is necessary to avoid
aborting the build due to failing the link step in CMake's
check_c_compiler test.
Reviewed By: dvyukov, kubamracek
Differential Revision: https://reviews.llvm.org/D59334
llvm-svn: 356281
Including <dispatch/dispatch.h> and <Blocks.h> transitively pulls in
other system headers. Let's try to avoid that.
Blocks.h: compiler-rt already includes a blocks runtime. Just use the
header file that comes with it.
dispatch.h: Declare the bare minimum required for our implementation,
i.e., everything needed to define the interceptors, but not the
interceptors themselves. See tsan_dispatch_defs.h. I spotted a few other
places in compile-rt, where we declare libdispatch types. Maybe this
file can be moved to sanitizer_common if we deem it useful enough.
tsan_libdispatch.cc now compiles on Linux/Clang (requires support for
-fblocks). Linking still requires some manual configuration.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D59145
llvm-svn: 356201
Summary:
Explicitly print 4 registers/line in each iteration during register
dump. Reduces logcat spam as we get a single logcat message per call to
Printf(), even if the output isn't newline-terminated. This brings the
output format in logcat closer to that of the normal textual dump.
Reviewers: eugenis, pcc
Reviewed By: pcc
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D59320
llvm-svn: 356166
Summary:
This patch contains the bits required to make the AddressSanitizer work on SPARC64/Linux (SPARC-T4 and later).
Patch by Eric Botcazou.
Reviewers: #sanitizers, vitalybuka, krytarowski
Reviewed By: #sanitizers, vitalybuka
Subscribers: brad, vitalybuka, ro, jyknight, kubamracek, fedor.sergeev, jdoerfert, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58434
llvm-svn: 355980
Summary:
This patch contains a fixlet for the AddressSanitizer on the SPARC with GCC, which would otherwise generate a problematic call to the intercepted memcpy routine. It was tested with GCC on SPARC/Solaris and SPARC/Linux.
Patch by Eric Botcazou.
Reviewers: #sanitizers, vitalybuka
Reviewed By: #sanitizers, vitalybuka
Subscribers: vitalybuka, ro, jyknight, kubamracek, fedor.sergeev, jdoerfert, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58433
llvm-svn: 355979
Summary:
This patch contains the bits required to make the common 32-bit allocator work on SPARC64/Linux.
Patch by Eric Botcazou.
Reviewers: #sanitizers, vitalybuka
Reviewed By: #sanitizers, vitalybuka
Subscribers: krytarowski, vitalybuka, ro, jyknight, kubamracek, fedor.sergeev, jdoerfert, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58432
llvm-svn: 355978
Summary:
This patch contains various fixes for the unwinding and backtrace machinery on the SPARC, which doesn't work correctly in various cases. It was tested with GCC on SPARC/Solaris and SPARC/Linux.
Patch by Eric Botcazou.
Reviewers: #sanitizers, vitalybuka
Reviewed By: #sanitizers, vitalybuka
Subscribers: jrtc27, delcypher, vitalybuka, ro, jyknight, kubamracek, fedor.sergeev, jdoerfert, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58431
llvm-svn: 355965
Summary:
This CL implements the checksumming functions. This departs from the
current Scudo code in one aspect: the software version is no longer
CRC32 but a BSD checksum. This is because the software CRC32 was too
impactful in terms of performances, the BSD checksum has no array
lookup which is better (and saves 1KB of data).
As with the current version, we only flip the CRC compiler flag for
a single compilation unit by default, to allow for a library compiled
with HW CRC32 to work on a system without HW CRC32.
There is some platform & hardware specific code in those files, but
since departs from a mere platform split, it felt right to me to have
it that way.
Reviewers: morehouse, eugenis, vitalybuka, hctim, mcgrathr
Reviewed By: morehouse
Subscribers: mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D59116
llvm-svn: 355923
It hasn't seen active development in years, and it hasn't reached a
state where it was useful.
Remove the code until someone is interested in working on it again.
Differential Revision: https://reviews.llvm.org/D59133
llvm-svn: 355862
Summary:
This change change the instrumentation to allow users to view the registers at the point at which tag mismatch occured. Most of the heavy lifting is done in the runtime library, where we save the registers to the stack and emit unwind information. This allows us to reduce the overhead, as very little additional work needs to be done in each __hwasan_check instance.
In this implementation, the fast path of __hwasan_check is unmodified. There are an additional 4 instructions (16B) emitted in the slow path in every __hwasan_check instance. This may increase binary size somewhat, but as most of the work is done in the runtime library, it's manageable.
The failure trace now contains a list of registers at the point of which the failure occured, in a format similar to that of Android's tombstones. It currently has the following format:
Registers where the failure occurred (pc 0x0055555561b4):
x0 0000000000000014 x1 0000007ffffff6c0 x2 1100007ffffff6d0 x3 12000056ffffe025
x4 0000007fff800000 x5 0000000000000014 x6 0000007fff800000 x7 0000000000000001
x8 12000056ffffe020 x9 0200007700000000 x10 0200007700000000 x11 0000000000000000
x12 0000007fffffdde0 x13 0000000000000000 x14 02b65b01f7a97490 x15 0000000000000000
x16 0000007fb77376b8 x17 0000000000000012 x18 0000007fb7ed6000 x19 0000005555556078
x20 0000007ffffff768 x21 0000007ffffff778 x22 0000000000000001 x23 0000000000000000
x24 0000000000000000 x25 0000000000000000 x26 0000000000000000 x27 0000000000000000
x28 0000000000000000 x29 0000007ffffff6f0 x30 00000055555561b4
... and prints after the dump of memory tags around the buggy address.
Every register is saved exactly as it was at the point where the tag mismatch occurs, with the exception of x16/x17. These registers are used in the tag mismatch calculation as scratch registers during __hwasan_check, and cannot be saved without affecting the fast path. As these registers are designated as scratch registers for linking, there should be no important information in them that could aid in debugging.
Reviewers: pcc, eugenis
Reviewed By: pcc, eugenis
Subscribers: srhines, kubamracek, mgorny, javed.absar, krytarowski, kristof.beyls, hiraditya, jdoerfert, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58857
llvm-svn: 355738
