shylie/llvm-project
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity
llvm-project/compiler-rt/lib/fuzzer/FuzzerDefs.h
Max Moroz f054067f27 [libFuzzer] Make -merge=1 to reuse coverage information from the control file. 
Summary:
This change allows to perform corpus merging in two steps. This is useful when
the user wants to address the following two points simultaneously:

1) Get trustworthy incremental stats for the coverage and corpus size changes
    when adding new corpus units.
2) Make sure the shorter units will be preferred when two or more units give the
    same unique signal (equivalent to the `REDUCE` logic).

This solution was brainstormed together with @kcc, hopefully it looks good to
the other people too. The proposed use case scenario:

1) We have a `fuzz_target` binary and `existing_corpus` directory.
2) We do fuzzing and write new units into the `new_corpus` directory.
3) We want to merge the new corpus into the existing corpus and satisfy the
    points mentioned above.
4) We create an empty directory `merged_corpus` and run the first merge step:

    `
    ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus
    `

    this provides the initial stats for `existing_corpus`, e.g. from the output:

    `
    MERGE-OUTER: 3 new files with 11 new features added; 11 new coverage edges
    `

5) We recreate `merged_corpus` directory and run the second merge step:

    `
    ./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ./new_corpus
    `

    this provides the final stats for the merged corpus, e.g. from the output:

    `
    MERGE-OUTER: 6 new files with 14 new features added; 14 new coverage edges
    `

Alternative solutions to this approach are:

A) Store precise coverage information for every unit (not only unique signal).
B) Execute the same two steps without reusing the control file.

Either of these would be suboptimal as it would impose an extra disk or CPU load
respectively, which is bad given the quadratic complexity in the worst case.

Tested on Linux, Mac, Windows.

Reviewers: morehouse, metzman, hctim, kcc

Reviewed By: morehouse

Subscribers: JDevlieghere, delcypher, mgrang, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66107

llvm-svn: 371620
2019-09-11 14:11:08 +00:00

210 lines
5.7 KiB
C++
Raw Blame History

//===- FuzzerDefs.h - Internal header for the Fuzzer ------------*- C++ -* ===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
// Basic definitions.
//===----------------------------------------------------------------------===//
#ifndef LLVM_FUZZER_DEFS_H
#define LLVM_FUZZER_DEFS_H
#include <cassert>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <memory>
#include <set>
#include <string>
#include <vector>
// Platform detection.
#ifdef __linux__
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 1
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 0
#elif __APPLE__
#define LIBFUZZER_APPLE 1
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 0
#elif __NetBSD__
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 1
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 0
#elif __FreeBSD__
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 1
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 0
#elif __OpenBSD__
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 1
#define LIBFUZZER_WINDOWS 0
#elif _WIN32
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 0
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 1
#elif __Fuchsia__
#define LIBFUZZER_APPLE 0
#define LIBFUZZER_FUCHSIA 1
#define LIBFUZZER_LINUX 0
#define LIBFUZZER_NETBSD 0
#define LIBFUZZER_FREEBSD 0
#define LIBFUZZER_OPENBSD 0
#define LIBFUZZER_WINDOWS 0
#else
#error "Support for your platform has not been implemented"
#endif
#if defined(_MSC_VER) && !defined(__clang__)
// MSVC compiler is being used.
#define LIBFUZZER_MSVC 1
#else
#define LIBFUZZER_MSVC 0
#endif
#ifndef __has_attribute
# define __has_attribute(x) 0
#endif
#define LIBFUZZER_POSIX \
(LIBFUZZER_APPLE || LIBFUZZER_LINUX || LIBFUZZER_NETBSD || \
LIBFUZZER_FREEBSD || LIBFUZZER_OPENBSD)
#ifdef __x86_64
# if __has_attribute(target)
# define ATTRIBUTE_TARGET_POPCNT __attribute__((target("popcnt")))
# else
# define ATTRIBUTE_TARGET_POPCNT
# endif
#else
# define ATTRIBUTE_TARGET_POPCNT
#endif
#ifdef __clang__ // avoid gcc warning.
# if __has_attribute(no_sanitize)
# define ATTRIBUTE_NO_SANITIZE_MEMORY __attribute__((no_sanitize("memory")))
# else
# define ATTRIBUTE_NO_SANITIZE_MEMORY
# endif
# define ALWAYS_INLINE __attribute__((always_inline))
#else
# define ATTRIBUTE_NO_SANITIZE_MEMORY
# define ALWAYS_INLINE
#endif // __clang__
#if LIBFUZZER_WINDOWS
#define ATTRIBUTE_NO_SANITIZE_ADDRESS
#else
#define ATTRIBUTE_NO_SANITIZE_ADDRESS __attribute__((no_sanitize_address))
#endif
#if LIBFUZZER_WINDOWS
#define ATTRIBUTE_ALIGNED(X) __declspec(align(X))
#define ATTRIBUTE_INTERFACE __declspec(dllexport)
// This is used for __sancov_lowest_stack which is needed for
// -fsanitize-coverage=stack-depth. That feature is not yet available on
// Windows, so make the symbol static to avoid linking errors.
#define ATTRIBUTES_INTERFACE_TLS_INITIAL_EXEC static
#define ATTRIBUTE_NOINLINE __declspec(noinline)
#else
#define ATTRIBUTE_ALIGNED(X) __attribute__((aligned(X)))
#define ATTRIBUTE_INTERFACE __attribute__((visibility("default")))
#define ATTRIBUTES_INTERFACE_TLS_INITIAL_EXEC \
ATTRIBUTE_INTERFACE __attribute__((tls_model("initial-exec"))) thread_local
#define ATTRIBUTE_NOINLINE __attribute__((noinline))
#endif
#if defined(__has_feature)
# if __has_feature(address_sanitizer)
# define ATTRIBUTE_NO_SANITIZE_ALL ATTRIBUTE_NO_SANITIZE_ADDRESS
# elif __has_feature(memory_sanitizer)
# define ATTRIBUTE_NO_SANITIZE_ALL ATTRIBUTE_NO_SANITIZE_MEMORY
# else
# define ATTRIBUTE_NO_SANITIZE_ALL
# endif
#else
# define ATTRIBUTE_NO_SANITIZE_ALL
#endif
namespace fuzzer {
template <class T> T Min(T a, T b) { return a < b ? a : b; }
template <class T> T Max(T a, T b) { return a > b ? a : b; }
class Random;
class Dictionary;
class DictionaryEntry;
class MutationDispatcher;
struct FuzzingOptions;
class InputCorpus;
struct InputInfo;
struct ExternalFunctions;
// Global interface to functions that may or may not be available.
extern ExternalFunctions *EF;
// We are using a custom allocator to give a different symbol name to STL
// containers in order to avoid ODR violations.
template<typename T>
class fuzzer_allocator: public std::allocator<T> {
public:
fuzzer_allocator() = default;
template<class U>
fuzzer_allocator(const fuzzer_allocator<U>&) {}
template<class Other>
struct rebind { typedef fuzzer_allocator<Other> other; };
};
template<typename T>
using Vector = std::vector<T, fuzzer_allocator<T>>;
template<typename T>
using Set = std::set<T, std::less<T>, fuzzer_allocator<T>>;
typedef Vector<uint8_t> Unit;
typedef Vector<Unit> UnitVector;
typedef int (*UserCallback)(const uint8_t *Data, size_t Size);
int FuzzerDriver(int *argc, char ***argv, UserCallback Callback);
uint8_t *ExtraCountersBegin();
uint8_t *ExtraCountersEnd();
void ClearExtraCounters();
extern bool RunningUserCallback;
} // namespace fuzzer
#endif // LLVM_FUZZER_DEFS_H
Reference in New Issue View Git Blame Copy Permalink