101 Commits

Author SHA1 Message Date
DavidKorczynski
3d7880b64c
[llvm-opt-fuzzer] Add more passes options (#76474)
The goal is to enable analysis of these passes by way of OSS-Fuzz, where
they currently lack code coverage

(https://storage.googleapis.com/oss-fuzz-coverage/llvm/reports/20231225/linux/src/llvm-project/llvm/lib/Transforms/Scalar/report.html)

---------

Signed-off-by: David Korczynski <david@adalogics.com>
2023-12-28 16:28:10 +01:00
Kazu Hirata
586ecdf205
[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
2023-12-11 21:01:36 -08:00
Nikita Popov
69365ae56f [RandomIRBuilder] Remove use of getNonOpaquePointerElementType() (NFC) 2023-07-14 10:54:38 +02:00
Nikita Popov
61e0822efa [llvm][clang] Remove uses of isOpaquePointerTy() (NFC)
This now always returns true (for pointer types).
2023-07-14 10:27:58 +02:00
Henry Yu
258cd1fc38 [FuzzMutate] Handle BB without predecessor, avoid insertion after musttail call, avoid sinking token type
FuzzMutate didn't consider some corner cases and leads to mutation failure when mutating some modules.
This patch fixes 3 bugs:

- Add null check when encountering basic blocks without predecessor to avoid segmentation fault
- Avoid insertion after `musttail call` instruction
- Avoid sinking token type

Unit tests are also added.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D151936
2023-06-01 19:52:16 -07:00
Henry Yu
6a28d3c21d [FuzzMutate] Avoid calling function with metadata/token parameter/return type for InsertFunctionStrategy
When there is a function with metadata/token parameter/return type, `InsertFunctionStrategy` will crash.

This patch fixes the problem by falling back to create function declaration when the sampled function contains metadata/token parameter/return type.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D150627
2023-06-01 13:59:47 -07:00
Zhenkai Weng
39b6a7f06e [FuzzMutate] Module size heuristics
IRMutation::mutateModule() currently requires the bitcode size of the module.
To compute the bitcode size, one way is to write the module to a buffer using
BitcodeWriter and calculating the buffer size. This would be fine for a single
mutation, but infeasible for repeated mutations due to the large overhead. It
turns out that the only IR strategy weight calculation method that depends on
the current module size is InstDeleterStrategy, which deletes instructions more
frequently as the module size approaches a given max size. However, there is no
real need for the size to be in bytes of bitcode, so we can use a different
metric. One alternative is to let the size be the number of objects in the
Module, including instructions, basic blocks, globals, and aliases. Although
getting the number of instructions is still O(n), it should have significantly
less overhead than BitcodeWriter. This suggestion would cause a change to the
IRMutator API, since IRMutator::mutateModule() can calculate the Module size
itself.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D149989
2023-05-09 13:58:10 -07:00
Zhenkai Weng
261db5fdbe [FuzzMutate] Make ShuffleBlockStrategy deterministic
This revision makes ShuffleBlockStrategy deterministic by replacing
SmallPtrSet with other data structures that has a deterministic iteration
order.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D149676
2023-05-04 20:54:25 -07:00
Mikael Holmen
a5eae04386 [FuzzMutate] Remove unused variable [NFC]
gcc warned with
 ../lib/FuzzMutate/RandomIRBuilder.cpp:319:28: warning: unused variable 'PtrTy' [-Wunused-variable]
   319 |           if (PointerType *PtrTy = dyn_cast<PointerType>(I.getType()))
       |                            ^~~~~
2023-05-04 09:58:05 +02:00
Henry Yu
66892f25af [FuzzMutate] Skip EHPad during mutation and avoid replacing callee with pointer when sinking
This patch addresses 2 problems:

- In `ShuffleBlockStrategy`, when `BB` is an EHPad, `BB.getFirstInsertionPt()` will return `BB.end()`, which cannot be dereferenced and will cause crash in following loop.
- In `isCompatibleReplacement`, a call instruction's callee might be replaced by a pointer, causing 2 subproblems:
  - we cannot guarantee that the pointer is a function pointer (even if it is, we cannot guarantee it matches the signature).
  - after such a replacement, `getCalledFunction` will from then on return `nullptr` (since it's indirect call) which causes Segmentation Fault in the lines below.

This patch fixes the first problem by checking if a block to be mutated is an EHPad in base class `IRMutationStrategy` and skipping mutating it if so.

This patch fixes the second problem by avoiding replacing callee with pointer and adding a null check for indirect calls.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D148853
2023-04-26 16:45:49 -07:00
Zhenkai Weng
1f67602222 [FuzzMutate] Correct type cast and add unit test for FCmp
This revision fixes an incorrect type cast from Instruction to ICmpInstr, which should have been to FCmpInstr instead. It turns out that StrategiesTest.cpp was missing a test case for InstModificationIRStrategy and FCmp, which is also now implemented in this revision. After this revision, [[ https://reviews.llvm.org/D148854 | llvm-stress in D148854 ]] no longer crashes randomly.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D148972
2023-04-25 17:29:00 -07:00
Peter Rong
0f9b1c8872 [FuzzMutate] Correct type cast and add unit test for FCmp
This revision fixes an incorrect type cast from Instruction to ICmpInstr, which should have been to FCmpInstr instead. It turns out that StrategiesTest.cpp was missing a test case for InstModificationIRStrategy and FCmp, which is also now implemented in this revision. After this revision, [[ https://reviews.llvm.org/D148854 | llvm-stress in D148854 ]] no longer crashes randomly.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D148972
2023-04-25 17:28:01 -07:00
Peter Rong
1a3f158add [FuzzMutate] Correct type cast and add unit test for FCmp
This revision fixes an incorrect type cast from Instruction to ICmpInstr, which should have been to FCmpInstr instead. It turns out that StrategiesTest.cpp was missing a test case for InstModificationIRStrategy and FCmp, which is also now implemented in this revision. After this revision, [[ https://reviews.llvm.org/D148854 | llvm-stress in D148854 ]] no longer crashes randomly.

Reviewed By: Peter

Differential Revision: https://reviews.llvm.org/D148972
2023-04-21 16:37:36 -07:00
Peter Rong
6998b34c7a [FuzzMutate] InsertFunctionStrategy
InsertFunctionStrategy does two things:

1. Add a random function declaration or definition to the module. This would replace previously used `createEmptyFunction`.
2. Add a random function call between instructions.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D148568
2023-04-19 20:09:12 -07:00
Peter Rong
64ce140fa6 [FuzzMutate] RandomIRBuilder has more source and sink type now.
Source and Sink are required when generating a new instruction.
(Term defined by previous author, in LLVM terms it's probably Use and User.)
Previously, only instructions in the same block is considered when taking source and sink.

In this patch, more source and sink types are considered.
For source, we have SrcFromInstInCurBlock, FunctionArgument, InstInDominator, SrcFromGlobalVariable, and NewConstOrStack.
For sink, we have SinkToInstInCurBlock, PointersInDominator, InstInDominatee, NewStore, and SinkToGlobalVariable.

A unit test to make sure source always dominates an instruction, and the instruction always dominates the sink is included.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139907
2023-04-17 14:45:09 -07:00
Peter Rong
a933f6003e Revert "[FuzzMutate] RandomIRBuilder has more source and sink type now."
This reverts commit e0117a3efacf9620408393f162a7795b5e0965d2.
2023-04-16 09:08:26 -07:00
NAKAMURA Takumi
077a2a4bcd [CMake] Cleanup deps 2023-04-17 00:38:49 +09:00
Peter Rong
e0117a3efa [FuzzMutate] RandomIRBuilder has more source and sink type now.
Source and Sink are required when generating a new instruction.
(Term defined by previous author, in LLVM terms it's probably Use and User.)
Previously, only instructions in the same block is considered when taking source and sink.

In this patch, more source and sink types are considered.
For source, we have SrcFromInstInCurBlock, FunctionArgument, InstInDominator, SrcFromGlobalVariable, and NewConstOrStack.
For sink, we have SinkToInstInCurBlock, PointersInDominator, InstInDominatee, NewStore, and SinkToGlobalVariable.

A unit test to make sure source always dominates an instruction, and the instruction always dominates the sink is included.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139907
2023-04-15 15:46:40 -07:00
Peter Rong
c06adaeba6 [FuzzMutate] introduce vector operations, select and fneg into InstInjectorStrategy
Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139894
2023-03-31 17:48:27 -07:00
NAKAMURA Takumi
0e18b5feaa LLVMFuzzerCLI: [CMake] Prune the last PARTIAL_SOURCES_INTENDED to cover all sources. 2023-02-12 20:12:37 +09:00
Archibald Elliott
62c7f035b4 [NFC][TargetParser] Remove llvm/ADT/Triple.h
I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.
2023-02-07 12:39:46 +00:00
Kazu Hirata
da6642a1c8 [llvm] Include <optional> instead of "llvm/ADT/Optional.h" (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-14 15:09:00 -08:00
serge-sans-paille
38818b60c5
Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part
Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955
2023-01-05 14:11:08 +01:00
Kazu Hirata
77c90c8ce0 [llvm] Use std::optional instead of Optional
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-20 15:42:32 -08:00
Archibald Elliott
f09cf34d00 [Support] Move TargetParsers to new component
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
  component into a new LLVM Component called "TargetParser". This
  potentially enables using tablegen to maintain this information, as
  is shown in https://reviews.llvm.org/D137517. This cannot currently
  be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
  information in the TargetParser:
  - `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
    the current Host machine for info about it, primarily to support
    getting the host triple, but also for `-mcpu=native` support in e.g.
    Clang. This is fairly tightly intertwined with the information in
    `X86TargetParser.h`, so keeping them in the same component makes
    sense.
  - `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
    the target triple parser and representation. This is very intertwined
    with the Arm target parser, because the arm architecture version
    appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.

And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM

Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.

If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.

Differential Revision: https://reviews.llvm.org/D137838
2022-12-20 11:05:50 +00:00
Peter Rong
2328c0e071 [FuzzMutate] InstModStrategy: switch nsw/nuw/inbount instead of repeated setting it
Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139890
2022-12-13 20:50:00 -08:00
Peter Rong
80760e9174 [FuzzMutate] InstModificationStrategy, add FastMath flags and exact flags to instructions.
I think there are more attributes, flags we can add to `call`, functions declarations and global variables. Let's start with these two flags.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139594
2022-12-12 15:22:13 -08:00
Peter Rong
bc277eb16b [FuzzMutate] New InsertCFGStrategy
Mutating CFG is hard as we have to maintain dominator relations.
We avoid this problem by inserting a CFG into a splitted block.

switch, ret, and br instructions are generated.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D139067
2022-12-12 15:21:07 -08:00
Kazu Hirata
aadaaface2 [llvm] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 21:11:44 -08:00
Peter Rong
4be0873471 [FuzzMutate] New InsertPHINode strategy.
PHI Node can't be modeled like other instructions since its operand
number depends on predecessors. So we have a stand alone strategy for it.

Signed-off-by: Peter Rong <PeterRong96@gmail.com>

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D138959
2022-11-30 15:26:52 -08:00
Peter Rong
43db7cb4f5 [FuzzMutate] SinkInstructionStrategy
Randomlly select an instruction and try to use it in the future by replacing it with another instruction's operand.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D138948
2022-11-29 16:18:55 -08:00
Peter Rong
50921a2174 [FuzzMutate] Fix a bug in connectToSink which might invalidate the whole module.
`connectToSink` uses a value by putting it in a future instruction.
It will replace the operand of a future instruction with the current value.

However, if current value is an `Instruction` and put into a switch case, the module is invalid.
We fix that by only connecting to Br/Switch's condition, and don't touch other operands.

Will have other strategies to mutate other Br/Switch operands to be patched once this patch is passed

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D138890
2022-11-29 13:07:26 -08:00
Peter Rong
a7def9f7f2 [FuzzMutate] New strategy ShuffleBlockStrategy
`ShuffleBlockStrategy` will shuffle the instructions in a basic block without breaking the dependency of instructions.
It is implemented as a topological sort, only we randomly select instructions with no dependency.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D138339
2022-11-28 17:57:26 -08:00
Peter Rong
23481bfe5a [FuzzMutate] Update InstModifierStrategy
We can randomly switch two operands of an instruction now

Signed-off-by: Peter Rong <PeterRong96@gmail.com>
2022-11-18 15:54:22 -08:00
Peter Rong
db2aa9f2d8 [FuzzMutate] change of format and comment for further code
Signed-off-by: Peter Rong <PeterRong96@gmail.com>
2022-11-18 10:58:05 -08:00
Sam McCall
e571e1a6c3 Reland "[FuzzMutate] Split out FuzzerCLI library that doesn't depend on IR."
This reverts commit a1bb952e833b34fdf03bd571e7f8c948191be018.

I'd somehow missed updating llvm-yaml-parser-fuzzer, now fixed.
2022-05-07 13:49:54 +02:00
Aaron Ballman
a1bb952e83 Revert "[FuzzMutate] Split out FuzzerCLI library that doesn't depend on IR."
This reverts commit 1c5e85b3da649c89db87abecc53b42f6eaa574c2.

It broke a lot of bots with a link error:
https://lab.llvm.org/buildbot/#/builders/171/builds/14222
https://lab.llvm.org/buildbot/#/builders/188/builds/13748
https://lab.llvm.org/buildbot/#/builders/109/builds/38127
2022-05-07 07:29:57 -04:00
Sam McCall
1c5e85b3da [FuzzMutate] Split out FuzzerCLI library that doesn't depend on IR.
All llvm-project fuzzers use this library to parse command-line arguments.
Many of them don't deal with LLVM IR or modules in any way. Bundling those
functions in one library forces build dependencies that don't need to be there.

Among other things, this means check-clang-pseudo no longer depends on most of
LLVM.

Differential Revision: https://reviews.llvm.org/D125081
2022-05-07 12:11:51 +02:00
Sam McCall
0a83ff83af [FuzzMutate] Move LLVM module (de)serialization from FuzzerCLI -> IRMutator. NFC
These are not directly related to the CLI, and are mostly (always?) used when
mutating the modules as part of fuzzing.

Motivation: split FuzzerCLI into its own library that does not depend on IR.
Subprojects that don't use IR should be be fuzzed without the dependency.

Differential Revision: https://reviews.llvm.org/D125080
2022-05-07 12:09:49 +02:00
Nikita Popov
0958450251 Revert "[FuzzMutate] Don't insert instructions after musttail call"
This reverts commit 6a23d2764467bd45c2e02828f6175a0b9f9a1005.

The newly added tests fail on the llvm-clang-x86_64-sie-win
buildbot. Not sure why a failure only occurs there, possibly
differen PRNG sequence?
2022-03-16 17:29:27 +01:00
Nikita Popov
6a23d27644 [FuzzMutate] Don't insert instructions after musttail call 2022-03-16 16:58:33 +01:00
serge-sans-paille
ed98c1b376 Cleanup includes: DebugInfo & CodeGen
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121332
2022-03-12 17:26:40 +01:00
Nikita Popov
853beb55e1 [FuzzMutate] Support scalable shufflevector 2022-03-11 14:20:40 +01:00
Nikita Popov
a3aac5693d [IRMutator] Handle module with only declarations
There was a mismatch here, with one check checking whether there
are any functions, and the other collecting only non-declaration
functions.
2022-03-11 14:20:39 +01:00
Nikita Popov
3c47dd47a4 [FuzzMutate] Support opaque pointers
Avoid checks that are irrelevant for opaque pointers, and pick
load/GEP types independently of the pointer type.

The GEP case at least could be done more efficiently by directly
generating a type, but this would require some significant API
changes.
2022-03-10 14:36:20 +01:00
Nikita Popov
aa97bc116d [NFC] Remove uses of PointerType::getElementType()
Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.
2022-01-25 09:44:52 +01:00
Bjorn Pettersson
3485e9bfbd [NewPM][FuzzMutate] Fix renaming 'unswitch' to 'simple-loop-unswitch'
This is a follow up to D105628 (commit 1db2551cc1a356a67c096). I had
missed to replace 'unswitch' by 'simple-loop-unswitch' in one place.
2021-07-09 12:24:12 +02:00
Bjorn Pettersson
472462c472 [NewPM] Consistently use 'simplifycfg' rather than 'simplify-cfg'
There was an alias between 'simplifycfg' and 'simplify-cfg' in the
PassRegistry. That was the original reason for this patch, which
effectively removes the alias.

This patch also replaces all occurrances of 'simplify-cfg'
by 'simplifycfg'. Reason for choosing that form for the name is
that it matches the DEBUG_TYPE for the pass, and the legacy PM name
and also how it is spelled out in other passes such as
'loop-simplifycfg', and in other options such as
'simplifycfg-merge-cond-stores'.

I for some reason the name should be changed to 'simplify-cfg' in
the future, then I think such a renaming should be more widely done
and not only impacting the PassRegistry.

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D105627
2021-07-09 09:47:03 +02:00
Justin Bogner
7e976cd456 [FuzzMutate] Fix getWeight of InstDeleterIRStrategy
The comment states the following, for calculating the Line variable:

> Draw a line starting from when we only have 1k left and increasing
> linearly to double the current weight.

However, the value was not calculated as described. Instead, it would
result in a negative value, which resulted in the function always
returning 0 afterwards.

```
// Invariant: CurrentSize <= MaxSize - 200
// Invariant: CurrentWeight >= 0
int Line = (-2 * CurrentWeight) * (MaxSize - CurrentSize + 1000);
// {Line <= 0}
```

This commit fixes the issue and linearly interpolates as described.

Patch by Loris Reiff. Thanks!

Differential Revision: https://reviews.llvm.org/D96207
2021-06-08 11:14:33 -07:00
Abhina Sreeskantharajan
c83cd8feef [NFC] Reordering parameters in getFile and getFileOrSTDIN
In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.

```
  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFile(const Twine &Filename, bool IsText = false,
          bool RequiresNullTerminator = true, bool IsVolatile = false);

  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFileOrSTDIN(const Twine &Filename, bool IsText = false,
                 bool RequiresNullTerminator = true);

 static ErrorOr<std::unique_ptr<MB>>
 getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
            bool IsText, bool RequiresNullTerminator, bool IsVolatile);

  static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
  getFile(const Twine &Filename, bool IsVolatile = false);
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D99182
2021-03-25 09:47:49 -04:00