3166 Commits

Author SHA1 Message Date
Vitaly Buka
d927d1867f
[UBSAN] Emit optimization remarks (#88304) 2024-04-10 16:30:42 -07:00
Evgenii Stepanov
e72c949c15
[msan] Overflow intrinsics. (#88210) 2024-04-10 09:12:25 -07:00
Vitaly Buka
0646344062
[HWASAN][UBSAN] Reverse random logic (#88070)
It feels more intuitive to make higher P to keep more checks.
2024-04-08 17:23:47 -07:00
Evgenii Stepanov
5bc87dac75 Revert "Overflow and saturating intrinsics (#88068)"
This reverts commit 118a5d8236d8a483dd401fa35c8b1fcd058eacc1.
2024-04-08 17:02:21 -07:00
Evgenii Stepanov
118a5d8236
Overflow and saturating intrinsics (#88068) 2024-04-08 16:33:45 -07:00
Vitaly Buka
49f0b536fd
[UBSAN] Rename remove-traps to lower-allow-check (#84853) 2024-04-04 21:29:46 -07:00
Vitaly Buka
a9d93873f8
[clang][ubsan] Switch UBSAN optimization to llvm.allow.{runtime,ubsan}.check() (#84858)
Intrinsic introduced with #84850. Intrinsics improves performance
by 3% comparing to removing traps (on
"test-suite/MultiSource/Benchmarks" with PGO+ThinLTO).

The pass will be renamed with #84853.

RFC:
https://discourse.llvm.org/t/rfc-add-llvm-experimental-hot-intrinsic-or-llvm-hot/77641
2024-04-04 17:40:02 -07:00
Vitaly Buka
aa6ba23235 [UBSAN] Remove invalid assert added with #87709 2024-04-04 14:52:31 -07:00
Vitaly Buka
18380c522a
[UBSAN][HWASAN] Remove redundant flags (#87709)
Presense of `cutoff-hot` or `random-skip-rate`
should be enough to trigger optimization.
2024-04-04 14:32:30 -07:00
Vitaly Buka
e628581aaa
[NFC][HWASAN][UBSAN] Remove cl:init from few opts (#87692)
They are supposed to be used with `getNumOccurrences`.
2024-04-04 14:30:04 -07:00
Vitaly Buka
03f54725c3
[HWASAN][UBSAN] Don't use default profile-summary-cutoff-hot (#87691)
Default cutoff is not usefull here. Decision to
enable or not sanitizer causes more significant
performance impact, than a typical optimizations
which rely on `profile-summary-cutoff-hot`.
2024-04-04 14:25:53 -07:00
Vitaly Buka
9a0ae08104
[NFC][HWASAN] Simplify selectiveInstrumentationShouldSkip (#87670) 2024-04-04 12:21:50 -07:00
Mingming Liu
1351d17826
[InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (#66825)
(The profile format change is split into a standalone change into https://github.com/llvm/llvm-project/pull/81691)

* For InstrFDO value profiling, implement instrumentation and lowering for virtual table address.
* This is controlled by `-enable-vtable-value-profiling` and off by default.
* When the option is on, raw profiles will carry serialized `VTableProfData` structs and compressed vtables as payloads.
 
* Implement profile reader and writer support 
  * Raw profile reader is used by `llvm-profdata` but not compiler. Raw profile reader will construct InstrProfSymtab with symbol names, and map profiled runtime address to vtable symbols.
  * Indexed profile reader is used by `llvm-profdata` and compiler. When initialized, the reader stores a pointer to the beginning of in-memory compressed vtable names and the length of string. When used in `llvm-profdata`, reader decompress the string to show symbols of a profiled site. When used in compiler, string decompression doesn't
happen since IR is used to construct InstrProfSymtab.
  * Indexed profile writer collects the list of vtable names, and stores that to index profiles.
  * Text profile reader and writer support are added but mostly follow the implementation for indirect-call value type.
* `llvm-profdata show -show-vtables <args> <profile>` is implemented.

rfc in
https://discourse.llvm.org/t/rfc-dynamic-type-profiling-and-optimizations-in-llvm/74600#pick-instrumentation-points-and-instrument-runtime-types-7
2024-04-01 08:52:35 -07:00
Florian Mayer
39e8137467
[NFC] [HWASan] add example for ring buffer wrap (#87029)
Also another move comment to correct place.
2024-03-28 18:23:48 -07:00
Vitaly Buka
70e8cf0c31
[HWASAN] Don't instrument loads from global if globals are not tagged (#86774) 2024-03-27 13:43:18 -07:00
Vitaly Buka
453a63caeb
[NFC][HWASAN] Promote InstrumentGlobals to member (#86773) 2024-03-27 13:38:21 -07:00
Florian Mayer
880eb339ca
[NFC] [HWASan] make getAndroidSlotPtr function generic (#86200)
This is so we can use a different slot for stack MTE.
2024-03-26 13:46:08 -07:00
Florian Mayer
b0b8b16bb8
[NFC] [HWASan] Add link to proof for address calculation (#86367) 2024-03-26 13:41:13 -07:00
Florian Mayer
85ccfb5ed5
[HWASan] [NFC] pull logic to get sanitizer ptr out of hwasan (#86024)
Also some drive by cleanup removing an unnnecessary argument and a
redundant condition.
2024-03-21 12:42:55 -07:00
Heejin Ahn
2b7289d48a
[PGO] Use isScopedEHPersonality for funclet check (#85671)
This line should be `isScopedEHPersonality` rather than
`isFuncletEHPersonality` because this line is used for checking whether
we need to add `funclet` op bundles to newly added calls, and Wasm EH
needs that too.

The new test case is adapted from

https://github.com/llvm/llvm-project/blob/main/llvm/test/Transforms/PGOProfile/memop_profile_funclet.ll.
2024-03-20 14:22:14 -07:00
Florian Mayer
e28d3e9b4e [NFC] [hwasan] factor get[PC|FP] out of HWASan class (#84404)
Also be consistent about naming SP / FP.

This is to prepare for stack history buffer for memtag-stack
2024-03-19 16:14:14 -07:00
Stephen Tozer
ffd08c7759
[RemoveDIs][NFC] Rename DPValue -> DbgVariableRecord (#85216)
This is the major rename patch that prior patches have built towards.
The DPValue class is being renamed to DbgVariableRecord, which reflects
the updated terminology for the "final" implementation of the RemoveDI
feature. This is a pure string substitution + clang-format patch. The
only manual component of this patch was determining where to perform
these string substitutions: `DPValue` and `DPV` are almost exclusively
used for DbgRecords, *except* for:

- llvm/lib/target, where 'DP' is used to mean double-precision, and so
appears as part of .td files and in variable names. NB: There is a
single existing use of `DPValue` here that refers to debug info, which
I've manually updated.
- llvm/tools/gold, where 'LDPV' is used as a prefix for symbol
visibility enums.

Outside of these places, I've applied several basic string
substitutions, with the intent that they only affect DbgRecord-related
identifiers; I've checked them as I went through to verify this, with
reasonable confidence that there are no unintended changes that slipped
through the cracks. The substitutions applied are all case-sensitive,
and are applied in the order shown:

```
  DPValue -> DbgVariableRecord
  DPVal -> DbgVarRec
  DPV -> DVR
```

Following the previous rename patches, it should be the case that there
are no instances of any of these strings that are meant to refer to the
general case of DbgRecords, or anything other than the DPValue class.
The idea behind this patch is therefore that pure string substitution is
correct in all cases as long as these assumptions hold.
2024-03-19 20:07:07 +00:00
Florian Mayer
c20596ca2b
[NFC] [hwasan] also be more consistent when getting pointer types (#84399) 2024-03-19 08:38:53 -07:00
Nikita Popov
8a237ab7d9 [TSan] Avoid use of ReplaceInstWithInst()
This is mainly for consistency across code paths, but also makes
sure that all calls use IRInstrumentationBuilder and its special
debuginfo handling.

The two remaining uses don't actually need RAUW, they just have
to erase the original instruction.
2024-03-15 09:21:27 +01:00
Nikita Popov
ff2fb2a1d7
[TSan] Fix atomicrmw xchg with pointer and floats (#85228)
atomicrmw xchg also accepts pointer and floating-point values. To handle
those, insert necessary casts to and from integer. This is what we do
for cmpxchg as well.

Fixes https://github.com/llvm/llvm-project/issues/85226.
2024-03-15 09:02:10 +01:00
Florian Mayer
58a20a0b96
[MTE] fix bug that prevented stack coloring with MTE (#84422) 2024-03-14 09:26:58 -07:00
Florian Mayer
0bb30f9896
[NFC] [hwasan] factor out some opt handling (#84414) 2024-03-13 10:56:26 -07:00
Florian Mayer
672fc89347
[NFC] [hwasan] factor out selective instrumentation logic (#84408)
sanitizeFunction is long enough already.
2024-03-11 18:18:49 -07:00
Florian Mayer
03c6c73b02 [NFC] [hwasan] remove unused method 2024-03-08 13:28:18 -08:00
sylvain-audi
ea12c1fa15
[Asan] Add "funclet" OpBundle to generated runtime calls if required by EH personality (#82533)
Previously, runtime calls introduced by ASan instrumentation into EH
pads were missing the funclet token expected by WinEHPrepare.
WinEHPrepare would then identify the containing BB as invalid and
discard it, causing invalid code generation that most likely crashes.

Also fixed localescape test, switching its EH personality to match code
without funclets.

This PR is based on the Phabricator patch
https://reviews.llvm.org/D143108

Fixes https://github.com/llvm/llvm-project/issues/64990
2024-03-08 12:29:15 -05:00
Florian Mayer
9d3bf9b639 [NFC] [hwasan] consistent naming for cl::opt 2024-03-07 16:42:41 -08:00
Jie Fu
ddf79deb42 [Asan] Fix -Wunused-private-field in non-assertion builds (NFC)
llvm-project/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp:650:13:
error: private field 'OwnerFn' is not used [-Werror,-Wunused-private-field]
  Function *OwnerFn = nullptr;
            ^
1 error generated.
2024-03-08 07:42:01 +08:00
Florian Mayer
3e5afba8ef
[NFC] [hwasan] be consistent about how to get integer types (#84396) 2024-03-07 15:41:12 -08:00
Vitaly Buka
a6a6fca791
[ubsan][pgo] Pass to remove ubsan checks based on profile data (#83471)
UBSAN checks can be too expensive to be used
in release binaries. However not all code affect
performace in the same way. Removing small
number of checks in hot code we can performance
loss, preserving most of the checks.
2024-03-07 10:52:43 -08:00
sylvain-audi
d6b3be375f
[NFC][Asan] Prepare AddressSanitizer to detect inserted runtime calls (#84223)
This is in preparation for an upcoming commit that will add "funclet"
OpBundle to the inserted runtime calls where the function's EH
personality requires it.

See PR https://github.com/llvm/llvm-project/pull/82533
2024-03-07 10:54:41 -05:00
Kirill Stoimenov
4258b0e13f
[HWASAN] Follow up for #83503 implement selective instrumentation (#83942)
1. Change tests to use IR instead of -stats to avoid depending on Debug
mode
2. Add SkipInstrumentationRandomRate 
3. Remove HWASAN from stat strings
2024-03-06 16:16:59 -08:00
Jeremy Morse
2fe81edef6 [NFC][RemoveDIs] Insert instruction using iterators in Transforms/
As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.

There are two general flavours of update:
 * Almost all call-sites just call getIterator on an instruction
 * Several make use of an existing iterator (scenarios where the code is
   actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.

Noteworthy changes:
 * FindInsertedValue now takes an optional iterator rather than an
   instruction pointer, as we need to always insert with iterators,
 * I've added a few iterator-taking versions of some value-tracking and
   DomTree methods -- they just unwrap the iterator. These are purely
   convenience methods to avoid extra syntax in some passes.
 * A few calls to getNextNode become std::next instead (to keep in the
   theme of using iterators for positions),
 * SeparateConstOffsetFromGEP has it's insertion-position field changed.
   Noteworthy because it's not a purely localised spelling change.

All this should be NFC.
2024-03-05 15:12:22 +00:00
Kirill Stoimenov
56754478e7 Reapply "[HWASAN] Implement selective instrumentation based on profiling information (#83503)"
Added REQUIRES: asserts, which should fix the build problem.

This reverts commit f6f79d46e580b34b2c98bd9bda7ede3a38f43f77.
2024-03-04 18:16:38 +00:00
Mingming Liu
87e7140f72
[nfc][InstrProfiling]For comdat setting helper function, move comment closer to the code (#83757) 2024-03-04 09:46:34 -08:00
Mingming Liu
f83858f87c
[nfc][InstrProfiling]Compute a boolean state as a constant and use it everywhere (#83756) 2024-03-04 08:56:54 -08:00
Mircea Trofin
ce95a56db5
[pgo][nfc] Model Count as a std::optional in PGOUseEdge (#83505)
Similar to PR #83364.
2024-03-01 09:44:22 -08:00
Kirill Stoimenov
f6f79d46e5 Revert "[HWASAN] Implement selective instrumentation based on profiling information (#83503)"
Broke a build bot: https://lab.llvm.org/buildbot/#/builders/124/builds/9846

This reverts commit e7c3cd245665042bbae163f7280aceed35f0fee5.
2024-03-01 17:17:37 +00:00
Kirill Stoimenov
e7c3cd2456
[HWASAN] Implement selective instrumentation based on profiling information (#83503) 2024-02-29 17:34:32 -08:00
Mircea Trofin
1a2960bab6
[pgo][nfc] Model Count as a std::optional in PGOUseBBInfo (#83364)
Simpler code, compared to tracking state of 2 variables and the ambiguity of "0" CountValue (is it 0 or is it invalid?)
2024-02-29 15:12:07 -08:00
Wu Yingcong
3250330997
[asan] Disable instrumentation for available_externally global with COFF (#81109)
For COFF, available_externally global will be instrumented because of
the lack of filtering, and will trigger the Verifier pass assertion and
crash the compilation. This patch will filter out the
available_externally global for COFF.

For non-COFF, `!G->hasExactDefinition()` in line 1954 will filter out
the available_externally globals.

There is a related bug reported in
https://bugs.llvm.org/show_bug.cgi?id=47950 /
https://github.com/llvm/llvm-project/issues/47294. I tried the
reproducer posted on the page and this will fix the problem.

Reproducer:
```
#include <locale>

void grouping_impl() {
  std::use_facet<std::numpunct<char>>(std::locale());
}

// clang -fsanitize=address -D_DLL -std=c++14 -c format.cc
```
2024-02-27 12:52:04 -08:00
NAKAMURA Takumi
cc53707a5c LLVMInstrumentation: Simplify mcdc.tvbitmap.update with GEP. 2024-02-25 11:21:46 +09:00
Vitaly Buka
699c408c88 [NFC][HWASAN] Fix misleading name 2024-02-21 15:32:18 -08:00
Arthur Eubanks
6841395953
[PGOForceFunctionAttrs] Don't mark alwaysinline function optnone (#81930)
optnone requires noinline, which is incompatible with alwaysinline.
2024-02-16 11:33:45 -08:00
Orlando Cazalet-Hyams
d860ea96b1
[HWASAN] Update dbg.assign intrinsics in HWAsan pass (#79864)
llvm.dbg.assign intrinsics have 2 {value, expression} pairs; fix hwasan to
update the second expression.

Fixes #76545. This is #78606 rebased and with the addition of DPValue handling.
Note the addition of --try-experimental-debuginfo-iterators in the tests and
some shuffling of code in MemoryTaggingSupport.cpp.
2024-02-13 09:11:09 +00:00
Arthur Eubanks
93cdd1b5cf
[PGO] Add ability to mark cold functions as optsize/minsize/optnone (#69030)
The performance of cold functions shouldn't matter too much, so if we
care about binary sizes, add an option to mark cold functions as
optsize/minsize for binary size, or optnone for compile times [1]. Clang
patch will be in a future patch.

This is intended to replace `shouldOptimizeForSize(Function&, ...)`.
We've seen multiple cases where calls to this expensive function, if not
careful, can blow up compile times. I will clean up users of that
function in a followup patch.

Initial version: https://reviews.llvm.org/D149800

[1]
https://discourse.llvm.org/t/rfc-new-feature-proposal-de-optimizing-cold-functions-using-pgo-info/56388
2024-02-12 14:52:08 -08:00