7336 Commits

Author SHA1 Message Date
Matt Arsenault
f5c8242042
SimplifyLibCalls: Prefer to emit intrinsic in pow(2, x) -> ldexp(1, x) (#92363) 2024-05-17 14:28:03 +02:00
Jie Fu
e948da1021 [Transforms] Fix -Wunused-variable in DemoteRegToStack.cpp (NFC)
llvm-project/llvm/lib/Transforms/Utils/DemoteRegToStack.cpp:58:21:
error: unused variable 'BB' [-Werror,-Wunused-variable]
        BasicBlock *BB = SplitCriticalEdge(II, i);
                    ^
1 error generated.
2024-05-16 20:52:56 +08:00
Jie Fu
03d8e61391 [Transforms] Fix -Wsign-compare in DemoteRegToStack.cpp (NFC)
llvm-project/llvm/lib/Transforms/Utils/DemoteRegToStack.cpp:54:23:
error: comparison of integers of different signs: 'int' and 'unsigned int' [-Werror,-Wsign-compare]
    for (int i = 0; i < CBI->getNumSuccessors(); i++) {
                    ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
2024-05-16 20:29:54 +08:00
XChy
fdaad73875
[Reg2Mem] Handle CallBr instructions (#90953)
Fixes #90900
2024-05-16 20:13:39 +08:00
Matt Arsenault
0ea178b085
SimplifyLibCalls: Emit vector ldexp intrinsics in exp2->ldexp combine (#92219)
Co-authored-by: Nikita Popov <github@npopov.com>
2024-05-16 10:24:56 +02:00
Matt Arsenault
8389177710
SimplifyLibCalls: Use IRBuilder helpers for creating intrinsics (#92288) 2024-05-16 09:20:18 +02:00
Matt Arsenault
ce1ce5d30c
InstCombine: Try to use exp10 intrinsic instead of libcall (#92287)
Addresses old TODO about the exp10 intrinsic not existing.
2024-05-16 09:09:02 +02:00
Jay Foad
1650f1b3d7
Fix typo "indicies" (#92232) 2024-05-15 13:10:16 +01:00
Matt Arsenault
d7bb0723fe
InstCombine: Emit ldexp intrinsic in exp2->ldexp combine (#92039)
Prefer to emit the intrinsic over a libcall in the
intrinsic or no-math-errno case.
2024-05-15 07:41:28 +02:00
Nikita Popov
71fbbb69d6 [IR] Move GlobalValue::getGUID() out of line (NFC)
Avoid including MD5.h in a core IR header.
2024-05-15 10:49:25 +09:00
Mingming Liu
6c8ebc0535
[NFC][CallPromotionUtils]Extract a helper function versionCallSiteWithCond from versionCallSite (#81181)
* This is to be used by https://github.com/llvm/llvm-project/pull/81378
to implement a variant of versionCallSite that compares vtables.
* The parent patch is https://github.com/llvm/llvm-project/pull/81051
2024-05-14 10:13:57 -07:00
chenlin
79643565a8
[LoopUnroll] Remove redundant debug instructions after blocks have been merged (#91246)
Remove redundant debug instructions after blocks have been merged into
the predecessor, It can reduce some compile time in some cases.

This change only fixes the situation of loop unrolling, and other
situations are not considered. "RemoveRedundantDbgInstrs" seems to be
very time-consuming. Thus, we just add here after the "Dest" has been
merged into the "Fold", this may be a more targeted solution!!!

fixes: https://github.com/llvm/llvm-project/issues/89073
2024-05-13 09:42:04 -07:00
Paul Kirth
89a080cb79 [llvm][NFC] Document cl::opt MisExpectTolerance and fix typo
Pull Request: https://github.com/llvm/llvm-project/pull/90670
2024-05-13 16:19:09 +00:00
Orlando Cazalet-Hyams
91d7ca904c
[DebugInfo] Remap extracted DIAssignIDs in hotcoldsplit (#91940)
Fix #91814

When instructions are extracted into a new function the `DIAssignID` metadata
uses and attachments need to be remapped so that the stores and assignment
markers don't link to stores and assignment markers in the original function.

This matches existing inlining behaviour for DIAssignIDs.
2024-05-13 12:49:42 +01:00
Eli Friedman
f893dccbba
Replace uses of ConstantExpr::getCompare. (#91558)
Use ICmpInst::compare() where possible, ConstantFoldCompareInstOperands
in other places. This only changes places where the either the fold is
guaranteed to succeed, or the code doesn't use the resulting compare if
we fail to fold.
2024-05-09 16:50:01 -07:00
Mingming Liu
64f4ceb09e
[Inline][PGO] After inline, update InvokeInst profile counts in caller and cloned callee (#83809)
A related change is https://reviews.llvm.org/D133121, which correctly
preserves both branch weights and value profiles for invoke instruction.
* If the branch weight of the `invokeinst` specifies taken / not-taken branches, there is no scale.
2024-05-08 15:48:40 -07:00
Teresa Johnson
cec6665f2b
[MemProf] Optionally update hints on existing hot/cold new calls (#91047)
If directed by an option, update hints on calls to new that already
provide a hot/cold hint.
2024-05-08 13:41:29 -07:00
AdityaK
42d99013bd
NFC: Add a comment indicating UpdateAnalysisInformation invalidates DFS Numbering (#91252) 2024-05-08 11:03:46 -07:00
Harald van Dijk
a6171900a4
[RemoveDIs] Change remapDbgVariableRecord to remapDbgRecord (#91456)
We need to remap any DbgRecord, not just DbgVariableRecords.

This is the followup to #91447.

Co-authored-by: PietroGhg <pietro.ghiglio@codeplay.com>
2024-05-08 17:02:25 +01:00
Harald van Dijk
927913fac7
[RemoveDIs] Fix remapping of DbgLabelRecords. (#91447)
We already remapped DILocations for DbgVariableRecords, but
DbgLabelRecords have debug locations too that need to be mapped the same
way.
2024-05-08 11:33:47 +01:00
Noah Goldstein
6243395d7f [SCCP] Add nneg flag to uitofp if its operand is non-negative
Similiar to the `InstCombine` changes, just furthering the support of
the `uitofp nneg` support.

Closes #86154
2024-05-07 14:57:29 -05:00
Paul Walker
235cea720c
[NFC][LLVM] Refactor rounding mode detection of constrained fp intrinsic IDs (#90854)
I've refactored the code to genericise the implementation to better
allow for target specific constrained fp intrinsics.
2024-05-07 11:23:55 +01:00
Kazu Hirata
677dddebae
[Transforms] Use StringRef::operator== instead of StringRef::equals (NFC) (#91072)
I'm planning to remove StringRef::equals in favor of
StringRef::operator==.

- StringRef::operator==/!= outnumber StringRef::equals by a factor of
  31 under llvm/ in terms of their usage.

- The elimination of StringRef::equals brings StringRef closer to
  std::string_view, which has operator== but not equals.

- S == "foo" is more readable than S.equals("foo"), especially for
  !Long.Expression.equals("str") vs Long.Expression != "str".
2024-05-04 12:33:12 -07:00
Jorge Gorbe Moya
2cde0e2f97 Revert "[BasicBlockUtils] Remove redundant llvm.dbg instructions after blocks to reduce compile time (#89069)"
This reverts commit 2e3e0868748635b779ba89a772eae3664bd822e4. It caused
quadratic slowdown at compilation time in some cases. See the comments
in the original PR: https://github.com/llvm/llvm-project/pull/89069
2024-05-03 13:05:08 -07:00
Noah Goldstein
285dbed147 [Inliner] Propagate callee argument memory access attributes before inlining
To avoid losing information, we can propagate some access attribute
from the to-be-inlined callee to its callsites.

We can propagate argument memory access attributes to callsite
parameters if they are from the same underlying object.

Closes #89024
2024-05-03 14:10:24 -05:00
Florian Hahn
bccb7ed8ac
Reapply "[LV] Improve AnyOf reduction codegen. (#78304)"
This reverts the revert commit c6e01627acf859.

This patch includes a fix for any-of reductions and epilogue
vectorization. Extra test coverage for the issue that caused the revert
has been added in bce3bfced5fe0b019 and an assertion has been added in
c7209cbb8be7a3c65813.

--------------------------------
Original commit message:

Update AnyOf reduction code generation to only keep track of the AnyOf
property in a boolean vector in the loop, only selecting either the new
or start value in the middle block.

The patch incorporates feedback from https://reviews.llvm.org/D153697.

This fixes the #62565, as now there aren't multiple uses of the
start/new values.

Fixes https://github.com/llvm/llvm-project/issues/62565

PR: https://github.com/llvm/llvm-project/pull/78304
2024-05-03 14:40:49 +01:00
Vitaly Buka
70b5a22678
[hwasan] Don't crash on vscale allocas (#90932)
getAllocaSizeInBytes will crash casting size to
constant.
2024-05-03 00:16:57 -07:00
Antonio Frighetto
1bb929833b [Inline][Cloning] Drop incompatible attributes from NewFunc
Performing `instSimplify` while cloning is unsafe due to incomplete
remapping (as reported in #87534). Ideally, `instSimplify` ought to
reason on the updated newly-cloned function, after returns have been
rewritten and callee entry basic block / call-site have been fixed up.
This is in contrast to `CloneAndPruneIntoFromInst` behaviour, which
is inherently expected to clone basic blocks, with pruning on top of
– if any –, and not actually fixing up returns / CFG, which should be
up to the Inliner. We may solve this by letting `instSimplify` work on
the newly-cloned function, while maintaining old function attributes,
so as to avoid inconsistencies between the yet-to-be-solved return
type, and new function ret type attributes.
2024-05-02 16:29:09 +02:00
Antonio Frighetto
42c7cb6969 Reapply "[Inline][Cloning] Defer simplification after phi-nodes resolution"
Original commit: a61f9fe31750cee65c726fb51f1b14e31e177258

Multiple 2-stage buildbots were reporting failures. These issues have been
addressed separately.

Fixes: https://github.com/llvm/llvm-project/issues/87534.
2024-05-02 16:29:09 +02:00
Florian Hahn
175d297102
[LoopUnroll] Add CSE to remove redundant loads after unrolling. (#83860)
This patch adds loadCSE support to simplifyLoopAfterUnroll. It is based
on EarlyCSE's implementation using ScopeHashTable and is using SCEV for
accessed pointers to check to find redundant loads after unrolling.

This applies to the late unroll pass only, for full unrolling those
redundant loads will be cleaned up by the regular pipeline.

The current approach constructs MSSA on-demand per-loop, but there is
still small but notable compile-time impact:

stage1-O3  +0.04%
stage1-ReleaseThinLTO +0.06%
stage1-ReleaseLTO-g +0.05%
stage1-O0-g +0.02%
stage2-O3 +0.09%
stage2-O0-g +0.04%
stage2-clang +0.02%


https://llvm-compile-time-tracker.com/compare.php?from=c089fa5a729e217d0c0d4647656386dac1a1b135&to=ec7c0f27cb5c12b600d9adfc8543d131765ec7be&stat=instructions:u

This benefits some workloads with runtime-unrolling disabled,
where users use pragmas to force unrolling, as well as with
runtime unrolling enabled.

On SPEC/MultiSource, this removes a number of loads after unrolling
on AArch64 with runtime unrolling enabled.

```
External/S...te/526.blender_r/526.blender_r    96
MultiSourc...rks/mediabench/gsm/toast/toast    39
SingleSource/Benchmarks/Misc/ffbench            4
External/SPEC/CINT2006/403.gcc/403.gcc         18
MultiSourc.../Applications/JM/ldecod/ldecod     4
MultiSourc.../mediabench/jpeg/jpeg-6a/cjpeg     6
MultiSourc...OE-ProxyApps-C/miniGMG/miniGMG     9
MultiSourc...e/Applications/ClamAV/clamscan     4
MultiSourc.../MallocBench/espresso/espresso     3
MultiSourc...dence-flt/LinearDependence-flt     2
MultiSourc...ch/office-ispell/office-ispell     4
MultiSourc...ch/consumer-jpeg/consumer-jpeg     6
MultiSourc...ench/security-sha/security-sha    11
MultiSourc...chmarks/McCat/04-bisect/bisect     3
SingleSour...tTests/2020-01-06-coverage-009    12
MultiSourc...ench/telecomm-gsm/telecomm-gsm    39
MultiSourc...lds-flt/CrossingThresholds-flt    24
MultiSourc...dence-dbl/LinearDependence-dbl     2
External/S...C/CINT2006/445.gobmk/445.gobmk     6
MultiSourc...enchmarks/mafft/pairlocalalign    53
External/S...31.deepsjeng_r/531.deepsjeng_r     3
External/S...rate/510.parest_r/510.parest_r    58
External/S...NT2006/464.h264ref/464.h264ref    29
External/S...NT2017rate/502.gcc_r/502.gcc_r    45
External/S...C/CINT2006/456.hmmer/456.hmmer     6
External/S...te/538.imagick_r/538.imagick_r    18
External/S.../CFP2006/447.dealII/447.dealII     4
MultiSourc...OE-ProxyApps-C++/miniFE/miniFE    12
External/S...2017rate/525.x264_r/525.x264_r    36
MultiSourc...Benchmarks/7zip/7zip-benchmark    33
MultiSourc...hmarks/ASC_Sequoia/AMGmk/AMGmk     2
MultiSourc...chmarks/VersaBench/8b10b/8b10b     1
MultiSourc.../Applications/JM/lencod/lencod   116
MultiSourc...lds-dbl/CrossingThresholds-dbl    24
MultiSource/Benchmarks/McCat/05-eks/eks        15
```

PR: https://github.com/llvm/llvm-project/pull/83860
2024-05-02 11:01:24 +01:00
Andreas Jonson
3d65bd935a
[NFC] Reduce copies created of ConstantRange when getting ConstantRangeAttributes (#90335)
Think that it can be good to reduce the number of copies created when
working with ConstantRangeAttributes.
2024-05-02 10:48:23 +09:00
Paul Kirth
7538df90ae
[llvm][profdata][NFC] Support 64-bit weights in ProfDataUtils (#86607)
Since some places, like SimplifyCFG, work with 64-bit weights, we supply
an API in ProfDataUtils to extract the weights accordingly.

We change the API slightly to disambiguate the 64-bit version from the
32-bit version.
2024-04-30 13:53:04 -07:00
Kees Cook
869ffcf3f6
[CodeGen][i386] Move -mregparm storage earlier and fix Runtime calls (#89707)
When building the Linux kernel for i386, the -mregparm=3 option is
enabled. Crashes were observed in the sanitizer handler functions, and
the problem was found to be mismatched calling convention.

As was fixed in commit c167c0a4dcdb ("[BuildLibCalls] infer inreg param
attrs from NumRegisterParameters"), call arguments need to be marked as
"in register" when -mregparm is set. Use the same helper developed there
to update the function arguments.

Since CreateRuntimeFunction() is actually part of CodeGenModule, storage
of the -mregparm value is also moved to the constructor, as doing this
in Release() is too late.

Fixes: https://github.com/llvm/llvm-project/issues/89670
2024-04-29 14:54:10 -07:00
Matthias Braun
dede19caf0
GlobalOpt: Handle threadlocal.address intrinsic (#88454)
This changes `GlobalOpt` to skip/look-through `threadlocal.address`
intrinsic where apropriate.

This fixes issue #73314
2024-04-29 09:21:50 -07:00
Matthew Devereau
6561fa3d02
[LoopUnswitch] Allow i1 truncs in loop unswitch (#89738)
With the addition of #84628, truncs to i1 are being emitted as
conditions to branch instructions. This caused significant regressions
in cases which were previously improved by loop unswitch. Adding truncs
to i1 restore the previous performance seen.
2024-04-29 15:17:48 +01:00
Kazu Hirata
53cda4ca3b [Transforms] Use LLVMContext::MD_loop (NFC) 2024-04-28 20:18:03 -07:00
Florian Mayer
022dc6bab5
[NFC] [HWASan] factor out debug record annotation (#90252)
This will also be used by stack MTE
2024-04-26 13:11:12 -07:00
Carlos Alberto Enciso
7696d36b4e
[Transforms] Debug values are not remapped when cloning. (#87747)
When cloning instructions from one basic block to another,
the debug values are not remapped, in the same was as the
normal instructions.
2024-04-26 13:35:09 +01:00
CL
2e3e086874
[BasicBlockUtils] Remove redundant llvm.dbg instructions after blocks to reduce compile time (#89069)
this patch is to fix the compile time for some cases, before this
change, some targets (riscv-64, ve) will spend much more compile time on
this case (https://godbolt.org/z/rrov17cTo). With this change, the
compile time was reduced a lot.

Fixes https://github.com/llvm/llvm-project/issues/89073

PR: https://github.com/llvm/llvm-project/pull/89069
2024-04-26 10:57:37 +01:00
Yingwei Zheng
697fcd0098
[SimplifyCFG] Handle llvm.assume in passingValueIsAlwaysUndefined (#89929)
See the following example:
```
define i32 @test(i32 %cond) {
entry:
  switch i32 %cond, label %default [
    i32 0, label %case0
    i32 1, label %case1
    i32 2, label %case2
  ]

case0:
  br label %exit

case1:
  br label %exit

case2:
  br label %exit

default:
  br label %exit

exit:
  %bool = phi i1 [ false, %default ], [ true, %case0 ], [ true, %case1 ], [ true, %case2 ]
  %res = phi i32 [ 0, %default ], [ 1, %case0 ], [ 2, %case1 ], [ 3, %case2 ]
  call void @llvm.assume(i1 %bool)
  ret i32 %res
}
```
The edge `%default -> %bool` is dead since it will trigger an immediate
UB.

Alive2: https://alive2.llvm.org/ce/z/gywJiE

My benchmark shows many rust applications and some c/c++ applications
(e.g., arrow/php/qemu) will benefit from this patch :)
2024-04-25 20:47:56 +08:00
Vitaly Buka
29c98e59cd Revert "[Inline][Cloning] Defer simplification after phi-nodes resolution" #87963
Reopens #87534.

Breaks multiple bots:
https://lab.llvm.org/buildbot/#/builders/168/builds/20028
https://lab.llvm.org/buildbot/#/builders/74/builds/27773

And reproducer in a61f9fe31750cee65c726fb51f1b14e31e177258.

This reverts commit a61f9fe31750cee65c726fb51f1b14e31e177258.
2024-04-24 14:51:54 -07:00
Arthur Eubanks
63b6efeebf [EntryExitInstrumenter] Don't invalidate analyses if no change was made 2024-04-24 19:51:00 +00:00
Usman Nadeem
b10e4b82e6
[GVN] Restrict equality propagation for pointers (#82458)
This patch does the following:

Adds the following functions:
- replaceDominatedUsesWithIf() that takes a callback.

- canReplacePointersIfEqual(...) returns true if the underlying object
is the same, and for null and const dereferencable pointer replacements.

- canReplacePointersIfEqualInUse(...) returns true for the above as well
as if the use is in icmp/ptrtoint or phi/selects feeding into them.

Updates GVN using the functions above so that the pointer replacements
are only made using the above API.

https://reviews.llvm.org/D143129
2024-04-24 08:59:42 -07:00
Antonio Frighetto
a61f9fe317 [Inline][Cloning] Defer simplification after phi-nodes resolution
A logic issue arose when inlining via `CloneAndPruneFunctionInto`,
which, besides cloning, performs instruction simplification as well.
By the time a new cloned instruction is being simplified, phi-nodes
are not remapped yet as the whole CFG needs to be processed first.
As `VMap` state at this stage is incomplete, `threadCmpOverPHI` and
variants could lead to unsound optimizations. This issue has been
addressed by performing basic constant folding while cloning, and
postponing instruction simplification once phi-nodes are revisited.

Fixes: https://github.com/llvm/llvm-project/issues/87534.
2024-04-24 16:55:33 +02:00
Pierre van Houtryve
cf328ff96d
[IR] Memory Model Relaxation Annotations (#78569)
Implements the core/target-agnostic components of Memory Model
Relaxation Annotations.

RFC:
https://discourse.llvm.org/t/rfc-mmras-memory-model-relaxation-annotations/76361/5
2024-04-24 08:52:25 +02:00
Nikita Popov
883887493c [SimplifyCFG] Check alignment when speculating stores
When speculating a store based on a preceding load/store, we need
to ensure that the speculated store does not have a higher
alignment (which might only be guaranteed by the branch condition).

There are various ways in which this could be strengthened (we
could get or enforce the alignment), but for now just do the
simple check against the preceding load/store.

Fixes https://github.com/llvm/llvm-project/issues/89672.
2024-04-23 12:39:35 +09:00
Andreas Jonson
ff3523f67b
[IR] Drop poison-generating return attributes when necessary (#89138)
Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.
2024-04-18 15:27:36 +09:00
Arthur Eubanks
c6e01627ac Revert "Reapply "[LV] Improve AnyOf reduction codegen. (#78304)""
This reverts commit c6e38b928c56f562aea68a8e90f02dbdf0eada85.

Causes miscompiles, see comments on #78304.
2024-04-16 20:40:21 +00:00
Stephen Tozer
1c6b0f779f
[RemoveDI] Add support for debug records to debugify (#87383)
This patch changes debugify to support debug variable records, and
subsequently to no longer convert modules automatically to intrinsics
when entering debugify.
2024-04-16 17:07:46 +01:00
XChy
d2d4a1bbdc
Revert "[JumpThreading] Thread over BB with only an unconditional branch" (#88907)
Reverts llvm/llvm-project#86312
2024-04-16 22:52:19 +08:00