7881 Commits

Author SHA1 Message Date
Yingwei Zheng
4eac8daa38
[LoopPeel] Handle non-local instructions/arguments when updating exiting values (#142993)
Similar to
7e14161f49,
the exiting value may be a non-local instruction or an argument.

Closes https://github.com/llvm/llvm-project/issues/142895.
2025-06-06 12:56:28 +08:00
Peter Collingbourne
d1b0b4bb44
Add -funique-source-file-identifier option.
This option complements -funique-source-file-names and allows the user
to use a different unique identifier than the source file path.

Reviewers: teresajohnson

Reviewed By: teresajohnson

Pull Request: https://github.com/llvm/llvm-project/pull/142901
2025-06-05 10:52:01 -07:00
Yingwei Zheng
519cb460f6
[SCCP] Remove masking operations (#142736)
CVP version:
2d5820cd72
Compile-time impact:
https://llvm-compile-time-tracker.com/compare.php?from=3ec0c5c7fef03985b43432c6b914c289d8a5435e&to=92b4df90695dd37defdabf8a30f0b0322b648a00&stat=instructions:u
2025-06-04 22:31:08 +08:00
Yingwei Zheng
7e1fa09ce2
[SimplifyCFG] Bail out on vector GEPs in passingValueIsAlwaysUndefined (#142526)
Closes https://github.com/llvm/llvm-project/issues/142522.
2025-06-04 12:37:30 +08:00
Vitaly Buka
3531cc1cc7
[PromoteMem2Reg] Optimize memory usage in PromoteMem2Reg (#142474)
When BasicBlock has a large number of allocas, and
successors, we had to copy entire IncomingVals and
IncomingLocs vectors for successors.

Also updates to IncomingVals and
IncomingLocs are infrequent (only Load/Store into
alloca affect arrays).

Given the nature of DFS traversal, instead of copying
the entire vector, we can keep track of the changes
and undo all changes done by successors.

Fixes #142461

On the attached to issue #142461 IR RSS drops from 35Gb to 1.8Gb.

But it does not affect compile time on average

https://llvm-compile-time-tracker.com/compare.php?from=2e98ed8caa0b47ee79af4ad24b5436a89fe49dfa&to=effac6d1fd600e544f8bc21382c7e541973b1378&stat=instructions:u
2025-06-03 19:34:36 -07:00
Vitaly Buka
ac4893dd77
[NFC][PromoteMem2Reg] Move IncomingVals, IncomingLocs, Worklist into class (#142468)
They are all DFS state related, as `Visited`. But `Visited` is already a
class member, so we make things more consistent and less
parameters to pass around.

By itself, the patch has little value, but it simplifies stuff in the
#142474.

For #142461
2025-06-03 14:47:45 -07:00
Vitaly Buka
3cb967a2cd
[NFCI][PromoteMem2Reg] Don't handle the first successor out of order (#142464)
Just for consistency, to avoid confusing conditions.

`reverse` helps to avoid tests updates as nothing is
changing for for successors count <=2.

For #142461
2025-06-03 10:26:55 -07:00
Vitaly Buka
b9dec5aa79
[NFC] Remove goto in PromoteMem2Reg::RenamePass (#142454)
'goto' is essentially a shortcut for push/pop for worklist.
It can be expensive if we copy vectors, but if we move them, it
should not be an issue.

Without 'goto' it's easier to reason about the
code, when `PromoteMem2Reg::RenamePass` processes
exactly one edge at a time.

There is out of order processing of the first
successor, I keep it just to make this patch pure NFC. I'll
remove this in follow up patches.

For #142461
2025-06-03 09:14:04 -07:00
Ramkumar Ramachandra
b40e4ceaa6
[ValueTracking] Make Depth last default arg (NFC) (#142384)
Having a finite Depth (or recursion limit) for computeKnownBits is very
limiting, but is currently a load-bearing necessity, as all KnownBits
are recomputed on each call and there is no caching. As a prerequisite
for an effort to remove the recursion limit altogether, either using a
clever caching technique, or writing a easily-invalidable KnownBits
analysis, make the Depth argument in APIs in ValueTracking uniformly the
last argument with a default value. This would aid in removing the
argument when the time comes, as many callers that currently pass 0
explicitly are now updated to omit the argument altogether.
2025-06-03 17:12:24 +01:00
Kazu Hirata
54d836a080
[llvm] Use *Set::insert_range (NFC) (#138237) 2025-06-02 19:48:13 -07:00
Yingwei Zheng
1e08febf0a
[SimplifyCFG] Switch to use paramHasNonNullAttr (#125383) 2025-06-02 12:20:13 +08:00
dianqk
aa09dbbbcf
[RelLookupTableConverter] Drop unnamed_addr to avoid generating GOTPCREL relocations (#142304)
Follow
https://github.com/llvm/llvm-project/pull/72584#issuecomment-2925759253,
the patch will drop the `unnamed_addr` attribute when generating
relative lookup tables. I'm not very confident about this patch, but it
does resolve https://github.com/rust-lang/rust/issues/140686,
https://github.com/rust-lang/rust/issues/141306 and
https://github.com/rust-lang/rust/issues/141737.

But I don't think this will result in worse problems.

> LLVM provides that the calculation of such a constant initializer will
not overflow at link time under the medium code model if x is an
unnamed_addr function. However, it does not provide this guarantee for a
constant initializer folded into a function body. This intrinsic can be
used to avoid the possibility of overflows when loading from such a
constant. ([‘llvm.load.relative’
Intrinsic](https://llvm.org/docs/LangRef.html#id2592))

This is my concern. I'm not sure how unnamed_addr provides this
guarantee, and I haven't found any test cases.
2025-06-01 22:17:57 +08:00
Andreas Jonson
7c080e2677
[InstCombine] Avoid to create bitreverse.i1 for or of trunc to i1 (#142258) 2025-05-31 10:20:49 +02:00
Teresa Johnson
49d48c32e0
[MemProf] Emit remarks when hinting allocations not needing cloning (#141859)
The context disambiguation code already emits remarks when hinting
allocations (by adding hotness attributes) during cloning. However,
we did not yet emit hints when applying the hotness attributes during
building of the metadata (during matching and again after inlining).
Add remarks when we apply the hint attributes for these
non-context-sensitive allocations.
2025-05-28 16:44:44 -07:00
Florian Hahn
249301c779
[LoopUtils] Pass sentinel value directly to createFindLastIVRed (NFC).
Now that there is only a single FindLastIV recurrence kind, simply pass
the sentinel value instead of the full recurrence descriptor to tighten
the interface.
2025-05-28 22:00:11 +01:00
Florian Hahn
0d7b34bfc1
[LoopUtils] Pass start value directly to createAnyOfReduction (NFC).
Now that there is only a single AnyOf recurrence kind, simply pass the
start value instead of the full recurrence descriptor, to tighten the
interface.
2025-05-28 21:28:02 +01:00
Florian Hahn
fb923e98d1
[Local] Verify opcodes match for all insts passed to mergeFlags (NFC). (#141231)
The logic for tracking flags relies on all instructions having the same
opcode. Add an assert to check, as suggested in
https://github.com/llvm/llvm-project/pull/140406.

PR: https://github.com/llvm/llvm-project/pull/141231
2025-05-28 16:03:24 +01:00
Florian Hahn
f98bdd94e6
Reapply "[LoopPeel] Remove known trip count restriction when peeling last. (#140792)"
This reverts commit 580454526b936f7a576ddbc9bb932cf9be376ec4.

The recommitted version contains an extra check to not peel if the
latch exit is controlled by a pointer induction.

Original message:
Remove the restriction that the loop must be known to execute at least 2
iterations when peeling the last iteration. If we cannot prove at least
2 iterations are executed, a check and branch to skip the peeled loop is
inserted.

PR: https://github.com/llvm/llvm-project/pull/140792
2025-05-28 13:02:03 +01:00
Florian Hahn
580454526b
Revert "[LoopPeel] Remove known trip count restriction when peeling last. (#140792)"
This reverts commit 24b97756decb7bf0e26dcf0e30a7a9aaf27f417c.
Also reverts ac9a466e39bf97ffeab127982aa7c405cb257551.

Building CMake triggers a crash with the patch, revert while I
investigate.
2025-05-27 21:25:32 +01:00
Nikita Popov
904d0c293e
[Inline] Only consider provenance captures for scoped alias metadata (#138540)
When determining whether an escape source may alias with a noalias
argument, only take provenance captures into account. If only the
address of the argument was captured, an access through the escape
source is not legal.
2025-05-27 15:15:57 +02:00
Florian Hahn
ac9a466e39
[LoopPeel] Insert new phis before first non-PHI when peeling last iter.
Make sure the new phis are inserted before any non-phi instructions.
This fixes a crash when dbg_value instructions are present in the
original exit block.
2025-05-27 10:46:28 +01:00
Kazu Hirata
89308de4b0
[llvm] Value-initialize values with *Map::try_emplace (NFC) (#141522)
try_emplace value-initializes values, so we do not need to pass
nullptr to try_emplace when the value types are raw pointers or
std::unique_ptr<T>.
2025-05-26 15:13:02 -07:00
Florian Hahn
24b97756de
[LoopPeel] Remove known trip count restriction when peeling last. (#140792)
Remove the restriction that the loop must be known to execute at least 2
iterations when peeling the last iteration. If we cannot prove at least
2 iterations are executed, a check and branch to skip the peeled loop is
inserted.

PR: https://github.com/llvm/llvm-project/pull/140792
2025-05-26 20:08:02 +01:00
Florian Hahn
364d80e5c5
[LoopPeel] Make sure bound in exit condition is loop invariant.
Follow-up to post-commit comment for
(https://github.com/llvm/llvm-project/pull/139551.

This should effectively be NFC, given the other existing restrictions.
2025-05-25 19:21:49 +01:00
Florian Hahn
f755e6644a
[LoopPeel] Make sure AddRec is for correct loop when peeling last iter.
Follow-up to post-commit comment for
(https://github.com/llvm/llvm-project/pull/139551.

This should effectively be NFC, given the other existing restrictions.
2025-05-25 12:05:06 +01:00
GnSight
2f3bc37cd2
[PredicateInfo] Update comments for PredicateAssume (NFC) (#139269)
Previously, PredicateAssume was modified to materialize after the assume
statement rather than before it (See 353fa44). Update relevant comments
to match this.
2025-05-24 22:22:43 +02:00
Kazu Hirata
0ef8ef66cc
[Transforms] Remove unused includes (NFC) (#141357)
These are identified by misc-include-cleaner.  I've filtered out those
that break builds.  Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
2025-05-24 09:37:43 -07:00
Florian Hahn
34813d9d38
[Reassociate] Move Disjoint flag handling to OverflowTracking. (#140406)
Move disjoint flag tracking to OverflowTracking. This enables preserving
disjoint flags in Reassociate.

Depends on https://github.com/llvm/llvm-project/pull/140404

PR: https://github.com/llvm/llvm-project/pull/140406
2025-05-23 14:59:18 +01:00
Nikita Popov
eee958285b
[SimplifyCFG] Only consider provenance capture in store speculation (#138548)
The capture check here is to protect against concurrent accesses from
other threads. This requires the provenance to escape.
2025-05-22 17:01:37 +02:00
Kazu Hirata
99779b42d3
[SCCPSolver] Mark several functions const (NFC) (#140926) 2025-05-21 13:57:50 -07:00
Kazu Hirata
5a3776af52
[SCCPSolver] Make getMRVFunctionsTracked return a reference (NFC) (#140851)
This patch makes getMRVFunctionsTracked return a reference.
runIPSCCP, the sole user of getMRVFunctionsTracked, just needs a
read-access to the map.

The missing "&" is most likely an oversight as two "sibling" functions
getTrackedRetVals and getTrackedGlobals return maps by const
reference.
2025-05-21 08:56:15 -07:00
Kazu Hirata
fe6290ef5b
[llvm] Use *Map::try_emplace (NFC) (#140843)
try_emplace can default-construct values, so we do not need to do so
on our own.  Plus, try_emplace(Key) is much shorter than
insert(std::make_pair(Key, Value()).
2025-05-21 01:11:01 -07:00
Iris Shi
bdf03fcff3
Revert "[llvm][NFC] Use llvm::sort()" (#140668) 2025-05-20 11:27:03 +08:00
Florian Hahn
c92ff61cee
[Local] Move OverflowTracking to Local.h, move logic to helpers (NFC) (#140403)
Move parts of the logic used by Reassociate to OverflowTracking
(mergeFlags & applyFlags) and move the definition to Local.h.

For now it just moves the NUW/NSW handling, as this matches the uses in
LICM. I'll look into the FP math handling separately, as it looks like
there's a difference between Reassociate (takes all flags from I, while
LICM takes the intersection of the flags on both instructions).

PR: https://github.com/llvm/llvm-project/pull/140403
2025-05-19 21:47:50 +01:00
Florian Hahn
a0a2a1e095
[LoopPeel] Make sure exit condition has a single use when peeling last.
Update the check in canPeelLastIteration to make sure the exiting
condition has a single use. When peeling the last iteration, we adjust
the condition in the loop body to be true one iteration early, which
would be incorrect for other users.

Fixes https://github.com/llvm/llvm-project/issues/140444.
2025-05-18 11:47:12 +01:00
Florian Hahn
7e14161f49
[LoopPeel] Handle constants when updating exit values when peeling last.
Account for constant values when updating exit values after peeling an
iteration from the end. This can happen if the inner loop gets unrolled
and simplified.

Fixes https://github.com/llvm/llvm-project/issues/140442.
2025-05-18 10:17:21 +01:00
Florian Hahn
3fcfce4c5e
Reapply "[LoopPeel] Implement initial peeling off the last loop iteration. (#139551)"
This reverts the revert commit bf92b127d2637948f53d11a187e865aa10e2e74c.

This adds missing initialization of PeelLast in gatherPeelingPreferences.

Original message:
Generalize countToEliminateCompares to also consider peeling off the
last iteration if it eliminates a compare.

At the moment, codegen for peeling off the last iteration is quite
restrictive and callers have to make sure that the exit condition can be
adjusted when peeling and that the loop executes at least 2 iterations.

Both will be relaxed in follow-ups.

PR: https://github.com/llvm/llvm-project/pull/139551
2025-05-17 10:51:05 +01:00
Iris Shi
061a7699f3
[llvm][NFC] Use llvm::sort() (#140335) 2025-05-17 14:49:46 +08:00
Florian Hahn
bf92b127d2
Revert "[LoopPeel] Implement initial peeling off the last loop iteration. (#139551)"
This reverts commit bb10c3ba7f77d40a7fbfd4ac815015d3a4ae476a.

Also reverts 4f663cca15f2b53c2bc6a84d1b1f5bd81679356d:
  Revert "[LoopPeel] Make sure PeelLast is always initialized."

Revert for now to bring msan bots back to green

 https://lab.llvm.org/buildbot/#/builders/164/builds/9992
 https://lab.llvm.org/buildbot/#/builders/94/builds/7158
2025-05-16 08:33:12 +01:00
Florian Hahn
4f663cca15
[LoopPeel] Make sure PeelLast is always initialized.
Make sure PeelLast is initialized on all paths.

Should fix MSan bootstrap failures
     https://lab.llvm.org/buildbot/#/builders/164/builds/9992
     https://lab.llvm.org/buildbot/#/builders/94/builds/7158

Fixup after https://github.com/llvm/llvm-project/pull/139551.
2025-05-15 22:33:10 +01:00
Florian Hahn
bb10c3ba7f
[LoopPeel] Implement initial peeling off the last loop iteration. (#139551)
Generalize countToEliminateCompares to also consider peeling off the
last iteration if it eliminates a compare.

At the moment, codegen for peeling off the last iteration is quite
restrictive and callers have to make sure that the exit condition can be
adjusted when peeling and that the loop executes at least 2 iterations.

Both will be relaxed in follow-ups.

PR: https://github.com/llvm/llvm-project/pull/139551
2025-05-15 19:15:48 +01:00
Jessica Clarke
864f0ff4ef
[clang][IR] Overload @llvm.thread.pointer to support non-AS0 targets (#132489)
Thread-local globals live, by default, in the default globals address
space, which may not be 0, so we need to overload @llvm.thread.pointer
to support other address spaces, and use the default globals address
space in Clang.
2025-05-14 21:51:56 +01:00
Vladislav Dzhidzhoev
47144a0dc1
[CloneFunction][DebugInfo] Ensure DILocalVariables of inlined functions are not cloned (NFC) (#138590)
This change was separated from
https://github.com/llvm/llvm-project/pull/119001.

When cloning functions, use IdentityMDPredicate to ensure that if
DISubprogram is not cloned, then its DILocalVariables are not cloned
either.

This is currently expected to be an NFC, as DILocalVariables only
reference their subprograms (via DILocalScopes) and types, and inlined
DISubprograms and DITypes are not cloned. Thus, DILocalVariables are
mapped to self in ValueMapper (in mapTopLevelUniquedNode).

However, it will be needed for the original PR
https://github.com/llvm/llvm-project/pull/119001, where a
DILocalVariable may refer to a local type of a DISubprogram that is
being cloned. In that case, ValueMapper will clone DILocalVariable even
if it belongs to an inlined DISubprogram that is not cloned, which
should be avoided.
I'm making this change into a separate PR to make the original PR a bit
smaller, and because this has more to do with variables than with types.
2025-05-14 19:41:56 +02:00
Ellis Hoag
78f0af5d89
[SimplifyCFG][swifterror] Don't sink calls with swifterror params (#139015)
We've encountered an LLVM verification failure when building Swift with
the SimplifyCFG pass enabled. I found that
https://reviews.llvm.org/D158083 fixed this pass by preventing sinking
loads or stores of swifterror values, but it did not implement the same
protection for call or invokes.
In `Verifier.cpp`
[here](c685355811/llvm/lib/IR/Verifier.cpp (L4360-L4364))
and
[here](c685355811/llvm/lib/IR/Verifier.cpp (L3661-L3662))
we can see that swifterror values must also be used directly by call
instructions.
2025-05-12 14:37:26 -07:00
Kazu Hirata
85f3d60d81
[Utils] Use range-based for loops (NFC) (#139426) 2025-05-10 22:50:46 -07:00
Kazu Hirata
6e4f501b1b
[Utils] Remove redundant calls to std::unique_ptr<T>::get (NFC) (#139352) 2025-05-10 07:27:29 -07:00
Matt Arsenault
83551626d0
LoopRotationUtils: Use initializer list instead of repeated push_back (#139312) 2025-05-10 00:21:44 +02:00
Orlando Cazalet-Hyams
c453da7b7c
[KeyInstr][LoopUnroll] Remap atoms while unrolling (#133489)
RFC: https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668
2025-05-09 13:47:16 +01:00
Matt Arsenault
89d13f87c7
Analysis: Remove no-AssumptionCache path in getKnowledgeForValue (#139232)
As requested in https://github.com/llvm/llvm-project/pull/138961#discussion_r2078483175
2025-05-09 12:23:38 +02:00
Guy David
a1beb61940
[SimplifyLibCalls] Shrink sin, cos to sinf, cosf when allowed (#139082)
This optimization already exists, but for the libcall versions of these
functions and not for their intrinsic form.
Solves https://github.com/llvm/llvm-project/issues/139044.

There are probably more opportunities for other intrinsics, because the
switch-case in `LibCallSimplifier::optimizeCall` covers only `pow`,
`exp2`, `log`, `log2`, `log10`, `sqrt`, `memset`, `memcpy` and
`memmove`.
2025-05-09 07:25:35 +03:00