956 Commits

Author SHA1 Message Date
Nikita Popov
c23b4fbdbb
[IR] Remove size argument from lifetime intrinsics (#150248)
Now that #149310 has restricted lifetime intrinsics to only work on
allocas, we can also drop the explicit size argument. Instead, the size
is implied by the alloca.

This removes the ability to only mark a prefix of an alloca alive/dead.
We never used that capability, so we should remove the need to handle
that possibility everywhere (though many key places, including stack
coloring, did not actually respect this).
2025-08-08 11:09:34 +02:00
Florian Hahn
d74d841b65
[SECV] Try to push the op into ZExt: A + zext (-A + B) -> zext (B) (#151227)
Try to push the constant operand into a ZExt:
A + zext (-A + B) -> zext (B), if trunc (A) + -A + B does not
unsigned-wrap.

The actual code supports ZExts with arbitrary number of arguments, hence
the getAddExpr in the return.

This helps SCEV reasoning in some cases, commonly when adding an offset
to a zero-extended SCEV that subtracts the same offset.

Note that this is restricted to cases where we can fold away an operand
of the inner Add. This is needed to avoid bad interactions with patterns
when forming ZExts, which try to push to ZExt to add operands.

https://alive2.llvm.org/ce/z/q7d303

PR: https://github.com/llvm/llvm-project/pull/151227
2025-07-30 21:10:57 +01:00
Florian Hahn
c6f7fa7437
[SCEV] Add test for pushing constant add into zext.
Adds a SCEV-only tests for
https://github.com/llvm/llvm-project/pull/151227.
2025-07-30 10:04:40 +01:00
Nikita Popov
2c6eec219d [Tests] Avoid lifetime intrinsics on non-allocas (NFC)
Don't rely on auto-upgrade, instead either remove unnecessary
casts or remove no longer applicable tests.
2025-07-23 15:05:43 +02:00
Ramkumar Ramachandra
bdc8736b2d
[SCEV] Move a test into IndVars (#147360)
Move the guards.ll into IndVars, as it is really an IndVars test.
2025-07-09 13:32:30 +01:00
Ramkumar Ramachandra
a73daf4ade
[SCEV] Regen a test with UTC (#147361) 2025-07-08 11:53:58 +01:00
Florian Hahn
0ba63b2f22
[SCEV] Add additional test coverage for loop-guards reasoning.
Add additional tests showing missed opportunities when using loop guards
for reasoning in SCEV, depending on the order the guards appear in the
IR.
2025-06-01 22:39:37 +01:00
Craig Topper
52d2b589b2
[IndVarSimplify] Set samesign when converting signed comparison to unsigned comparison in eliminateIVComparison. (#138215) 2025-05-02 08:17:45 -07:00
Yingwei Zheng
bb9580a02b
[SCEV] Use ashr to adjust constant multipliers (#135534)
SCEV converts "-2 *nsw (i32 V)" into "2148473647 *nsw (i32 V)". But we
cannot preserve the nsw flag when the constant multiplier is negative.
This patch changes lshr to ashr so that we can preserve both nsw and nuw
flags.

Alive2 proof: https://alive2.llvm.org/ce/z/LZVSEa
Closes https://github.com/llvm/llvm-project/issues/135531.
2025-04-13 20:22:48 +08:00
Yingwei Zheng
f066d7504e
[Reland][SCEV] teach isImpliedViaOperations about samesign (#133711)
This patch relands https://github.com/llvm/llvm-project/pull/124270.
Closes https://github.com/llvm/llvm-project/issues/126409.

The root cause is that we incorrectly preserve the samesign flag after
truncating operands of an icmp:
https://alive2.llvm.org/ce/z/4NE9gS

---------

Co-authored-by: Ramkumar Ramachandra <ramkumar.ramachandra@codasip.com>
2025-04-02 18:45:33 +08:00
Yingwei Zheng
c5a491e9ea
[SCEV] Check whether the start is non-zero in ScalarEvolution::howFarToZero (#131522)
https://github.com/llvm/llvm-project/pull/94525 assumes that the loop
will be infinite when the stride is zero. However, it doesn't hold when
the start value of addrec is also zero.

Closes https://github.com/llvm/llvm-project/issues/131465.
2025-03-17 13:59:16 +08:00
Nikita Popov
a6f2a1ecaa [SCEV] Generate test checks (NFC) 2025-02-18 17:11:47 +01:00
Ramkumar Ramachandra
c6b13a2871
Revert "SCEV: teach isImpliedViaOperations about samesign" (#126506)
The commit f5d24e6c is buggy, and following miscompiles have been
reported: #126409 and
https://github.com/llvm/llvm-project/pull/124270#issuecomment-2647222903

Revert it while we investigate.
2025-02-10 13:31:18 +00:00
Ramkumar Ramachandra
52b59476cd
SCEV: re-org a test, regen via UTC (#126237) 2025-02-07 13:19:34 +00:00
Ramkumar Ramachandra
f5d24e6cbe
SCEV: teach isImpliedViaOperations about samesign (#124270)
Use CmpPredicate::getMatching in isImpliedCondBalancedTypes to pass
samesign information to isImpliedViaOperations, and teach it to call
CmpPredicate::getPreferredSignedPredicate, effectively making it
optimize with samesign information.
2025-02-06 18:14:54 +00:00
Ramkumar Ramachandra
16c6c48506
IndVarSimplify: add samesign test from a regression (#125539)
While attempting to teach ScalarEvolution about samesign in another
effort, a complicated testcase with nested loops, and zero-extends of
the induction-variable regresses, but only when the target datalayout is
present. The regression was originally reported on IndVarSimplify, but
an improvement of symbolic BTC was also observed on SCEV. Check in the
test into both IndVarSimplify and SCEV, to ease investigation and
further development.
2025-02-03 19:28:11 +00:00
Nikita Popov
07efe2c18a
[SCEV] Check correct value for UB (#124302)
This is a followup to #117152. That patch introduced a check for
UB/poison on BEValue. However, the SCEV we're actually going to use is
Shifted. In some cases, it's possible for Shifted to contain UB, while
BEValue doesn't.

In the test case the values are:

BEValue: (-1 * (zext i8 (-83 + ((-83 /u {1,+,1}<%loop>) *
{-1,+,-1}<%loop>)) to i32))<nuw><nsw>
Shifted: (-173 + (-1 * (zext i8 ((-83 /u {0,+,1}<%loop>) *
{0,+,-1}<%loop>) to i32))<nuw><nsw>)<nuw><nsw>

Fixes https://github.com/llvm/llvm-project/issues/123550.
2025-01-29 09:09:14 +01:00
Ramkumar Ramachandra
e5b0132d15
SCEV: add samesign tests for exit-limit computation (#124304)
As the tests demonstrate, computeExitLimitFromICmp needs no additional
changes to compute exit limits from an icmp with samesign.
2025-01-25 21:15:10 +00:00
Nikita Popov
37bf0a10fb [SCEV] Add test for #123550 (NFC) 2025-01-24 17:10:30 +01:00
Ramkumar Ramachandra
e069518f82
SCEV: cover a codepath in isImpliedCondBalancedTypes (#123070)
The code that checks a predicate against a swapped predicate in
isImpliedCondBalancedTypes is not covered by any existing test, within
any Analysis or Transform. Fix this by adding a test to SCEV.
2025-01-23 12:28:30 +00:00
Julian Nagele
137d706739
[SCEV] Do not attempt to collect loop guards for loops without predecessor. (#123662)
Attempting to collect loop guards for loops without a predecessor can
lead to non-terminating recursion trying to construct a SCEV.

Fixes https://github.com/llvm/llvm-project/issues/122913.
2025-01-22 18:36:37 +00:00
goldsteinn
a56ba1fab0
[ValueTracking] Handle recursive select/PHI in ComputeKnownBits (#114689)
Finish porting #114008 to `KnownBits` (Follow up to #113707).
2025-01-22 11:51:18 -06:00
Ramkumar Ramachandra
0ab368c573
SCEV/test: cover implied-via-addition (#123082)
Since cf2e828 (SCEV: regen some tests with UTC) had the side-effect of
moving an implied-via-addition test into IndVarSimplify, implication via
addition is no longer covered in the SCEV tests. Fix this by writing
fresh tests and checking backedge-taken output from SCEV.
2025-01-17 10:54:39 +00:00
Ramkumar Ramachandra
cf2e828925
SCEV: regen some tests with UTC (#123050)
While at it, move a test that calls the IndVarSimplify pass into the
IndVarSimplify directory.
2025-01-15 14:19:23 +00:00
Julian Nagele
f035351af7
[SCEV] Make sure starting block is marked as visited when recursively collecting loop guards. (#120749)
When `collectFromBlock` is called without a predecessor (in particular
for loops that don't have a unique predecessor outside the loop) we
never start climbing the predecessor chain, and thus don't mark the
starting block as visited.

Fixes https://github.com/llvm/llvm-project/issues/120615.
2024-12-31 09:24:48 +00:00
Julian Nagele
acfd26a93b
[SCEV] Fix exit condition for recursive loop guard collection (#120442)
When assumptions are present `Terms.size()` does not actually count the
number of conditions collected from dominating branches; introduce a
separate counter.

Fixes https://github.com/llvm/llvm-project/issues/120237
2024-12-20 15:44:15 +01:00
Akshay Deodhar
82c93b6f19
[SCEV] Simplify SCEVExpr for PHI to SCEV for operand if operands are identical (#115945)
Helps SCEV analyze some special phi nodes, allowing the computation of
loop trip count in cases like the following:

https://godbolt.org/z/xGs1d81TW
2024-12-06 15:26:45 +05:30
Yingwei Zheng
f7ef0721d6
[SCEV] Do not allow refinement in the rewriting of BEValue (#117152)
See the following case:
```
; bin/opt -passes="print<scalar-evolution>" test.ll --disable-output
define i32 @widget() {
b:
  br label %b1

b1:                                              ; preds = %b5, %b
  %phi = phi i32 [ 0, %b ], [ %udiv6, %b5 ]
  %phi2 = phi i32 [ 1, %b ], [ %add, %b5 ]
  %icmp = icmp eq i32 %phi, 0
  br i1 %icmp, label %b3, label %b8

b3:                                              ; preds = %b1
  %udiv = udiv i32 10, %phi2
  %urem = urem i32 %udiv, 10
  %icmp4 = icmp eq i32 %urem, 0
  br i1 %icmp4, label %b7, label %b5

b5:                                              ; preds = %b3
  %udiv6 = udiv i32 %phi2, 0
  %add = add i32 %phi2, 1
  br label %b1

b7:                                              ; preds = %b3
  ret i32 5

b8:                                              ; preds = %b1
  ret i32 7
}
```
```
%phi2 = phi i32 [ 1, %b ], [ %add, %b5 ] -->  {1,+,1}<nuw><nsw><%b1>
%udiv6 = udiv i32 %phi2, 0 --> ({1,+,1}<nuw><nsw><%b1> /u 0)
%phi = phi i32 [ 0, %b ], [ %udiv6, %b5 ] --> ({0,+,1}<nuw><nsw><%b1> /u 0)
```
`ScalarEvolution::createAddRecFromPHI` gives a wrong SCEV result for
`%phi`:

d7d6fb1804/llvm/lib/Analysis/ScalarEvolution.cpp (L5926-L5950)
It converts `phi(0, ({1,+,1}<nuw><nsw><%b1> /u 0))` into `phi(0 / 0,
({1,+,1}<nuw><nsw><%b1> /u 0))`. Then it simplifies the expr into
`{0,+,1}<nuw><nsw><%b1> /u 0`.

As we did in
acd700a24b,
this patch disallows udiv simplification if we cannot prove that the
denominator is a well-defined non-zero value.

Fixes https://github.com/llvm/llvm-project/issues/117133.
2024-12-01 20:11:09 +08:00
Yingwei Zheng
458dfbd855
[SCEV] Fix sext handling for getConstantMultiple (#117093)
Counterexample: 219 is a multiple of 73. But `sext i8 219 to i16 =
65499` is not.
Fixes https://github.com/llvm/llvm-project/issues/116483.
2024-11-21 17:23:04 +08:00
Julian Nagele
7c8e05aa45
[SCEV] Collect and merge loop guards through PHI nodes with multiple incoming values (#113915)
This patch aims to strengthen collection of loop guards by processing
PHI nodes with multiple incoming values as follows: collect guards for
all incoming values/blocks and try to merge them into a single one for
the PHI node.

The goal is to determine tighter bounds on the trip counts of scalar
tail loops after vectorization, helping to avoid unnecessary transforms.
In particular we'd like to avoid vectorizing scalar tails of
hand-vectorized loops, for example in
[Transforms/PhaseOrdering/X86/pr38280.ll](231e03ba7e/llvm/test/Transforms/PhaseOrdering/X86/pr38280.ll),
discovered via https://github.com/llvm/llvm-project/pull/108190

Compile-time impact: https://llvm-compile-time-tracker.com/compare.php?from=a55248789ed3f653740e0723d016203b9d585f26&to=500e4c46e79f60b93b11a752698c520e345948e3&stat=instructions:u

PR: https://github.com/llvm/llvm-project/pull/113915
2024-11-15 10:03:08 +00:00
Lee Wei
1469d82e1c
Remove br i1 undef from some regression tests [NFC] (#115130)
As defined in LangRef, branching on `undef` is undefined behavior.
This PR aims to remove undefined behavior from tests. As UB tests break
Alive2 and may be the root cause of breaking future optimizations.

Here's an Alive2 proof for one of the examples:
https://alive2.llvm.org/ce/z/TncxhP
2024-11-07 08:11:15 +00:00
Florian Hahn
7f06d8afb0
[SCEV] Retain SCEVSequentialMinMaxExpr if an operand may trigger UB. (#110824)
Retain SCEVSequentialMinMaxExpr if an operand may trigger UB, e.g. if
there is an UDiv operand that may divide by 0 or poison

PR: https://github.com/llvm/llvm-project/pull/110824
2024-10-14 13:08:49 +01:00
Florian Hahn
dce5bf8efc
[ValueTracking] AllowEphemerals for alignment assumptions. (#108632)
Allow AllowEphemerals in isValidAssumeForContext, as the CxtI might
be the producer of the pointer in the bundle. At the moment, align
assumptions aren't optimized away.

This allows using the assumption in the computeKnownBits call in
getConstantMultipleImpl.

We could extend the computeKnownBits API to allow callers to specify if
ephemerals are allowed, if the info from computeKnownBitsFromContext is
used to remove alignment assumptions.

PR: https://github.com/llvm/llvm-project/pull/108632
2024-10-03 16:02:34 +01:00
Florian Hahn
bdd40e39a4
[SCEV] Add tests for umin_seq change in #92177
SCEV-only tests for https://github.com/llvm/llvm-project/pull/92177
2024-10-02 11:06:00 +01:00
Nikita Popov
9f3d1695eb
[SCEVExpander] Preserve gep nuw during expansion (#102133)
When expanding SCEV adds to geps, transfer the nuw flag to the resulting
gep. (Note that this doesn't apply to IV increment GEPs, which go
through a different code path.)
2024-10-02 11:45:00 +02:00
Florian Hahn
383a67042a
[SCEV] Add early exit tests with alignment assumptions.
Precommit tests from https://github.com/llvm/llvm-project/pull/108632.
2024-10-02 10:30:04 +01:00
Florian Hahn
2f7ccaf4a8
[SCEV] Add predicate in SolveLinEq to ensure B is a multiple of A. (#108777)
This can help in cases where pointer alignment info is missing, e.g.
https://github.com/llvm/llvm-project/pull/108210

The predicate is formed for the complex expression that's passed to
SolveLinEquationWithOverflow and the checks could probably be pushed
closer to the root nodes, which in some cases may be cheaper to check.


PR: https://github.com/llvm/llvm-project/pull/108777
2024-09-28 14:19:57 +01:00
Florian Hahn
ac946e615c
[SCEV] Re-organize tests requiring remainder predicates.
Also adds additional test coverage in
Analysis/ScalarEvolution/trip-count-urem.ll

Extra test coverage is for https://github.com/llvm/llvm-project/pull/108777.
2024-09-27 21:03:52 +01:00
Florian Hahn
28439a19c1
[SCEV] Add tests with non-power-of-2 steps for #108777.
Adds extra tests for https://github.com/llvm/llvm-project/pull/108777.
2024-09-26 12:57:04 +01:00
David Sherwood
02ee96eca9
[Analysis] Teach isDereferenceableAndAlignedInLoop about SCEV predicates (#106562)
Currently if a loop contains loads that we can prove at compile time
are dereferenceable when certain conditions are satisfied the function
isDereferenceableAndAlignedInLoop will still return false because
getSmallConstantMaxTripCount will return 0 when SCEV predicates
are required. This patch changes getSmallConstantMaxTripCount to take
an optional Predicates pointer argument so that we can permit
functions such as isDereferenceableAndAlignedInLoop to consider more
cases.
2024-09-23 09:56:37 +01:00
David Sherwood
36adf8eced
[NFC][Analysis] Add more SCEV tests for ptr inductions (#108210)
I've added more tests to

Analysis/ScalarEvolution/max-backedge-taken-count-guard-info.ll

to cover more cases of ptr inductions, in particular highlighting what
seems to be a disparity between single exit and multiple exit loops.
2024-09-12 13:29:54 +01:00
Antonio Frighetto
e80f48986c [SCEV] BECount to zero if ((-C + (C smax %x)) /u %x), C > 0 holds
The SCEV expression `((-C + (C smax %x)) /u %x)` can be folded
to zero for any positive constant C.

Proof: https://alive2.llvm.org/ce/z/_dLm8C.
2024-09-05 17:01:56 +02:00
David Sherwood
df3d70b5a7
[Analysis] Add getPredicatedExitCount to ScalarEvolution (#105649)
Due to a reviewer request on PR #88385 I have created this patch
to add a getPredicatedExitCount function, which is similar to
getExitCount except that it uses the predicated backedge taken
information. With PR #88385 we will start to care about more
loops with multiple exits, and want the ability to query exit
counts for a particular exiting block. Such loops may require
predicates in order to be vectorised.

New tests added here:

Analysis/ScalarEvolution/predicated-exit-count.ll
2024-09-02 14:05:26 +01:00
Stephen Tozer
3d08ade7bd
[ExtendLifetimes] Implement llvm.fake.use to extend variable lifetimes (#86149)
This patch is part of a set of patches that add an `-fextend-lifetimes`
flag to clang, which extends the lifetimes of local variables and
parameters for improved debuggability. In addition to that flag, the
patch series adds a pragma to selectively disable `-fextend-lifetimes`,
and an `-fextend-this-ptr` flag which functions as `-fextend-lifetimes`
for this pointers only. All changes and tests in these patches were
written by Wolfgang Pieb (@wolfy1961), while Stephen Tozer (@SLTozer)
has handled review and merging. The extend lifetimes flag is intended to
eventually be set on by `-Og`, as discussed in the RFC
here:

https://discourse.llvm.org/t/rfc-redefine-og-o1-and-add-a-new-level-of-og/72850

This patch implements a new intrinsic instruction in LLVM,
`llvm.fake.use` in IR and `FAKE_USE` in MIR, that takes a single operand
and has no effect other than "using" its operand, to ensure that its
operand remains live until after the fake use. This patch does not emit
fake uses anywhere; the next patch in this sequence causes them to be
emitted from the clang frontend, such that for each variable (or this) a
fake.use operand is inserted at the end of that variable's scope, using
that variable's value. This patch covers everything post-frontend, which
is largely just the basic plumbing for a new intrinsic/instruction,
along with a few steps to preserve the fake uses through optimizations
(such as moving them ahead of a tail call or translating them through
SROA).

Co-authored-by: Stephen Tozer <stephen.tozer@sony.com>
2024-08-29 17:53:32 +01:00
David Sherwood
d46812a7be
[Analysis] Teach ScalarEvolution::getRangeRef about more dereferenceable objects (#104778)
Whilst dealing with review comments on

https://github.com/llvm/llvm-project/pull/96752

I discovered that SCEV does not know about the dereferenceable attribute
on function arguments so I have updated getRangeRef to make use of it
by calling getPointerDereferenceableBytes.
2024-08-22 14:45:14 +01:00
Philip Reames
d385485450 [SCEV] Autogenerate a test for ease of upcoming update 2024-08-13 08:09:47 -07:00
Philip Reames
b812e57ac3
[SCEV] Consolidate code for proving wrap flags of controlling finite IVs (#101404)
The canAssumeNoSelfWrap routine in howManyLessThans was doing two subtly
inter-related things. First, it was proving no-self-wrap. This exactly
duplicates the existing logic in the caller. Second, it was establishing
the precondition for the nw->nsw/nuw inference. Specifically, we need to
know that *this* exit must be taken for the inference to be sound.
Otherwise, another (possible abnormal) exit could be taken in the
iteration where this IV would become poison.

This change moves all of that logic into the caller, and caches the
resulting nuw/nsw flags in the AddRec. This centralizes the logic in one
place, and makes it clear that it all depends on controlling the sole
exit.

We do loose a couple cases with SCEV predication. Specifically, if SCEV
predication was able to convert e.g. zext(addrec) into an addrec(zext)
using predication, but didn't record the nuw fact on the new addrec,
then the consuming code can no longer fix this up. I don't think this
case particularly matters.

---------

Co-authored-by: Nikita Popov <github@npopov.com>
2024-08-12 11:40:24 -07:00
Philip Reames
f0944f4be0
[SCEV] Prove no-self-wrap from negative power of two step (#101416)
We have existing code which reasons about a step evenly dividing the
iteration space is a finite loop with a single exit implying
no-self-wrap. The sign of the step doesn't effect this.

---------

Co-authored-by: Nikita Popov <github@npopov.com>
2024-08-01 13:17:07 -07:00
Philip Reames
7583c484c8
[SCEV] Use power of two facts involving vscale when inferring wrap flags (#101380)
SCEV has logic for inferring wrap flags on AddRecs which are known to
control an exit based on whether the step is a power of two. This logic
only considered constants, and thus did not trigger for steps such as (4
x vscale) which are common in scalably vectorized loops.

The net effect is that we were very sensative to the preservation of
nsw/nuw flags on such IVs, and could not infer trip counts if they got
lost for any reason.

---------

Co-authored-by: Nikita Popov <github@npopov.com>
2024-07-31 14:18:20 -07:00
Philip Reames
35a2e6d24b [SCEV] Regen a couple auto-gen tests 2024-07-31 10:38:07 -07:00