There are a number of issues with the current code for converting
ule -> ult (etc) predicates for comparisons controlling finite loops:
* It sets nowrap flags, which may only hold for that particular
comparison, not globally. (PR60944)
* It doesn't check that the RHS is invariant. (I'm not sure this
can cause practical issues independently of the previous point.)
* It runs before simplifications that may be more profitable. (PR54191)
This patch moves the handling for this into computeExitLimitFromICmp(),
because it is somewhat tightly coupled with assumptions in that code,
and addresses the aforementioned issues.
Fixes https://github.com/llvm/llvm-project/issues/60944.
Fixes https://github.com/llvm/llvm-project/issues/54191.
Differential Revision: https://reviews.llvm.org/D145510
This replaces several rewriting rules in ScalarEvolution::applyLoopGuards
that are applied to min/max expressions with the equivalent ones but
applied to its arguments.
So previously given we had a loop guard min(a, b) >= c,
the min expression would get rewritten as max(c, min(a, b)).
With such approach, we were unable to apply the rewrite if min operands
were zext for example (min(zext(a), zext(b))), however it's equivalent
to the expression zext(min(a, b)) for which we could apply the rewrite.
Now we'd rewrite the min operands also with these expressions:
a -> max(c, a) and
b -> max(c, b).
and this would allow us to apply the loop guard in this and similar cases:
min(zext(a), zext(b)) would get rewritten as min(zext(max(c, a)), zext(max(c, b)))
instead of just being skipped.
The list of added rules (omitting predicates signedness for simplicity):
1. Guard: min(a, b) >= c
Old rule: min(a, b) -> max(c, min(a, b))
New rules: a -> max(a, c) and b -> max(b, c)
2. Guard: min(a, b) > c
Old rule: min(a, b) -> max(c + 1, min(a, b))
New rules: a -> max(a, c + 1) and b -> max(b, c + 1)
3. Guard: max(a, b) <= c
Old rule: max(a, b) -> min(c, max(a, b))
New rules: a -> min(a, c) and b -> min(b, c)
4. Guard: max(a, b) < c
Old rule: max(a, b) -> min(c - 1, max(a, b))
New rules: a -> min(a, c - 1) and b -> min(b, c - 1)
The old rewrites still hold.
Differential Revision: https://reviews.llvm.org/D145230
Just the scevUnconditionallyPropagatesPoisonFromOperands() check
is sufficient. Also rename the flag to be more in line with the
more general predicate.
This factors out two utilities used with RewriteMap in applyLoopGuards:
- AddRewrite, which puts a rewrite rule in the map and if needed registers
the rewrite in the list of rewritten expressions,
- GetMaybeRewritten, which checks whether an expression has already been
rewritten, and if so, returns the rewrite. Otherwise, returns the given
expression.
This may be needed when adding new rewrite rules as not to copy-paste this
code.
This is part of an effort to remove ConstantExpr based
representations of `vscale` so that its LangRef definiton can
be relaxed to accommodate a less strict definition of constant.
Differential Revision: https://reviews.llvm.org/D144891
Represent scalable type sizes using C * vscale, where vscale is
the vscale constant expression. This exposes a bit more information
to SCEV, because the vscale multiplier is explicitly modeled in SCEV
(rather than part of the sizeof expression).
This is mainly intended as an alternative to D143642.
Differential Revision: https://reviews.llvm.org/D144624
Passes in general shouldn't replace an alias with the aliasee (see
https://reviews.llvm.org/D66606). This can lead to situations where a
linkonce_odr symbol (which could be interposable if lowered to weak
linkage) can be replaced with a local aliasee which won't be
interposable. SVEC does this when the function is invoked by
FunctionPass Manager -> Loop Pass Manager -> Induction Variable Users in
the codegen pipeline. This was found in hwasan instrumented code where a
linonce_odr alias was replaced with its private aliasee.
This fixes the bug descriped at
https://github.com/llvm/llvm-project/issues/60668.
Differential Revision: https://reviews.llvm.org/D144035
applyLoopGuards doesn't always preserve information when there are multiple assumes.
This patch tries to deal with multiple assumes regarding a SCEV's divisibility and min/max values, and rewrite it into a SCEV that still preserves all of the information.
For example, let the trip count of the loop be TC. Consider the 3 following assumes:
1. __builtin_assume(TC % 8 == 0);
2. __builtin_assume(TC > 0);
3. __builtin_assume(TC < 100);
Before this patch, depending on the assume processing order applyLoopGuards could create the following SCEV:
max(min((8 * (TC / 8)) , 99), 1)
Looking at this SCEV, it doesn't preserve the divisibility by 8 information.
After this patch, depending on the assume processing order applyLoopGuards could create the following SCEV:
max(min((8 * (TC / 8)) , 96), 8)
By aligning up 1 to 8, and aligning down 99 to 96, the new SCEV still preserves all of the original assumes.
Differential Revision: https://reviews.llvm.org/D141850
These shouldn't really reach SCEV without being folded away first,
and we don't have any tests that hit these cases.
The sizeof case does occur with scalable types.
This reverts commit f9599bbc7a3f831e1793a549d8a7a19265f3e504.
For some reason it caused us a huge compile time regression in downstream
workloads. Not sure whether the source of it is in upstream code ir not.
Temporarily reverting until investigated.
Differential Revision: https://reviews.llvm.org/D142330
I stumbled over this while trying to improve our exit count work. These expressions
are equivalent for complementary signed/unsigned ext and min/max (including umin_seq),
but they are not canonicalized and SCEV cannot recognize them as the same.
The benefit of this canonicalization is that SCEV can prove some new equivalences which
it coudln't prove because of different forms. There is 1 test where trip count seems pessimized,
I could not directly figure out why, but it just seems an unrelated issue that we can fix.
Other changes seem neutral or positive to me.
Differential Revision: https://reviews.llvm.org/D141481
Reviewed By: nikic
I guess its only reason to exist is potential CT optimization, otherwise it is
just creating cohesion between this code and rewriter internals. We plan to
extend the rewriter. I'd rather not have this cohesion, unless there is a serious
reason to have it.
Differential Revision: https://reviews.llvm.org/D143246
If LHS <u RHS holds, RHS should be guaranteed to be > 0. By using
using 'umax(RHS, 1) -1' instead of 'RHS - 1' the results in
applyLoopGuards can be improved in some cases.
Note that the TODO for the tests mentioned the max BTC being 11, but
unless I am missing something 10 should be correct.
https://alive2.llvm.org/ce/z/44nP7F
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D126503
As discussed in https://github.com/llvm/llvm-project/issues/59901
This change is not NFC. There is one SCEV and EarlyCSE test that have an
improved analysis/optimization case. Rest of the tests are not failing.
I've mostly only added cleanup to SCEV since that is where this issue
started. As a follow up, I believe there is more cleanup opportunity in
SCEV and other affected passes.
There could be cases where there are missed registerAssumption of
guards, but this case is not so bad because there will be no
miscompilation. AssumptionCacheTracker should take care of deleted
guards.
Differential Revision: https://reviews.llvm.org/D142330
I'm not really sure the problem can be nicely exposed via a lit test,
since we don't give up on range calculation for deeply nested ranges,
but if i add an assertion that those opcodes are never encountered,
the assertion fails in a number of existing tests.
In reality, the iterative approach is still pretty partial:
1. `Seen` should not be there. We want the last instance of expression, not the first one
2. There should be a check that `getRangeRefIter()` does not self-recurse
And, this shows a bug in the original code:
why do we not recurse into casts?
If i add an assertion that those opcodes are never encountered,
the assertion fails in a number of existing tests.
Otherwise instead of a compile-time error that you forgot to modify it,
you'd get a run-time error, which happened every time i've added new expr.
This is completely NFC, there are no other changes here.
There is already support for U/SMax. No reason why Min and SequentialMin
should not be supported.
NFC: code in GetMinTrailingZeroes is copied for a couple node types.
Refactor them into a single code block.
Differential Revision: https://reviews.llvm.org/D141568
Conditions for dominating branches and llvm.assumes are already
collected. This also adds conditions from guards.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D141243
The current code did not properly handled duplicated FoldCacheUser ID
entries when overwriting an existing entry in the FoldCache.
This triggered verification failures reported by @uabelho and #59721.
The patch fixes that by removing stale IDs when overwriting an existing
entry in the cache.
Fixes#59721.
