This was only called on CondBr instructions, where it is always faster
to access the successors directly than to use successors().
Multi-edges don't dominate anything, so this rare case is often already
handled by dominates().
There is also a very small (hardly measurable) performance
improvement here (it did show up in profiles at 0.03% or so).
Restrict PHI nodes that getRangeRef is allowed to recursively examine so
we don't need a "visited" set. And fix createSCEVIter so it creates all
the relevant SCEV nodes before getRangeRef tries to examine them.
The tests that are affected have induction variables that aren't
AddRecs. (Other cases are theoretically affected, but don't seem to show
up in our tests.)
Add a ConstantExpr::getPtrAdd() API that creates a getelementptr i8
constant expression, similar to IRBuilder::CreatePtrAdd(). In the future
this will create a ptradd expression.
`Constant::isZeroValue` currently behaves same as
`Constant::isNullValue` for all types except floating-point, where it
additionally returns true for negative zero (`-0.0`). However, in
practice, almost all callers operate on integer/pointer types where the
two are equivalent, and the few FP-relevant callers have no meaningful
dependence on the `-0.0` behavior.
This PR removes `isZeroValue` to eliminate the confusing API. All
callers are changed to `isNullValue` with no test failures.
`isZeroValue` will be reintroduced in a future change with clearer
semantics: when null pointers may have non-zero bit patterns,
`isZeroValue` will check for bitwise-all-zeros, while `isNullValue` will
check for the semantic null (which
may be non-zero).
If the predicate has samesign set, we could either perform the checks
with the unsigned predicate and return and unsigned invariant predicate,
or we could perform them with the signed predicate and return a signed
invariant predicate. The current implementation can end up mixing both,
using a signed predicate for one check and an unsigned one for the
other.
Avoid this by dropping the samesign flag.
Fixes https://github.com/llvm/llvm-project/issues/180870.
Move SCEVPtrToIntSinkingRewriter out of getLosslessPtrToIntExpr to be
re-used for PtrToAddr. Also streamline code in getLosslessPtrToIntExpr
by moving zero handling to the rewriter and removing special handling
for SCEVUnknown in getLosslessPtrToIntExpr. Instead, always use the
rewriter, which will automatically handle the case where the expression
is a SCEVUnknown.
This makes it slightly easier to add support for PtrToAddr as follow-up
to https://github.com/llvm/llvm-project/pull/158032
PR: https://github.com/llvm/llvm-project/pull/174435
Split out from https://github.com/llvm/llvm-project/pull/171456.
This explicitly allows implicit truncation in a number of places,
prior to switching the default. This limits the scope of the
initial change.
getSCEVExprForVPValue is used to create SCEVs for expressions from the
original loop, which may be predicated. Use PSE to construct predicated
SCEVs if possible. This matches the legacy LV code behavior.
Currently should be NFC, but will enable migrating more SCEV/cost-based
computations to VPlan.
The patch requires exposing a new getPredicatedSCEV helper to
PredicatedScalarEvolution which just takes a SCEV, to avoid needing to
go through IR values, which isn't an option for getSCEVExprForVPValue.
This is a slightly different API than ConstantRange's
areInsensitiveToSignednessOfICmpPredicate. The only actual difference
(beyond naming) is the handling of empty ranges (i.e. unreachable code).
I wanted to keep the existing SCEV behavior for the unreachable code as
we should be folding that to poison, not reasoning about samesign. I
tried the other variant locally, and saw no test changes.
At the moment, the effectivness of guards that contain divisibility
information (A % B == 0 ) depends on the order of the conditions.
This patch makes using divisibility information independent of the
order, by collecting and applying the divisibility information
separately.
We first collect all conditions in a vector, then collect the
divisibility information from all guards.
When processing other guards, we apply divisibility info collected
earlier.
After all guards have been processed, we add the divisibility info,
rewriting the existing rewrite. This ensures we apply the divisibility
info to the largest rewrite expression.
This helps to improve results in a few cases, one in
https://github.com/dtcxzyw/llvm-opt-benchmark/pull/2921 and another one
in a different large C/C++ based IR corpus.
PR: https://github.com/llvm/llvm-project/pull/163021
Move getPreviousSCEVDivisibleByDivisor from a lambda to a static
function and clarify the name (DividesBy -> DivisibleBy).
Split off refactoring from https://github.com/llvm/llvm-project/pull/163021.
Add a new variant of m_scev_Mul that binds a SCEVMulExpr and use it in
SCEVURem_match and also update 2 more places in ScalarEvolution.cpp that
can use m_scev_Mul as well.
PR: https://github.com/llvm/llvm-project/pull/163364
Follow-up to https://github.com/llvm/llvm-project/pull/160941.
Even if we don't have a context instruction for the caller, we should be
able to provide context instructions for SCEVUnknowns. Unless I am
missing something, SCEVUnknown only become available at the point their
underlying IR instruction has been defined. If it is an argument, it
should be safe to use the first instruction in the entry block or the
instruction itself if it wraps an instruction.
This allows getConstantMultiple to make better use of alignment
assumptions.
PR: https://github.com/llvm/llvm-project/pull/163260
My usecase is simplifying the control flow generated by LoopVectorize
when vectorising loops whose tripcount is a function of the runtime
vector length. This can be problematic because:
* CSE is a pre-LoopVectorize transform and so it's common for an IR
function to include several calls to llvm.vscale(). (NOTE: Code
generation will typically remove the duplicates)
* Pre-LoopVectorize instcombines will rewrite some multiplies as shifts.
This leads to a mismatch between VL based maths of the scalar loop and
that created for the vector loop, which prevents some obvious
simplifications.
SCEV does not suffer these issues because it effectively does CSE during
construction and shifts are represented as multiplies.
When adding a new predicate to a union, we currently do a bidirectional
implication for all the contained predicates. This means that the number
of implication checks is quadratic in the number of total predicates (if
they don't end up being eliminated).
Fix this by not checking for implication if the number of predicates
grows too large. The expectation is that if there is a large number of
predicates, we should be discarding them later anyway, as expanding them
would be too expensive.
Fixes https://github.com/llvm/llvm-project/issues/156114.
Reverts llvm/llvm-project#157656
There are multiple reports that this is causing miscompiles in the MSan
test suite after bootstrapping and that this is causing miscompiles in
rustc. Let's revert for now, and work to capture a reproducer next week.
If we have a phi where one of it's source blocks is an unreachable
block, we don't want to traverse back into the unreachable region. Doing
so allows e.g. finding a trivial self loop when walking back the
predecessor chain.
