Add a test in which the trip count could have been calculated
correctly thanks to an assume since it is known to divide by the
step of the IV.
Patch by Alon Kom!
Differential Revision: https://reviews.llvm.org/D143644
This effectively reverts 5c38c6a and 4f772b0.
A recently introduced LazyValueInfo::getConstantRangeAtUse returns incorrect
ranges for values in certain cases. One such example is described in PR60629.
The issue has something to do with traversing PHI uses of a value transitively.
As nikic pointed out, we're effectively reasoning about values from different
loop iterations.
In the faulting test case, CVP made a miscompilation because the calculated
range for a shift argument was incorrect. It returned empty-set, however it is
clearly not a dead code. CVP then erased the shift instruction because
of empty range.
An indexed LD1 instruction, or "ASIMD load, 1 element, one lane, B/H/S"
instruction that loads a value and inserts an element into a vector is
an expensive instruction. It has a latency of 8 on modern cores. We
generate an indexed LD1 when an insertelement instruction has a load as an
operand and this patch is recognising and makes indexed LD1 more expensive.
Differential Revision: https://reviews.llvm.org/D141602
IsNeverUniform can be set to 1 to mark instructions which are
inherently never-uniform/divergent. Enabling this bit in
Writelane instruction for now. To be extended to all required
instructions.
Reviewed By: arsenm, sameerds, #amdgpu
Differential Revision: https://reviews.llvm.org/D143154
This test is equivalent to another one, where we can infer a
reasonable iteration count. The only difference is that umin and zext
here are swapped, and rewriter fails here.
If LHS <u RHS holds, RHS should be guaranteed to be > 0. By using
using 'umax(RHS, 1) -1' instead of 'RHS - 1' the results in
applyLoopGuards can be improved in some cases.
Note that the TODO for the tests mentioned the max BTC being 11, but
unless I am missing something 10 should be correct.
https://alive2.llvm.org/ce/z/44nP7F
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D126503
As discussed in https://github.com/llvm/llvm-project/issues/59901
This change is not NFC. There is one SCEV and EarlyCSE test that have an
improved analysis/optimization case. Rest of the tests are not failing.
I've mostly only added cleanup to SCEV since that is where this issue
started. As a follow up, I believe there is more cleanup opportunity in
SCEV and other affected passes.
There could be cases where there are missed registerAssumption of
guards, but this case is not so bad because there will be no
miscompilation. AssumptionCacheTracker should take care of deleted
guards.
Differential Revision: https://reviews.llvm.org/D142330
LoopUnroll estimates the loop size via getInstructionCost(),
but getInstructionCost() cannot pass CostKind to getVectorInstrCost().
And so does getShuffleCost() to getBroadcastShuffleOverhead(),
getPermuteShuffleOverhead(), getExtractSubvectorOverhead(),
and getInsertSubvectorOverhead().
To address this, this patch adds an argument CostKind to these
functions.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D142116
Delete mention of the llvm.eh.begincatch/llvm.eh.endcatch intrinsics,
and remove them from a few remaining test-cases. These intrinsics were
from a previous attempt at implementing Windows exception-handling,
but were removed from LLVM in 2015.
Also mention that dynamic exception specifications ("throw filters")
were removed from the spec in C++17.
There is already support for U/SMax. No reason why Min and SequentialMin
should not be supported.
NFC: code in GetMinTrailingZeroes is copied for a couple node types.
Refactor them into a single code block.
Differential Revision: https://reviews.llvm.org/D141568
I do not believe any code can hit this, but these do not give
a uniform answer with all unifirm sources.
Differential Revision: https://reviews.llvm.org/D141544
We could have same SCEVs for the following expressions:
zext(umin(x, y)) and umin(zext(x), zext(y));
zext(umax(x, y)) and umax(zext(x), zext(y))
sext(smin(x, y)) and smin(zsxt(x), sext(y));
sext(smax(x, y)) and smax(sext(x), sext(y)).
Intrinsics have historically been excluded from the call graph with an
exception of 3 special ones added at some point. This meant that passes
depending on the call graph needed to handle intrinsics explicitly as
the underlying assumption, namely that intrinsics can't call or modify
things, doesn't hold. We are slowly moving away from special handling of
intrinsics, or at least towards explicitly checking what intrinsics we
want to handle differently.
This patch:
- Includes most intrinsics in the call graph. Debug intrinsics are
still excluded.
- Removes the special handling of intrinsics in the GlobalsAA pass.
- Removes the `IntrinsicInst::isLeaf` method.
Properly
Fixes: https://github.com/llvm/llvm-project/issues/52706
See also:
https://discourse.llvm.org/t/intrinsics-are-not-special-stop-pretending-i-mean-it/67545
Differential Revision: https://reviews.llvm.org/D14119
Conditions for dominating branches and llvm.assumes are already
collected. This also adds conditions from guards.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D141243
1. Refactor for costs of sqrt/fabs
2. Add half type support for the cost model of sqrt/fabs
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D132908
Need to include the cost of the initial insertelement to the cost of the
broadcasts. Also, need to adjust the cost of the gather/buildvector if
the element is inserted into poison/undef vector.
Differential Revision: https://reviews.llvm.org/D140498
The patch also adds expandVPCTLZ and expandVPCTTZ to expand vp.ctlz/cttz nodes
and the cost model of vp.ctlz/cttz.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D140370
The current code did not properly handled duplicated FoldCacheUser ID
entries when overwriting an existing entry in the FoldCache.
This triggered verification failures reported by @uabelho and #59721.
The patch fixes that by removing stale IDs when overwriting an existing
entry in the cache.
Fixes#59721.
Uniformity analysis is a generalization of divergence analysis to
include irreducible control flow:
1. The proposed spec presents a notion of "maximal convergence" that
captures the existing convention of converging threads at the
headers of natual loops.
2. Maximal convergence is then extended to irreducible cycles. The
identity of irreducible cycles is determined by the choices made
in a depth-first traversal of the control flow graph. Uniformity
analysis uses criteria that depend only on closed paths and not
cycles, to determine maximal convergence. This makes it a
conservative analysis that is independent of the effect of DFS on
CycleInfo.
3. The analysis is implemented as a template that can be
instantiated for both LLVM IR and Machine IR.
Validation:
- passes existing tests for divergence analysis
- passes new tests with irreducible control flow
- passes equivalent tests in MIR and GMIR
Based on concepts originally outlined by
Nicolai Haehnle <nicolai.haehnle@amd.com>
With contributions from Ruiling Song <ruiling.song@amd.com> and
Jay Foad <jay.foad@amd.com>.
Support for GMIR and lit tests for GMIR/MIR added by
Yashwant Singh <yashwant.singh@amd.com>.
Differential Revision: https://reviews.llvm.org/D130746
When converting this test to opaque pointers (and dropping bitcast),
we get improved memory checks. Per fhahn:
> It looks like the difference is due to the logic that determines
> pointer strides in LAA not handling bitcasts. Without the
> bitcasts, the logic now triggers successfully.
Differential Revision: https://reviews.llvm.org/D140204
This patch updates propgatesPoison to take a Use as argument and
propagatesPoison now returns true if the passed in operand causes the
user to yield poison if the operand is poison
This allows propagating poison if the condition of a select is poison.
This helps improve results for programUndefinedIfUndefOrPoison.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D111643
This operand bundle on an assume informs alias analysis that the
arguments point to regions of memory that were allocated separately
(i.e. different heap allocations, different allocas, or different
globals).
As a safety measure, we leave the analysis flag-disabled by default.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D136514
Opaque ptr types have a size in bits of 0. The legalised type is an i64 or
vector of i64s, which do have a size. Because of this difference in size, target
hook getMemoryOpCost modelled stores of ptr types as extending/truncating
load/stores. Now we just check for opaque ptr types and return the legalised
cost. This makes stores of pointers cheaper, and as a result we now SLP
vectorise the changed test case.
Differential Revision: https://reviews.llvm.org/D140193
