Make codegen emit correctly rounded sqrt by default.
Emit the fast but only kind of fast expansion in AMDGPUCodeGenPrepare
based on !fpmath, like the fdiv case. Hack around visitation ordering
problems from AMDGPUCodeGenPrepare using forward iteration instead of
a well behaved combiner.
https://reviews.llvm.org/D158129
Fixes a corner case of the analysis: previously GlobalValues could be
affected by assumptions, but were not tracked within AffectedValues.
This patch allows assumptions which affect a given GlobalValue to be
looked up via `assumptionsFor()`.
A small update to llvm/test/Analysis/ScalarEvolution/ranges.ll was
necessary due to knowledge about a global value now being propagated
from AssumptionCache -> ValueTracking -> ScalarEvolution.
After D154102 multi-line labels would get split incorrectly.
When CFG is generated for a function with basic block name longer
than 80 lines, then the header separator will be placed after the
line break for the label name instead of after the whole label name.
The fix is simple by just moving the insert of | character before the
line splitting happens.
Differential Revision: https://reviews.llvm.org/D159207
This adds some basic and/or/xor reduction costs for NEON/MVE, handling them
like other reductions where vector operations are used to reduce to legal
sizes, followed by an optional VREV+VAND/VORR/VEOR step and scalarization from
there.
This adds some basic smin/smax/umin/umax reduction costs for MVE/NEON, similar
to the existing Add reduction costs. They follow the same style as Add
reductions, but include a higher cost as the costs tend to be dependant on the
element size for vminv/vmaxv. These costs may not be precise, but will be more
inline than the default that extracts each element.
Similar to the other reductions, this changes the cost of fmin/fmax reductions
under MVE/NEON to perform vector operations until the types need to be
scalarized. The fp16 vectors can perform a VREV+FMIN/FMAX to skip a step of the
reduction, and otherwise need lanewise extract fro the top lanes.
This adds some basic fadd/fmul reduction costs for MVE/NEON. It reduces by
halving the vector size until it it gets scalarized, with some additional costs
for fp16 which may require extracting the top lanes.
Differential Revision: https://reviews.llvm.org/D159367
The following commit enabled the analysis of ranges for heap allocations:
22ca38da25e19a7c5fcfeb3f22159aba92ec381e
The range turns out to be empty in cases such as the one in test (which
is [1,1)), leading to an assertion failure. This patch fixes for the
same case.
Fixes https://github.com/llvm/llvm-project/issues/63856
Reviewed By: fhahn
Differential Revision: https://reviews.llvm.org/D159160
Bug was introduced in: https://reviews.llvm.org/D157807
The prior logic assumed that the information from the knownbits
returned from analyzing the `icmp` and its operands in the context
basicblock would be consistent.
This is not necessarily the case if we are analyzing deadcode.
For example with `(icmp sgt (select cond, 0, 1), -1)`. If we take
knownbits for the `select` using knownbits from the operator, we will
know the signbit is zero. If we are analyzing a not-taken from based
on the `icmp` (deadcode), we will also "know" that the `select` must
be negative (signbit is one). This will result in a conflict in
knownbits.
The fix is to just give up on analying the phi-node if its deadcode. We 1) don't want to waste time continuing to analyze it and 2) will be removing it (and its dependencies) later.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D158960
This has two benefits:
* we can now mark allocas that are used in atomic operations as safe
* this fixes a bug that would incorrectly mark all atomic writes as safe
in HWASan instrumentation. this is because stack safety keeps a list
of all *unsafe* operations that are reachable from an alloca, but it
did not analyze atomic writes, so it would always mark them as safe.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D159153
In particular, high LMULs, constant offsets within high LMUL, and types which require splitting. Note that most of these are way off with current lowering.
This patch makes structural-hash-detailed.ll look at diffs rather than
specific values to make the test more robust against the underlying
hashing implementation while still testing that the hash value is
deterministic across multiple runs.
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D159154
This patch adds in a StructuralHash printer pass that prints out the
hexadeicmal representation of the hash of a module and all of the
functions within it.
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D158317
When SVE2 is enabled, we can combine an add of 1, add & shift right by 1
to a single s/urhadd instruction. If the operands to the adds are extended,
these extends will fold into the s/urhadd and their costs should be 0.
Reviewed By: david-arm, dtemirbulatov
Differential Revision: https://reviews.llvm.org/D157628
This patch adds in a couple more properties related to call instructions
and the CFG within the function that should expose a little bit more
about the characteristics of the function.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D158681
%x = shl i64 %w, n
%y = add i64 %x, c
%z = ashr i64 %y, m
The above given instruction triplet is seen many times in the generated
LLVM IR, but SCEV model is not able to compute the SCEV value of AShr
instruction in this case.
This patch models the two cases of the above instruction pattern using
the following expression:
=> sext(add(mul(trunc(w), 2^(n-m)), c >> m))
1) when n = m the expression reduces to sext(add(trunc(w), c >> n))
as n-m=0, and multiplying with 2^0 gives the same result.
2) when n > m the expression works as given above.
It also adds several unittest to verify that SCEV is able to compute
the value.
$ opt sext-add-inreg.ll -passes="print<scalar-evolution>"
Comparing the snippets of the result of SCEV analysis:
* SCEV of ashr before change
----------------------------
%idxprom = ashr exact i64 %sext, 32
--> %idxprom U: [-2147483648,2147483648) S: [-2147483648,2147483648)
Exits: 8 LoopDispositions: { %for.body: Variant }
* SCEV of ashr after change
---------------------------
%idxprom = ashr exact i64 %sext, 32
--> {0,+,1}<nuw><nsw><%for.body> U: [0,9) S: [0,9)
Exits: 8 LoopDispositions: { %for.body: Computable }
LoopDisposition of the given SCEV was LoopVariant before, after adding
the new way to model the instruction, the LoopDisposition becomes
LoopComputable as it is able to compute the SCEV of the instruction.
Differential Revision: https://reviews.llvm.org/D152278
Use the comparison based analysis to strengthen the standard
knownbits analysis rather than choosing either/or.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D157807
This is basically a copy and paste of the same logic we do in
`computeKnownBits` but adapts it for just `isKnownNonZero`.
Differential Revision: https://reviews.llvm.org/D157801
Just fill in missing cases (TODO) for `ugt`, `uge`, `sgt`, `sge`,
`slt`, and `sle`. These are all in the same spirit as `ult`/`uge`, but
each of the other conditions have different constraints.
Proofs: https://alive2.llvm.org/ce/z/gnj4o-
Differential Revision: https://reviews.llvm.org/D157800
Refresh of the generic scheduling model to use A510 instead of A55.
Main benefits are to the little core, and introducing SVE scheduling information.
Changes tested on various OoO cores, no performance degradation is seen.
Differential Revision: https://reviews.llvm.org/D156799
This patch adds operand type counts to the detailed function properties
analysis. This is intended to enable more interesting and detailed
comparisons across different languages on specific metrics (like usage
of inline assembly or global values).
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D158018
When diverged paths reach an irreducible cycle C, every block inside C gets
marked as a join block. Such a join block J may be contained in a nest of
reducible cycles inside C. When visiting J, we can only expect that the
outermost C is irreducible, which we now correctly assert.
As the test case demonstrates, it is possible for a block to be identified as a
join point while not being the header of a reducible cycle. To address this,
when searching for the outermost cycle made divergent by branch outside it, we
first check for an irreducible outermost cycle before checking if the parent is
reducible.
Reviewed By: sameerds
Differential Revision: https://reviews.llvm.org/D158014
The subtarget was unconditionally reporting that SVE was to be used to
lower vectors when Neon was unavailable, even when SVE itself was
unavailable. This decision leads other parts of the compiler to crash,
e.g., when querying SVE vector sizes.
Reviewed By: sdesmalen
Differential Revision: https://reviews.llvm.org/D158179
This change adds separators for basic block names, which makes it
easier to find a basic block based on its name and separates it
from the code.
Currently there is also a chance that the basic block label will
be present twice, that is in case the basic block has explicit
numbering, this change fixes this bug.
Differential Revision: https://reviews.llvm.org/D154102
`MaxSafeDepDistBytes` was not correct based on its name an semantics
in instances when there was a non-unit stride loop. For example,
```
for (int k = 0; k < len; k+=3) {
a[k] = a[k+4];
a[k+2] = a[k+6];
}
```
Here, the smallest dependence distance is 24 bytes, but only vectorizing 8 bytes
is safe. `MaxSafeVectorWidthInBits` reported the correct number of bits
that could be vectorized as 64 bits.
The semantics of of `MaxSafeDepDistBytes` should be:
The smallest dependence distance in bytes in the loop. This may not be
the same as the maximum number of bytes that are safe to operate on
simultaneously.
The name of this variable should reflect those semantics and
its docstring should be updated accordingly, `MinDepDistBytes`.
A debug message that used `MaxSafeDepDistBytes` to signify to the user
how many bytes could be accessed in parallel is updated to use
`MaxSafeVectorWidthInBits` instead. That way, the same message if
communicated to the user, just in different units.
This patch makes sure that when `MinDepDistBytes` is modified in a way
that should impact `MaxSafeVectorWidthInBits`, that we update the latter
accordingly. This patch also clarifies why `MaxSafeVectorWidthInBits`
does not to be updated when `MinDepDistBytes` is (i.e. in the case of a
forward dependency).
Differential Revision: https://reviews.llvm.org/D156158
This check is redundant as it is covered by the call to
isPotentiallyReachable.
Depends on D155726.
Differential Revision: https://reviews.llvm.org/D155718
Try to avoid some unprofitable predication on PPC. Recognize in the cost model that computing on i1 values will require extra mask or compare operation.
Differential Revision: https://reviews.llvm.org/D155876
If an argument is readnone we know that it isn't dereferenced. Then
it should be OK if that argument alias with a noalias argument.
Differential Revision: https://reviews.llvm.org/D157737
This patch adds more detailed function properties gated behind a command
line flag for use primarily in experimentation and gathering statistics
on the functions in a module or project. The runtime cost should be
minimal as the computation is only done when the flag is set. There will
be a slight memory overhead when the ML inliner is enabled, but it
should be fairly small at a handful of bytes per function.
This is an adapted form of https://reviews.llvm.org/D109661.
Reviewed By: mtrofin
Differential Revision: https://reviews.llvm.org/D157358
This test is checking alias analysis. The attribute tests are
brittle but fortunately unneccesary. Delete them.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D156600
