We were missing the signed flag on the negative value, so the
range was incorrectly interpreted for integers larger than 64-bit.
Split out from https://github.com/llvm/llvm-project/pull/80309.
The minbitwidth restrictions can be skipped only for immediate reduced
values, for other nodes still need to check if external users allow
bitwidth reduction.
Fixes https://github.com/llvm/llvm-project/issues/104422
Continuing from #102084, which introduced the analysis, we now populate
it with info about functions contained in the module.
When we will update the profile due to e.g. inlined callsites, we'll
ingest the callee's counters and callsites to the caller. We'll move
those to the caller's respective index space (counter and callers), so
we need to know and maintain where those currently end.
We also don't need to keep profiles not pertinent to this module.
This patch also introduces an arguably much simpler way to track the
GUID of a function from the frontend compilation, through ThinLTO, and
into the post-thinlink compilation step, which doesn't rely on keeping
names around. A separate RFC and patches will discuss extending this to
the current PGO (instrumented and sampled) and other consumers as an
infrastructural component.
Given an unsigned integer comparison of `add nsw X, C1` with some
constant `C2` we can fold it into a signed comparison of `X` and `C2 -
C1` under the following conditions:
* There's a `nsw` flag on the addition
* `C2` is non-negative
* `X + C1` is non-negative
* `C2 - C1` is non-negative
DiffChecks will get populated twice when re-trying with runtime checks.
Without clearing it like the regular Checks vector, it will contain some
duplicates and the order the checks are created may not match the order
the checks have been queued when re-trying.
Add a test where diff checks are generated initial and then re-generated
when re-trying with runtime checks.
At the moment, the order doesn't match the order they are created in, as
the DiffChecks field in LAI isn't cleared as other fields holding
runtime checks.
This is a reland of #96287. This change makes tests in logf128.ll ignore
the sign of NaNs for negative value tests and moves an #include <cmath>
to be blocked behind #ifndef _GLIBCXX_MATH_H.
Cyclic dependencies in NewGVN can result in miscompilation and
termination issues.
This patch ensures that the Class Leader is always the member with the
lowest RPO number. This ensures that the Class Leader is processed
before all other members, making the cyclic dependence impossible.
This fixes#35683.
Regressions:
- 'simp-to-self.ll' regresses due to a known limitation in the way
NewGVN and InstSimplify interact. With the new leader, InstSimplify does
not know that %conv is 1 and fails to simplify.
This patch increases scatter overhead on Neoverse-V2 to 13. This
benefits s128 kernel from TSVC_2 test suite.
SPEC 17, RAJAPerf, and Sptter are unaffected by this patch.
This patch boosts s128 kernel's performance from TSVC test suite by about
40% as this enables vectorization. Also, handle minor code refactoring
for gather related part.
When looking at PHI operand for combining, only look at instructions and
arguments. The loop later iteraters over Arg's users, which is not
useful if Arg is a constant -- it's users are not meaningful and might
be in different functions, which causes problems for the dominates()
query.
Pull Request: https://github.com/llvm/llvm-project/pull/103302
We should undo this https://reviews.llvm.org/D102007 after undoing
https://reviews.llvm.org/D104477 to use missing probe to represent
unknown count, zero count and unknown count are different to profile
inference.
It only affects post-inline(linker) pipeline(with
`--overwrite-existing-weights` on ) and flow-sensitive FDO.
This patch updates the GlobalMerge pass to be able to handle private
global variables, which is required for a follow-up PowerPC specific
GlobalMerge patch to merge internal and private globals.
A new LIT test is also added to exhibit the ability to merge private
globals.
A dominance query of a block that is in a different function is
ill-defined, so assert that getNode() is only called for blocks that are
in the same function.
There are three cases, where this behavior did occur. LoopFuse didn't
explicitly do this, but didn't invalidate the SCEV block dispositions,
leaving dangling pointers to free'ed basic blocks behind, causing
use-after-free. We do, however, want to be able to dereference basic
blocks inside the dominator tree, so that we can refer to them by a
number stored inside the basic block.
Reverts #102780
Reland #101198Fixes#102784
Co-authored-by: Alexis Engelke <engelke@in.tum.de>
computeConstantDifference() can currently look through addrecs with
identical steps, and then through adds with identical operands (apart
from constants).
However, it fails to handle minor variations, such as two nested add
recs, or an outer add with an inner addrec (rather than the other way
around).
This patch supports these cases by adding a loop over the
simplifications, limited to a small number of iterations. The motivation
is the same as in #101339, to make
computeConstantDifference() powerful enough to replace existing uses of
`dyn_cast<SCEVConstant>(getMinusSCEV())` with it. Though as the IR test
diff shows, other callers may also benefit.
Add getShufflevectorNumGroups to vectorize shufflevector.
Current getShufflevectorNumGroups can only vectorize limited pattern
(e.g., the masks of shufflevector use the elements of the source in
order).
In addition, ReuseShuffleIndices and ReorderIndices are not supported.
- [AArch64]: TargetLowering is updated to spot load/store (de)interleave4 like sequences using PatternMatch,
and emit equivalent sve.ld4 and sve.st4 intrinsics.
Now that the branches to the scalar epilogue are modeled in VPlan
directly, check the VPlan to see if a scalar epilogue is required.
Preparation for https://github.com/llvm/llvm-project/pull/100658.
Currently SLP vectorizer tries to keep only GEPs as scalar, if they are
vectorized but used externally. Same approach can be used for all scalar
values. This patch tries to keep original scalars if all its operands
remain scalar or externally used, the cost of the original scalar is
lower than the cost of the extractelement instruction, or if the number
of externally used scalars in the same entry is power of 2. Last
criterion allows better revectorization for multiply used scalars.
Reviewers: RKSimon
Reviewed By: RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/100904
Include chain of ops feeding inductions in cost precomputation for
inductions, not just the induction increment. In VPlan, those
instructions will be cleaned up, as both phi and increment are generated
by VPWidenIntOrFpInductionRecipe independently.
Fixes https://github.com/llvm/llvm-project/issues/101337.
The Mul factor was zero-extended here, resulting in incorrect
results for integers larger than 64-bit.
As we currently only multiply by 1 or -1, just split this into
two cases -- there's no need for a full multiplication here.
Fixes https://github.com/llvm/llvm-project/issues/102597.
transformConstExprCastCall() implements a number of highly dubious
transforms attempting to make a call function type line up with the
function type of the called function. Historically, the main value this
had was to avoid function type mismatches due to pointer type
differences, which is no longer relevant with opaque pointers.
This patch is a step towards reducing the scope of the transform, by
applying it only to definitions, not declarations. For declarations, the
declared signature might not match the actual function signature, e.g.
`void @fn()` is sometimes used as a placeholder for functions with
unknown signature. The implementation already bailed out in some cases
for declarations, but I think it would be safer to disable the transform
entirely.
For the test cases, I've updated some of them to use definitions
instead, so that the test coverage is preserved.
After f0df4fbd0c7b, isPredicatedInst needs to handle SwitchInst as well.
Handle it the same as BranchInst.
This fixes a crash in the newly added test and improves the results for
one of the existing tests in predicate-switch.ll
Should fix https://lab.llvm.org/buildbot/#/builders/113/builds/2099.
