https://alive2.llvm.org/ce/z/YGT5SNhttps://alive2.llvm.org/ce/z/PVDxCwhttps://alive2.llvm.org/ce/z/8buR2N
This is tricky because with positive numbers, we only go up, so we can
in fact always hit the signed_max boundary. This is important because
the intrinsic we use has the behavior of going the OTHER way, aka clamp
to INT_MIN if it goes in that direction.
And the range checking we do only works for positive numbers.
Because of this issue, we can only do this for constants as well.
This is a small code size optimization that lets us avoid both shifting
and comparing to a constant if we need the shifted value anyway. On most
architectures the zero comparison is cheaper than a constant comparison
(or free if the shift sets flags).
Although this change appears to remove the optimization entirely, we
continue to do this transform if there is one use because of the code
below the removed code that transforms the shift into an and, followed
by the PR10267 case in InstCombinerImpl::foldICmpAndConstConst that
transforms the and into a ult/ugt. Added a test case to verify this
explicitly.
Per [1] reduces clang .text size by 0.09% and dynamic instruction count
by 0.01%.
[1] https://llvm-compile-time-tracker.com/compare.php?from=1f38d49ebe96417e368a567efa4d650b8a9ac30f&to=0873787a12b8f2eab019d8211ace4bccc1807343&stat=size-text
Reviewers: nikic, dtcxzyw
Reviewed By: dtcxzyw
Pull Request: https://github.com/llvm/llvm-project/pull/168007
A new InstCombine transform uses this attribute to rewrite calls to a
modular version of the implementation along with llvm.reloc.none
relocations against aspects of the implementation needed by the call.
This change only adds support for the 'float' aspect, but it also builds
the structure needed for others.
See issue #146159
These checks ensure that retained nodes of a DISubprogram belong to the
subprogram.
Tests with incorrect IR are fixed. We should not have variables of one subprogram present in retained nodes of other subprograms.
Also, interface for accessing DISubprogram's retained nodes is slightly
refactored. `DISubprogram::visitRetainedNodes` and
`DISubprogram::forEachRetainedNode` are added to avoid repeating checks
like
```
if (const auto *LV = dyn_cast<DILocalVariable>(N))
...
else if (const auto *L = dyn_cast<DILabel>(N))
...
else if (const auto *IE = dyn_cast<DIImportedEntity>(N))
...
```
Currently sinking assumes in instcombine drops assumes if they would
prevent sinking. Removing dereferenceable assumptions earlier on can
inhibit vectorization of early-exit loops in practice.
Special-case deferenceable assumptions so that they block sinking. This
can be combined with a separate change to drop dereferencebale
assumptions after vectorization: https://clang.godbolt.org/z/jGqcx3sbs
PR: https://github.com/llvm/llvm-project/pull/166945
In this patch, we try to detect the lerp pattern: a * (b - c) + c * d
where a >= 0, b >= 0, c >= 0, d >= 0, and b >= c.
In that particular case, we can use the following chain of reasoning:
a * (b - c) + c * d <= a' * (b - c) + a' * c = a' * b where a' = max(a,
d)
Since that is true for arbitrary a, b, c and d within our constraints,
we can
conclude that:
max(a * (b - c) + c * d) <= max(max(a), max(d)) * max(b) = U
Considering that any result of the lerp would be less or equal to U, it
would
have at least the number of leading 0s as in U.
While being quite a specific situation, it is fairly common in computer
graphics in the shape of alpha blending.
In conjunction with #165877, increases vectorization factor for lerp
loops.
This patch enables `FoldOpIntoSelect` and `foldOpIntoPhi` for the cases
when Op's second parameter is a non-constant.
It doesn't seem to bring significant improvements, but the compile
time impact is neglegable.
For the simplification
```
(C && A) || (!C && B) --> sel C, A, B
```
(and related), if `C` (or (`!C`)) is the condition in the select
instruction representing the logical and, we can preserve that logical
and's branch weights when emitting the new instruction. Otherwise, the
profile data is unknown.
If `C` is the condition of both logical ands, then we just take the
branch weights of the first logical and (though in practice they should
be equal.)
Furthermore, `select-safe-transforms.ii` now passes under the profcheck
configuration, so we remove it from the failing tests.
Tracking issue: #147390
The IR pattern is compiled from OpenCL code:
__builtin_astype(x > (uchar2)(0) ? x : -x, uchar2);
where smax is created by foldSelectInstWithICmp + canonicalizeSPF.
smax could also come from direct elementwise max call:
int c = b > (int)(0) ? (int)(0) : -b;
int d = __builtin_elementwise_max(b, (int)(0));
*a = c | d;
https://alive2.llvm.org/ce/z/2-brvrhttps://alive2.llvm.org/ce/z/Dowjzkhttps://alive2.llvm.org/ce/z/kathwZ
---------
Co-authored-by: Yingwei Zheng <dtcxzyw@qq.com>
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
Baseline test for https://github.com/llvm/llvm-project/issues/165793
exposing the change from a vector fp-to-int conversion into a scalar one
inside the loop.
Additionally, removed
`llvm/test/Transforms/InstCombine/vec_extract_var_elt-inseltpoison.ll`,
which was a duplicate of
`llvm/test/Transforms/InstCombine/vec_extract_var_elt.ll` differing only
by the use of poison in the insert element.
The poison variant has been merged into
`llvm/test/Transforms/InstCombine/vec_extract_var_elt.ll`.
Also improved test naming and updated test checks.
This fold can be directly reused for ptrtoaddr. One caveat is
that for an inttoptr base, it currently won't work for pointers
with non-address bits. It's possible to support this case.
For now not trying to share the code with ptrtoint, as there's very
little code.
Also fix IRBuilder::CreatePtrToAddr to actually create a PtrToAddr
instruction...
Skip the transform that replaces:
%a = trunc i64 %x to i16
%b = sext i16 %a to i32
with
%a = trunc i64 %x to i32
%b = shl i32 %a, 16
%c = ashr exact i32 %b, 16
when (pre-trunc) source type is wider than the final destination type.
Modern architectures typically have efficient sign-extend instruction.
It is preferable to preserve the sext for this case.
Resolves#116019
Also fold sext(trunc nsw) to trunc nsw when narrowing source.
This extends the `ptradd x, ptrtoint(y) - ptrtoint(x)` to `y`
InstCombine fold to support ptrtoaddr. In the case where x and y have
the same underlying object, this is handled by InstSimplify already. If
the underlying object may differ, the replacement can only be performed
if provenance does not matter.
For pointers with non-address bits we need to be careful here, because
the pattern will return a pointer with the non-address bits of x and the
address bits of y. As such, uses in ptrtoaddr are safe to replace, but
uses in ptrtoint are not. Whether uses in icmp are safe to replace
depends on the outcome of the pending discussion on icmp semantics (I'll
adjust this in https://github.com/llvm/llvm-project/pull/163936 if/when
that lands).
This reverts commit f1c1063.
PR #163453 was merged and reverted since it exposed a crash.
After investigation the crash was unrelated and is then fixed in #164628.
This is an attempt to reland #163453.
This patch improves constant folding through `llvm.vector.insert`. It
does not change anything for fixed-length vectors (which can already be
folded to ConstantVectors for these cases), but folds scalable vectors
that otherwise would not be folded.
These folds preserve the destination vector (which could be undef or
poison), giving targets more freedom in lowering the operations.
When a predicate is of the form "%a = sve.from.vsbool(%b)" we know all
bits in %a come from %b and thus if %b is all true then %a must also be
all true.
This adds support for folding `ptrtoaddr(p2) - ptrtoaddr(p)` pointer
subtractions. We can treat ptrtoaddr the same as ptrtoint as the
transform is truncation safe anyway (and in fact supports explicit
truncation as well).
The only interesting case is the subtraction of zext of ptrtoaddr. For
this transform it's important that the address bits are not truncated --
and if any pointer bits are truncated, that the truncation is consistent
for both operands. I've explicitly spelled out the three different cases
for this, which also fixes a miscompile in the existing ptrtoint fold.
Previously, cross-lane operations were disallowed here, but they are
only problematic if the `select` condition is a vector, as the input of
the operation is not simply one of the arms of the phi/select.
Fold select instructions with true and false values that act as the same
phi, which cleans up the IR and open up opportunities for other passes
such as loop vectorization.
If a select instruction is replaced with one whose conditional is the
negation of the original, then the replacement's branch weights are the
reverse of the original's.
Tracking issue: #147390
The ptrtoaddr result type is required to match the pointer address
width. This means that, unlikely with ptrtoint, it's not legal to
merge a ptrtoint and trunc. This previously resulted in an IR
verifier failure.
While my objective is to make the shrinkfp path safe for ConstantFP
based splats I discovered the following issues also affect
ConstantVector based splats:
1. PreferBFloat is not set for bfloat vectors.
2. getMinimumFPType() returns a scalar type for vector constants where
getSplatValue() is successful.
All the existing tests test code either in ConstantFolding or
InstSimplify, so move them to use -passes=instsimplify instead of
-passes=instcombine. This makes sure we keep InstSimplify coverage
even if there are subsuming InstCombine folds.
This requires writing some of the constant folding tests in a
different way, as InstSimplify does not try to re-fold already
existing constant expressions.
The `masked.load`, `masked.store`, `masked.gather` and `masked.scatter`
intrinsics currently accept a separate alignment immarg. Replace this
with an `align` attribute on the pointer / vector of pointers argument.
This is the standard representation for alignment information on
intrinsics, and is already used by all other memory intrinsics. This
means the signatures now match llvm.expandload, llvm.vp.load, etc.
(Things like llvm.memcpy used to have a separate alignment argument as
well, but were already migrated a long time ago.)
It's worth noting that the masked.gather and masked.scatter intrinsics
previously accepted a zero alignment to indicate the ABI type alignment
of the element type. This special case is gone now: If the align
attribute is omitted, the implied alignment is 1, as usual. If ABI
alignment is desired, it needs to be explicitly emitted (which the
IRBuilder API already requires anyway).
InstCombine currently fails to call into InstSimplify for cast
instructions. I noticed this because the transform from
https://github.com/llvm/llvm-project/pull/98649 can be triggered via
`-passes=instsimplify` but not `-passes=instcombine`, which is not
supposed to happen.
In the case where we have a conditional that is implied by a previous
conditional (like x < 10 => x < 20 in a select), we can simply propagate
the profile information along the select.
For ptrtoint(inttoptr) and ptrtoaddr(inttoptr), handle the case where
the source and destination size do not match and convert to either zext
or trunc. We can't do this if the middle size is smaller than both
src/dest, because we'd have to perform an additional masking operation
in that case.
Most of these cases are handled by dint of ptrtoint/inttoptr size
canonicalization (so I added some unit tests instead). However, the
ptrtoaddr(inttoptr) case where the pointer size and address size differ
is relevant, as in that case the mismatch in integer sizes is canonical.
Converting a vector float op into a vector int op may be non-profitable,
especially for targets where the float op for a given type is legal, but
the integer op is not.
We could of course also try to address this via a reverse transform in
the backend, but I don't think it's worth the bother, given that vectors
were never the intended use case for this transform in the first place.
Fixes https://github.com/llvm/llvm-project/issues/162749.
Consider the following transform:
```
C = binop float A, nnan OOp
D = select ninf, i1 cond, float C, float A
->
E = select ninf, i1 cond, float OOp, float Identity
F = binop float A, E
```
We cannot propagate ninf from the original select, because OOp may be
inf, and the flag only guarantees that FalseVal (op OOp) is never
infinity.
Examples: -inf + +inf = NaN, -inf - -inf = NaN, 0 * inf = NaN
Specifically, if the original select has both ninf and nnan, we can
safely propagate the flag.
Alive2:
+ fadd: https://alive2.llvm.org/ce/z/TWfktv
+ fsub: https://alive2.llvm.org/ce/z/RAsjJb
+ fmul: https://alive2.llvm.org/ce/z/8eg4ND
Closes https://github.com/llvm/llvm-project/issues/161634.
Add a new m_PtrToIntOrAddr() matcher which matches both ptrtoint and
ptrtoaddr. Pointer arithmetic only works on the address bits, so
supporting ptrtoaddr is always fine here.
And document that we're intentionally not folding this case.
For inttoptr of ptrtoint this is controlled by the
-disable-i2p-p2i-opt flag, but I don't think there is a need to
replicate this known unsound fold for ptrtoaddr, given their
different usage characteristics.
We can support the same folds for as for ptrtoint. For the cast pair
fold we just need to use the address type instead. The gep based folds
were already operating on the address (aka index) width anyway.
