8347 Commits

Author SHA1 Message Date
Yingwei Zheng
345d7b1618
[InstCombine] Fold minmax intrinsic using KnownBits information (#76242)
This patch tries to fold minmax intrinsic by using
`computeConstantRangeIncludingKnownBits`.
Fixes regression in
[_karatsuba_rec:cpython/Modules/_decimal/libmpdec/mpdecimal.c](c31943af16/Modules/_decimal/libmpdec/mpdecimal.c (L5460-L5462)),
which was introduced by #71396.
See also
https://github.com/dtcxzyw/llvm-opt-benchmark/issues/16#issuecomment-1865875756.

Alive2 for splat vectors with undef: https://alive2.llvm.org/ce/z/J8hKWd
2023-12-23 04:41:32 +08:00
Mikhail Gudim
411cba215a
Revert "[InstCombine] Extend foldICmpBinOp to add-like or. (#71… (#76167)
…396)"

This reverts commit 8773c9be3d9868288f1f46957945d50ff58e4e91.
2023-12-21 11:41:09 -05:00
Nikita Popov
b8df88b41c [InstCombine] Support zext nneg in gep of sext add fold
Add m_NNegZext() and m_SExtLike() matchers to make doing these kinds
of changes simpler in the future.
2023-12-21 16:38:09 +01:00
Nikita Popov
4d7112435e [InstCombine] Add zext nneg test variant for gep of sext add fold (NFC) 2023-12-21 16:38:09 +01:00
Chia
8674a023bc
[InstCombine] fold (Binop phi(a, b) phi(b, a)) -> (Binop a, b) while Binop is commutative. (#75765)
Alive2 proof: https://alive2.llvm.org/ce/z/2P8gq-
This patch closes #73905
2023-12-21 22:47:21 +08:00
Z572
e6d2bb0ed8
[InstCombine] Simplifiy (-x * y * -x) into (x * y * x) (#72953)
fix https://github.com/llvm/llvm-project/issues/72259
proof: https://alive2.llvm.org/ce/z/HsrmTC
2023-12-21 19:13:09 +08:00
Mikhail Gudim
8773c9be3d
[InstCombine] Extend foldICmpBinOp to add-like or. (#71396)
InstCombine canonicalizes `add` to `or` when possible, but this makes
some optimizations applicable to `add` to be missed because they don't
realize that the `or` is equivalent to `add`.

In this patch we generalize `foldICmpBinOp` to handle such cases.
2023-12-20 17:28:57 -05:00
bipmis
64987c648f
[ValueTracking] isNonZero sub of ptr2int's with recursive GEP (#68680)
When the sub arguments are ptr2int it is not possible to determine
computeKnownBits() of its arguments.
For scalar case generally sub of 2 ptr2int are converted to sub of
indexes.
However a loop with recursive GEP/PHI where the arguments to sub is of
type ptr2int, if it is possible to determine that a sub of this GEP and
another pointer with the same base is KnownNonZero we can return this.
This helps subsequent passes to optimize the loop further.
2023-12-20 14:11:58 +00:00
Nikita Popov
9d60e95bcd
[AMDGPU] Use poison instead of undef for non-demanded elements (#75914)
Return poison instead of undef for non-demanded lanes in the AMDGPU
demanded element simplification hook.

Also bail out of dmask is 0, as this case has special semantics:

> If DMASK==0, the TA overrides DMASK=1 and puts zeros in VGPR followed by
> LWE status if exists. TFE status is not generated since the fetch is dropped.
2023-12-20 11:01:59 +01:00
Nikita Popov
92fc4b482f [InstCombine] Preserve poison in bitcast of insertelement fold
If the base was poison, retain the poison value.
2023-12-19 13:06:04 +01:00
Nikita Popov
f412b78ffc [InstCombine] Return poison if all lanes are poison 2023-12-19 12:43:23 +01:00
Nikita Popov
e879f44b28 [InstCombine] Regenerate test checks (NFC) 2023-12-19 12:29:22 +01:00
Nikita Popov
9d4557920f [InstCombine] Don't treat undef as poison in demanded element simplification
We can only set PoisonElts if the element is poison, not if it is
undef.
2023-12-19 12:26:48 +01:00
Nikita Popov
e400c59beb Revert "[InstCombine] Favour m_Poison in SimplifyDemandedVectorElts"
This reverts commit 318d5bff0b65aa7d52fc7004d49587416f0fb564.

Has incomplete test updates.
2023-12-18 18:08:57 +01:00
Antonio Frighetto
318d5bff0b [InstCombine] Favour m_Poison in SimplifyDemandedVectorElts
A miscompilation issue has been addressed with refined checking.
2023-12-18 17:28:39 +01:00
Nikita Popov
cd54c47424 [InstCombine] Match poison instead of undef in foldVectorBinop()
Some negative tests turn into positive tests, as the differences
between undef and poison propagation allow additional transforms.
2023-12-18 17:01:59 +01:00
Nikita Popov
9d25b28b9e [InstCombine] Explicitly canonicalize splat shuffles to use poison RHS
This is usually handled by demanded elements simplification. However,
as that is not supported for scalable vectors, also handle it
explicitly here.
2023-12-18 16:30:40 +01:00
Nikita Popov
a5f3415533 [InstCombine] Replace non-demanded undef vector with poison
If an operand (esp to shufflevector or insertelement) is not
demanded, canonicalize it from undef to poison.
2023-12-18 16:12:37 +01:00
Nikita Popov
d0605e21af [InstCombine] Canonicalize splat shuffles to use poison operand
If the splat shuffle is represented using an undef RHS, replace it
with poison.
2023-12-18 15:57:49 +01:00
Nikita Popov
e93d324adb [InstCombine] Preserve poison in evaluateInDifferentElementOrder()
Don't unnecessarily replace poison with undef.
2023-12-18 15:36:22 +01:00
Nikita Popov
6c9813aa02 [InstCombine] Check for poison instead of undef in shuffle combine
Otherwise we may replace undef with poison.

Note that a lot of tests regressing here already have variants
that use poison instead of undef (often in a separate
inseltpoison file), which is why I'm not adjusting them to the
new pattern.
2023-12-18 15:19:16 +01:00
Nikita Popov
fda3a134bc [InstCombine] Regenerate test checks (NFC) 2023-12-18 15:00:59 +01:00
Antonio Frighetto
151ddf07a6 [InstCombine] Stop propagating undef when element is demanded
Do not poison `undef` demanded elements in `SimplifyDemandedVectorElts`.
A miscompilation issue has been addressed with refined checking.

Proofs: https://alive2.llvm.org/ce/z/WA5oD5.
2023-12-17 21:41:03 +01:00
Antonio Frighetto
fc520f8b29 [InstCombine] Precommit tests for PR75745 (NFC) 2023-12-17 21:41:02 +01:00
Yingwei Zheng
c59ea32f82
[InstCombine] Canonicalize icmp pred (X +/- C1), C2 into icmp pred X, C2 -/+ C1 with nowrap flag implied by with.overflow intrinsic (#75511)
This patch tries to canonicalize the pattern `Overflow | icmp pred Res,
C2` into `Overflow | icmp pred X, C2 +/- C1`, where `Overflow` and `Res`
are return values of `xxx.with.overflow X, C1`.
Alive2: https://alive2.llvm.org/ce/z/PhR_3S

Fixes #75360.
2023-12-16 17:58:57 +08:00
Craig Topper
4faeb7dbe9 [InstCombine] Add test for missed opportunity to fold 'or' into 'mul' operand. NFC
We are able to fold

or (mul X, Y), X --> mul X, (add Y, 1) (when the multiply has no common bits with X)

but we miss it if the mul operands are commuted.
2023-12-15 21:30:29 -08:00
Jessica Del
32f9983c06
[AMDGPU] - Add address space for strided buffers (#74471)
This is an experimental address space for strided buffers. These buffers
can have structs as elements and
a stride > 1.
These pointers allow the indexed access in units of stride, i.e., they
point at `buffer[index * stride]`.
Thus, we can use the `idxen` modifier for buffer loads.

We assign address space 9 to 192-bit buffer pointers which contain a
128-bit descriptor, a 32-bit offset and a 32-bit index. Essentially,
they are fat buffer pointers with an additional 32-bit index.
2023-12-15 15:49:25 +01:00
Mirko Brkušanin
26b14aedb7
[AMDGPU] CodeGen for GFX12 VIMAGE and VSAMPLE instructions (#75488) 2023-12-15 12:40:23 +01:00
bipmis
6df6320374
[ValueTracking] isNonEqual Pointers with with a recursive GEP (#70459)
Handles canonical icmp eq(ptr1, ptr2) -> where ptr1/ptr2 is a recursive
GEP.
Can helps scenarios where InstCombineCompares folds icmp eq(sub(ptr2int,
ptr2int), 0) -> icmp eq(ptr1, ptr2)
and
icmp eq(phi(sub(ptr2int, ptr2int), ...)) -> phi i1 (icmp eq(sub(ptr2int,
ptr2int), 0), ....)
2023-12-15 10:02:57 +00:00
Yingwei Zheng
af2d740d2d
[InstCombine] Treat lshr nneg as ashr in getBinOpsForFactorization (#75521)
This patch reinterprets `lshr nneg C, X` as `ashr nneg C, X` to allow
more factorization opportunities.
Fixes #70582.
2023-12-15 16:32:18 +08:00
Yingwei Zheng
9cf3e31172
[InstCombine] Explicitly fold ~(~X >>u Y) into X >>s Y (#75473)
Fixes #75369.

This patch explicitly folds `~(~X >>u Y)` into `X >>s Y` to fix assertion failure in #75369.
2023-12-14 23:06:38 +08:00
Nikita Popov
9100228e45 [InstCombine] Fix insertion point 2023-12-14 15:27:17 +01:00
Fujun Han
7be5dabbc2
[InstCombine] Change (add x, c) to (xor x, c) (#75129)
Change (add x, c) to (xor x, c) iff c is constant and c equals the top bit of the demanded bits.
Alive2: https://alive2.llvm.org/ce/z/DKmkwF

---------

Signed-off-by: Peter Han <fujun.han@iluvatar.com>
Co-authored-by: Peter Han <fujun.han@iluvatar.com>
2023-12-14 21:19:15 +08:00
Z572
1c494198c3
[InstCombine] simplify (X * C0) / (X * C1) into C0 / C1. (#73204)
fix #72114
proof: https://alive2.llvm.org/ce/z/xqprFm
2023-12-13 17:17:06 +08:00
Sizov Nikita
88cc35b27e
[InstCombine] Fold binop (select cond, a, b), (select cond, b, a) to binop a, b (#74953)
```
CommutativeBinOp(select(V, A, B), select(V, B, A) --> CommutativeBinOp(A, B)
CommutativeIntrinsicCall(select(V, A, B), select(V, B, A), ...) --> CommutativeIntrinsicCall(A, B, ...)
```

https://alive2.llvm.org/ce/z/8CDUZ4

Closes #73904
2023-12-13 14:09:27 +08:00
Orlando Cazalet-Hyams
cb8690ff6f
[RemoveDIs] Handle DPValues in LowerDbgDeclare (#73504)
The tests will become "live" once #74090 lands (see for more info).
2023-12-12 14:28:05 +00:00
Nikita Popov
97efd8aa43 [InstCombine] Preserve inalloca tag when transforming alloca
This is not meaningful in any practical sense, and just makes sure
we don't cause verifier failures.
2023-12-11 14:27:00 +01:00
Nikita Popov
4a2a6397f1 [InstCombine] Relax one-use check for icmp of gep fold
Instead of checking whether the GEP as a whole is constant, only
check whether it has constant incides. This matches what we do in
other places in this code.

This has little practical impact, because it is mostly already
handled through other cases anyway. We see a difference for
non-inbounds equality comparisons.
2023-12-08 15:45:58 +01:00
Nikita Popov
cf029a22bd [InstCombine] Add some more multi-use icmp gep folding tests (NFC) 2023-12-08 15:45:58 +01:00
Nikita Popov
cf47af493b
[InstCombine] Generalize folds for inversion of icmp operands (#74317)
We have a bunch of folds that basically perform X pred Y to ~Y pred ~X
for various special cases where this saves an instruction.

Generalize these folds to use isFreeToInvert(). We have to make sure
that we consume an instruction in either of the inversions, otherwise
we're just going to swap the icmp back and forth.

Fixes https://github.com/llvm/llvm-project/issues/74302.
2023-12-08 11:25:41 +01:00
Sizov Nikita
827f8a7ef6
Add opt with ctlz and shifts of power of 2 constants (#74175)
This patch does the following simplifications:
```
cttz(shl(C, X), 1) -> add(cttz(C, 1), X)
cttz(lshr exact(C, X), 1) -> sub(cttz(C, 1), X)
ctlz(lshr(C, X), 1) --> add(ctlz(C, 1), X)
ctlz(shl nuw (C, X), 1) --> sub(ctlz(C, 1), X)
```
Alive2: https://alive2.llvm.org/ce/z/9KHlKc
Closes #41333
2023-12-08 15:06:23 +08:00
Craig Topper
09a05f5dcb [InstCombine] Drop poison generating flags on Or in simplifyAssocCastAssoc.
Fixes #74739.
2023-12-07 13:35:28 -08:00
Craig Topper
c54cbf82b8 [InstCombine] Add test case for #74739. NFC 2023-12-07 13:35:28 -08:00
Nikita Popov
9026dff382 [InstCombine] Add extra tests for known bits from dominating conds (NFC)
This adds test coverage for conditions with and/or.
2023-12-07 16:23:20 +01:00
Nikita Popov
6e8b17d821 [InstCombine] Support or disjoint in displaced shift fold
When I originally added this fold, it did not actually fix my
motivation case, where the add was represented as an or. Now that
we have the disjoint flag this can finally be cleanly supported.
2023-12-07 15:00:40 +01:00
Nikita Popov
04697aa18a [InstCombine] Add test for displaced shift fold with or disjoint (NFC) 2023-12-07 15:00:40 +01:00
Nikita Popov
d77067d08a
[ValueTracking] Add dominating condition support in computeKnownBits() (#73662)
This adds support for using dominating conditions in computeKnownBits()
when called from InstCombine. The implementation uses a
DomConditionCache, which stores which branches may provide information
that is relevant for a given value.

DomConditionCache is similar to AssumptionCache, but does not try to do
any kind of automatic tracking. Relevant branches have to be explicitly
registered and invalidated values explicitly removed. The necessary
tracking is done inside InstCombine.

The reason why this doesn't just do exactly the same thing as
AssumptionCache is that a lot more transforms touch branches and branch
conditions than assumptions. AssumptionCache is an immutable analysis
and mostly gets away with this because only a handful of places have to
register additional assumptions (mostly as a result of cloning). This is
very much not the case for branches.

This change regresses compile-time by about ~0.2%. It also improves
stage2-O0-g builds by about ~0.2%, which indicates that this change results
in additional optimizations inside clang itself.

Fixes https://github.com/llvm/llvm-project/issues/74242.
2023-12-06 14:17:18 +01:00
Craig Topper
5c3496ff33
[InstCombine] Check isGuaranteedNotToBeUndef in haveNoCommonBitsSetSpecialCases. (#74390)
It's not safe for InstCombine to add disjoint metadata when converting
Add to Or otherwise.

I've added noundef attribute to preserve existing test behavior.
2023-12-05 10:33:44 -08:00
Nikita Popov
410bf5e142 [InstCombine] Use disjoint flag in mul of or fold
Slightly more powerful if the information used to infer disjoint
was lost.
2023-12-05 15:24:50 +01:00
Nikita Popov
e4710872e9 [InstCombine] Use disjoint flag in add of or fold
Use disjoint instead of haveNoCommonBitsSet(), which is slightly
stronger in case the information used to infer disjoint has been
lost.

Introduce the m_DisjointOr() matcher to make handling cases like
this cleaner.
2023-12-05 15:06:40 +01:00