5907 Commits

Author SHA1 Message Date
Yingwei Zheng
345d7b1618
[InstCombine] Fold minmax intrinsic using KnownBits information (#76242)
This patch tries to fold minmax intrinsic by using
`computeConstantRangeIncludingKnownBits`.
Fixes regression in
[_karatsuba_rec:cpython/Modules/_decimal/libmpdec/mpdecimal.c](c31943af16/Modules/_decimal/libmpdec/mpdecimal.c (L5460-L5462)),
which was introduced by #71396.
See also
https://github.com/dtcxzyw/llvm-opt-benchmark/issues/16#issuecomment-1865875756.

Alive2 for splat vectors with undef: https://alive2.llvm.org/ce/z/J8hKWd
2023-12-23 04:41:32 +08:00
Mikhail Gudim
411cba215a
Revert "[InstCombine] Extend foldICmpBinOp to add-like or. (#71… (#76167)
…396)"

This reverts commit 8773c9be3d9868288f1f46957945d50ff58e4e91.
2023-12-21 11:41:09 -05:00
Nikita Popov
b8df88b41c [InstCombine] Support zext nneg in gep of sext add fold
Add m_NNegZext() and m_SExtLike() matchers to make doing these kinds
of changes simpler in the future.
2023-12-21 16:38:09 +01:00
Chia
8674a023bc
[InstCombine] fold (Binop phi(a, b) phi(b, a)) -> (Binop a, b) while Binop is commutative. (#75765)
Alive2 proof: https://alive2.llvm.org/ce/z/2P8gq-
This patch closes #73905
2023-12-21 22:47:21 +08:00
Z572
e6d2bb0ed8
[InstCombine] Simplifiy (-x * y * -x) into (x * y * x) (#72953)
fix https://github.com/llvm/llvm-project/issues/72259
proof: https://alive2.llvm.org/ce/z/HsrmTC
2023-12-21 19:13:09 +08:00
Mikhail Gudim
8773c9be3d
[InstCombine] Extend foldICmpBinOp to add-like or. (#71396)
InstCombine canonicalizes `add` to `or` when possible, but this makes
some optimizations applicable to `add` to be missed because they don't
realize that the `or` is equivalent to `add`.

In this patch we generalize `foldICmpBinOp` to handle such cases.
2023-12-20 17:28:57 -05:00
Nikita Popov
92fc4b482f [InstCombine] Preserve poison in bitcast of insertelement fold
If the base was poison, retain the poison value.
2023-12-19 13:06:04 +01:00
Nikita Popov
67fd4e3408 [InstCombine] Check for poison instead of undef in shuffle transform
This one doesn't seem to make a practical difference because we'd
canonicalize undef -> poison in the relevant cases anywy.
2023-12-19 12:56:52 +01:00
Nikita Popov
f412b78ffc [InstCombine] Return poison if all lanes are poison 2023-12-19 12:43:23 +01:00
Nikita Popov
9d4557920f [InstCombine] Don't treat undef as poison in demanded element simplification
We can only set PoisonElts if the element is poison, not if it is
undef.
2023-12-19 12:26:48 +01:00
Nikita Popov
e400c59beb Revert "[InstCombine] Favour m_Poison in SimplifyDemandedVectorElts"
This reverts commit 318d5bff0b65aa7d52fc7004d49587416f0fb564.

Has incomplete test updates.
2023-12-18 18:08:57 +01:00
Antonio Frighetto
318d5bff0b [InstCombine] Favour m_Poison in SimplifyDemandedVectorElts
A miscompilation issue has been addressed with refined checking.
2023-12-18 17:28:39 +01:00
Nikita Popov
cd54c47424 [InstCombine] Match poison instead of undef in foldVectorBinop()
Some negative tests turn into positive tests, as the differences
between undef and poison propagation allow additional transforms.
2023-12-18 17:01:59 +01:00
Nikita Popov
ddd11537e2 [InstCombine] Match poison instead of undef in binop of same-mask shuffle fold 2023-12-18 16:41:38 +01:00
Nikita Popov
9d25b28b9e [InstCombine] Explicitly canonicalize splat shuffles to use poison RHS
This is usually handled by demanded elements simplification. However,
as that is not supported for scalable vectors, also handle it
explicitly here.
2023-12-18 16:30:40 +01:00
Nikita Popov
a5f3415533 [InstCombine] Replace non-demanded undef vector with poison
If an operand (esp to shufflevector or insertelement) is not
demanded, canonicalize it from undef to poison.
2023-12-18 16:12:37 +01:00
Nikita Popov
d0605e21af [InstCombine] Canonicalize splat shuffles to use poison operand
If the splat shuffle is represented using an undef RHS, replace it
with poison.
2023-12-18 15:57:49 +01:00
Nikita Popov
e93d324adb [InstCombine] Preserve poison in evaluateInDifferentElementOrder()
Don't unnecessarily replace poison with undef.
2023-12-18 15:36:22 +01:00
Nikita Popov
6c9813aa02 [InstCombine] Check for poison instead of undef in shuffle combine
Otherwise we may replace undef with poison.

Note that a lot of tests regressing here already have variants
that use poison instead of undef (often in a separate
inseltpoison file), which is why I'm not adjusting them to the
new pattern.
2023-12-18 15:19:16 +01:00
Paul Walker
dea16ebd26
[LLVM][IR] Replace ConstantInt's specialisation of getType() with getIntegerType(). (#75217)
The specialisation will not be valid when ConstantInt gains native
support for vector types.

This is largely a mechanical change but with extra attention paid to constant
folding, InstCombineVectorOps.cpp, LoopFlatten.cpp and Verifier.cpp to
remove the need to call `getIntegerType()`.

Co-authored-by: Nikita Popov <github@npopov.com>
2023-12-18 11:58:42 +00:00
Nikita Popov
465ecf872e [InstCombine] Rename UndefElts -> PoisonElts (NFC)
In line with updated shufflevector semantics, this represents the
poison elements rather than undef elements now. This commit is a
pure rename, without any logic changes.
2023-12-18 12:36:19 +01:00
Antonio Frighetto
151ddf07a6 [InstCombine] Stop propagating undef when element is demanded
Do not poison `undef` demanded elements in `SimplifyDemandedVectorElts`.
A miscompilation issue has been addressed with refined checking.

Proofs: https://alive2.llvm.org/ce/z/WA5oD5.
2023-12-17 21:41:03 +01:00
Yingwei Zheng
c59ea32f82
[InstCombine] Canonicalize icmp pred (X +/- C1), C2 into icmp pred X, C2 -/+ C1 with nowrap flag implied by with.overflow intrinsic (#75511)
This patch tries to canonicalize the pattern `Overflow | icmp pred Res,
C2` into `Overflow | icmp pred X, C2 +/- C1`, where `Overflow` and `Res`
are return values of `xxx.with.overflow X, C1`.
Alive2: https://alive2.llvm.org/ce/z/PhR_3S

Fixes #75360.
2023-12-16 17:58:57 +08:00
Yingwei Zheng
af2d740d2d
[InstCombine] Treat lshr nneg as ashr in getBinOpsForFactorization (#75521)
This patch reinterprets `lshr nneg C, X` as `ashr nneg C, X` to allow
more factorization opportunities.
Fixes #70582.
2023-12-15 16:32:18 +08:00
Yingwei Zheng
9cf3e31172
[InstCombine] Explicitly fold ~(~X >>u Y) into X >>s Y (#75473)
Fixes #75369.

This patch explicitly folds `~(~X >>u Y)` into `X >>s Y` to fix assertion failure in #75369.
2023-12-14 23:06:38 +08:00
Nikita Popov
9100228e45 [InstCombine] Fix insertion point 2023-12-14 15:27:17 +01:00
Fujun Han
7be5dabbc2
[InstCombine] Change (add x, c) to (xor x, c) (#75129)
Change (add x, c) to (xor x, c) iff c is constant and c equals the top bit of the demanded bits.
Alive2: https://alive2.llvm.org/ce/z/DKmkwF

---------

Signed-off-by: Peter Han <fujun.han@iluvatar.com>
Co-authored-by: Peter Han <fujun.han@iluvatar.com>
2023-12-14 21:19:15 +08:00
Yingwei Zheng
b7f50e13d8
[InstCombine] Improve foldICmpWithDominatingICmp with DomConditionCache (#75370)
This patch uses affected values from DomConditionCache(introduced by #73662), instead of a cheap/incomplete check `getSinglePredecessor`.
2023-12-14 21:02:10 +08:00
Benjamin Kramer
60aeea21fd [InstCombine] Fix uninitialized variable usage
m_Specific can only be used if the previous check suceeded. Found by
msan.
2023-12-13 16:31:19 +01:00
Z572
1c494198c3
[InstCombine] simplify (X * C0) / (X * C1) into C0 / C1. (#73204)
fix #72114
proof: https://alive2.llvm.org/ce/z/xqprFm
2023-12-13 17:17:06 +08:00
Sizov Nikita
88cc35b27e
[InstCombine] Fold binop (select cond, a, b), (select cond, b, a) to binop a, b (#74953)
```
CommutativeBinOp(select(V, A, B), select(V, B, A) --> CommutativeBinOp(A, B)
CommutativeIntrinsicCall(select(V, A, B), select(V, B, A), ...) --> CommutativeIntrinsicCall(A, B, ...)
```

https://alive2.llvm.org/ce/z/8CDUZ4

Closes #73904
2023-12-13 14:09:27 +08:00
Nikita Popov
97efd8aa43 [InstCombine] Preserve inalloca tag when transforming alloca
This is not meaningful in any practical sense, and just makes sure
we don't cause verifier failures.
2023-12-11 14:27:00 +01:00
Nikita Popov
ae7bffd71c [InstCombine] Don't create unnecessary zero-index GEP (NFCI)
Note needed with opaque pointers.
2023-12-11 13:09:09 +01:00
Kazu Hirata
8b1181133d [Transforms] Remove unused forward declarations (NFC) 2023-12-10 10:07:12 -08:00
Kazu Hirata
a16429365c [Transforms] Remove unnecessary includes (NFC) 2023-12-09 18:23:06 -08:00
Nikita Popov
4a2a6397f1 [InstCombine] Relax one-use check for icmp of gep fold
Instead of checking whether the GEP as a whole is constant, only
check whether it has constant incides. This matches what we do in
other places in this code.

This has little practical impact, because it is mostly already
handled through other cases anyway. We see a difference for
non-inbounds equality comparisons.
2023-12-08 15:45:58 +01:00
Nikita Popov
cf47af493b
[InstCombine] Generalize folds for inversion of icmp operands (#74317)
We have a bunch of folds that basically perform X pred Y to ~Y pred ~X
for various special cases where this saves an instruction.

Generalize these folds to use isFreeToInvert(). We have to make sure
that we consume an instruction in either of the inversions, otherwise
we're just going to swap the icmp back and forth.

Fixes https://github.com/llvm/llvm-project/issues/74302.
2023-12-08 11:25:41 +01:00
Sizov Nikita
827f8a7ef6
Add opt with ctlz and shifts of power of 2 constants (#74175)
This patch does the following simplifications:
```
cttz(shl(C, X), 1) -> add(cttz(C, 1), X)
cttz(lshr exact(C, X), 1) -> sub(cttz(C, 1), X)
ctlz(lshr(C, X), 1) --> add(ctlz(C, 1), X)
ctlz(shl nuw (C, X), 1) --> sub(ctlz(C, 1), X)
```
Alive2: https://alive2.llvm.org/ce/z/9KHlKc
Closes #41333
2023-12-08 15:06:23 +08:00
Craig Topper
09a05f5dcb [InstCombine] Drop poison generating flags on Or in simplifyAssocCastAssoc.
Fixes #74739.
2023-12-07 13:35:28 -08:00
Nikita Popov
6e8b17d821 [InstCombine] Support or disjoint in displaced shift fold
When I originally added this fold, it did not actually fix my
motivation case, where the add was represented as an or. Now that
we have the disjoint flag this can finally be cleanly supported.
2023-12-07 15:00:40 +01:00
Nikita Popov
5295b12cd0 [PatternMatch] Add m_AddLike matcher (NFC)
This matches either a plain "add" or an "or disjoint" that can
be converted into an add. The AddLike terminology is adopted from
the SDAG layer.
2023-12-07 14:45:12 +01:00
Craig Topper
56248caa3b
[InstCombine] Explicitly set disjoint flag when converting xor to or. (#74229) 2023-12-06 09:41:59 -08:00
Nikita Popov
d77067d08a
[ValueTracking] Add dominating condition support in computeKnownBits() (#73662)
This adds support for using dominating conditions in computeKnownBits()
when called from InstCombine. The implementation uses a
DomConditionCache, which stores which branches may provide information
that is relevant for a given value.

DomConditionCache is similar to AssumptionCache, but does not try to do
any kind of automatic tracking. Relevant branches have to be explicitly
registered and invalidated values explicitly removed. The necessary
tracking is done inside InstCombine.

The reason why this doesn't just do exactly the same thing as
AssumptionCache is that a lot more transforms touch branches and branch
conditions than assumptions. AssumptionCache is an immutable analysis
and mostly gets away with this because only a handful of places have to
register additional assumptions (mostly as a result of cloning). This is
very much not the case for branches.

This change regresses compile-time by about ~0.2%. It also improves
stage2-O0-g builds by about ~0.2%, which indicates that this change results
in additional optimizations inside clang itself.

Fixes https://github.com/llvm/llvm-project/issues/74242.
2023-12-06 14:17:18 +01:00
Nikita Popov
a1b9736e9b [PatternMatch] Add m_c_DisjointOr (NFC)
Add commutative variant of m_DisjointOr.
2023-12-06 14:05:02 +01:00
Nikita Popov
410bf5e142 [InstCombine] Use disjoint flag in mul of or fold
Slightly more powerful if the information used to infer disjoint
was lost.
2023-12-05 15:24:50 +01:00
Nikita Popov
e4710872e9 [InstCombine] Use disjoint flag in add of or fold
Use disjoint instead of haveNoCommonBitsSet(), which is slightly
stronger in case the information used to infer disjoint has been
lost.

Introduce the m_DisjointOr() matcher to make handling cases like
this cleaner.
2023-12-05 15:06:40 +01:00
Nikita Popov
cd865e36db [InstCombine] Use disjoint flag instead of haveNoCommonBitsSet()
Slightly stronger, if disjoint was inferred earlier with information
that is no longer available.
2023-12-05 14:44:48 +01:00
shaojingzhi
9a99a1a39e
[InstCombine] Add one-use limitation to box multiply fold (#72876)
Check the operands of I are used in no more than one place, which can
not be deleted, cause a mul instruction has far more weight than add and
shl instruction in IR, thus this method cannot achieve the goal of
simplifying instructions, just return null.
2023-12-04 14:14:59 +01:00
Craig Topper
3e7ca05e93
[InstCombine] Use disjoint flag instead of calling haveNoCommonBitsSet. (#74222) 2023-12-03 12:34:49 -08:00
Yingwei Zheng
741975df92
[InstCombine][InstSimplify] Pass SimplifyQuery to computeKnownBits directly. NFC. (#74246)
This patch passes `SimplifyQuery` to `computeKnownBits` directly in
`InstSimplify` and `InstCombine`.
As the `DomConditionCache` in #73662 is only used in `InstCombine`, it
is inconvenient to introduce a new argument `DC` to `computeKnownBits`.
2023-12-04 02:26:39 +08:00