339 Commits

Author SHA1 Message Date
Yingwei Zheng
c9d9c3e349
[InstCombine] Fold icmp pred X + K, Y -> icmp pred2 X, Y if both X and Y is divisible by K (#147130)
This patch generalizes `icmp ule X +nuw 1, Y -> icmp ult X, Y`-like
optimizations to handle the case that the added RHS constant is a common
power-of-2 divisor of both X and Y. We can further generalize this
pattern to handle non-power-of-2 divisors as well.
Alive2: https://alive2.llvm.org/ce/z/QgpeM_

Compile-time improvement (Stage2-O3 -0.09%):
https://llvm-compile-time-tracker.com/compare.php?from=0ba59587fa98849ed5107fee4134e810e84b69a3&to=f80e5fe0bb2e63c05401bde7cd42899ea270909b&stat=instructions:u

The original case is from the comparison of expanded GEP offsets:
https://github.com/dtcxzyw/llvm-opt-benchmark/pull/2530/files#r2183005292
2025-07-05 23:42:53 +08:00
Nikita Popov
20e8de9c8f
[InstCombine] Support nested GEPs in OptimizePointerDifference (#142958)
Currently OptimizePointerDifference() only handles single GEPs with a
common base, not GEP chains. This patch generalizes the support to
nested GEPs with a common base.

Finding the common base is a bit annoying because we want to stop as
soon as possible and not recurse into common GEP prefixes.

This helps avoids regressions from
https://github.com/llvm/llvm-project/pull/137297.
2025-06-10 09:28:15 +02:00
Matt Arsenault
48585caf72
InstCombine: Avoid counting uses of constants (#136566)
Logically it does not matter; getFreelyInvertedImpl doesn't
depend on the value for the m_ImmConstant case.

This use count logic should probably sink into getFreelyInvertedImpl,
every use of this appears to just be a hasOneUse or hasNUse count,
so this could change to just be a use count threshold.
2025-04-23 10:51:55 +02:00
Nikita Popov
462cb3cd6c
[InstCombine] Infer nusw + nneg -> nuw for getelementptr (#111144)
If the gep is nusw (usually via inbounds) and the offset is
non-negative, we can infer nuw.

Proof: https://alive2.llvm.org/ce/z/ihztLy
2024-12-05 14:36:40 +01:00
Yingwei Zheng
c1ad064dd3
[InstCombine] Fold icmp spred (and X, highmask), C1 into icmp spred X, C2 (#118197)
Alive2: https://alive2.llvm.org/ce/z/Ffg64g
Closes https://github.com/llvm/llvm-project/issues/104772.
2024-12-03 16:19:12 +08:00
Paul Walker
38fffa630e
[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548) 2024-11-06 11:53:33 +00:00
Noah Goldstein
294726d738 Reapply "[InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)" (#111236)
The underlying issue with msan was fixed by #113200
2024-10-23 09:12:08 -05:00
Yingwei Zheng
095d49da76
[InstCombine] Set samesign when converting signed predicates into unsigned (#112642)
Alive2: https://alive2.llvm.org/ce/z/6cqdt-
2024-10-17 20:43:48 +08:00
Yingwei Zheng
0936195311
[InstCombine] Drop samesign in InstCombine (#112480)
Closes https://github.com/llvm/llvm-project/issues/112476.
2024-10-16 19:13:52 +08:00
Yingwei Zheng
9b7491e866
[IR] Add support for samesign in Operator::hasPoisonGeneratingFlags (#112358)
Fix https://github.com/llvm/llvm-project/issues/112356.
2024-10-15 23:07:16 +08:00
Vitaly Buka
574266ce33
Revert "[InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)" (#111236)
Reverts #110880 because of exposed issue is Msan instrumentation
#111212.

This reverts commit a64643688526114b50c25b3eda8a57855bd2be87.
2024-10-04 23:20:40 -07:00
Noah Goldstein
a646436885 [InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)
Folds to `(icmp slt/sge X, (INT_MIN + P2))`

Proofs: https://alive2.llvm.org/ce/z/vpNFY5

Closes #110880
2024-10-03 13:05:08 -05:00
Yingwei Zheng
ffcff4af59
[ValueTracking] Infer is-power-of-2 from assumptions. (#107745)
This patch tries to infer is-power-of-2 from assumptions. I don't see
that this kind of assumption exists in my dataset.
Related issue: https://github.com/rust-lang/rust/issues/129795

Close https://github.com/llvm/llvm-project/issues/58996.
2024-09-10 10:38:21 +08:00
Nikita Popov
a105877646
[InstCombine] Remove some of the complexity-based canonicalization (#91185)
The idea behind this canonicalization is that it allows us to handle less
patterns, because we know that some will be canonicalized away. This is
indeed very useful to e.g. know that constants are always on the right.

However, this is only useful if the canonicalization is actually
reliable. This is the case for constants, but not for arguments: Moving
these to the right makes it look like the "more complex" expression is
guaranteed to be on the left, but this is not actually the case in
practice. It fails as soon as you replace the argument with another
instruction.

The end result is that it looks like things correctly work in tests,
while they actually don't. We use the "thwart complexity-based
canonicalization" trick to handle this in tests, but it's often a
challenge for new contributors to get this right, and based on the
regressions this PR originally exposed, we clearly don't get this right
in many cases.

For this reason, I think that it's better to remove this complexity
canonicalization. It will make it much easier to write tests for
commuted cases and make sure that they are handled.
2024-08-21 12:02:54 +02:00
Yingwei Zheng
e4b0655b29
[InstCombine] Fix missing argument typo in InstCombinerImpl::foldICmpShlConstant (#94899)
Closes #94897.
2024-06-10 03:17:01 +08:00
Yingwei Zheng
5c7c1f6aba
[InstCombine] Try the flipped strictness of predicate in foldICmpShlConstant (#92773)
This patch extends the transform `(icmp pred iM (shl iM %v, N), C) ->
(icmp pred i(M-N) (trunc %v iM to i(M-N)), (trunc (C>>N))` to handle
icmps with the flipped strictness of predicate.

See the following case:
```
icmp ult i64 (shl X, 32), 8589934593 ->
icmp ule i64 (shl X, 32), 8589934592 ->
icmp ule i32 (trunc X, i32), 2 ->
icmp ult i32 (trunc X, i32), 3
```

Fixes the regression introduced by
https://github.com/llvm/llvm-project/pull/86111#issuecomment-2098203152.

Alive2 proofs: https://alive2.llvm.org/ce/z/-sp5n3

`nuw` cannot be propagated as we always use `ashr` here. I don't see the
value of fixing this (see the test `test_icmp_shl_nuw`).
2024-05-28 12:47:27 +08:00
Yingwei Zheng
d085b42cbb
[InstSimplify] Do not simplify freeze in simplifyWithOpReplaced (#91215)
See the LangRef:
> All uses of a value returned by the same ‘freeze’ instruction are
guaranteed to always observe the same value, while different ‘freeze’
instructions may yield different values.

It is incorrect to replace freezes with the simplified value.

Proof:
https://alive2.llvm.org/ce/z/3Dn9Cd
https://alive2.llvm.org/ce/z/Qyh5h6

Fixes https://github.com/llvm/llvm-project/issues/91178
2024-05-08 10:04:09 +08:00
Nikita Popov
d9a5aa8e2d
[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)
Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elements inside vector
splats, not undef elements.

Historically, we used to represent non-demanded elements in vectors
using undef. Nowadays, we use poison instead. As such, I believe that
support for undef in vector splats is no longer useful.

At the same time, while poison splat elements are pretty much always
safe to ignore, this is not generally the case for undef elements. We
have existing miscompiles in our tests due to this (see the
masked-merge-*.ll tests changed here) and it's easy to miss such cases
in the future, now that we write tests using poison instead of undef
elements.

I think overall, keeping support for undef elements no longer makes
sense, and we should drop it. Once this is done consistently, I think we
may also consider allowing poison in m_APInt by default, as doing that
change is much less risky than doing the same with undef.

This change involves a substantial amount of test changes. For most
tests, I've just replaced undef with poison, as I don't think there is
value in retaining both. For some tests (where the distinction between
undef and poison is important), I've duplicated tests.
2024-04-17 18:22:05 +09:00
Nikita Popov
de8f782355 Revert "Simplify (a % b) lt/ge (b-1) into (a % b) eq/ne (b-1) (#72504)"
This reverts commit 01f4d40aad58c5c34a8ae30edbf4e0ebbf235838.

Causes test failures.
2024-01-16 11:39:42 +01:00
elhewaty
01f4d40aad
Simplify (a % b) lt/ge (b-1) into (a % b) eq/ne (b-1) (#72504)
Alive2: https://alive2.llvm.org/ce/z/i7zYtE
Fixes: https://github.com/llvm/llvm-project/issues/71280
2024-01-16 10:15:15 +01:00
Yingwei Zheng
2eb7a82af3
[InstCombine] Relax the one-use constraints for icmp pred (binop X, Z), (binop Y, Z) (#76384)
This patch relaxes the one-use constraints for `icmp pred (binop X, Z),
(binop Y, Z)`. It will enable more optimizations with pointer
arithmetic.
One example in `boost::match_results::set_size`:

```
declare void @use(i64)
define i1 @src(ptr %a1, ptr %a2, ptr %add.ptr.i66, i64 %sub.ptr.rhs.cast.i) {
  %sub.ptr.lhs.cast.i = ptrtoint ptr %a1 to i64
  %sub.ptr.rhs.cast.i = ptrtoint ptr %a2 to i64
  %sub.ptr.sub.i = sub i64 %sub.ptr.lhs.cast.i, %sub.ptr.rhs.cast.i
  %sub.ptr.div.i = sdiv exact i64 %sub.ptr.sub.i, 24
  call void @use(i64 %sub.ptr.div.i)
  %sub.ptr.lhs.cast.i.i = ptrtoint ptr %add.ptr.i66 to i64
  %sub.ptr.sub.i.i = sub i64 %sub.ptr.lhs.cast.i.i, %sub.ptr.rhs.cast.i
  %sub.ptr.div.i.i = sdiv exact i64 %sub.ptr.sub.i.i, 24
  %cmp.i.not.i.i = icmp eq i64 %sub.ptr.div.i.i, %sub.ptr.div.i
  ret i1 %cmp.i.not.i.i
}
define i1 @tgt(ptr %a1, ptr %a2, ptr %add.ptr.i66, i64 %sub.ptr.rhs.cast.i) {
  %sub.ptr.lhs.cast.i = ptrtoint ptr %a1 to i64
  %sub.ptr.rhs.cast.i = ptrtoint ptr %a2 to i64
  %sub.ptr.sub.i = sub i64 %sub.ptr.lhs.cast.i, %sub.ptr.rhs.cast.i
  %sub.ptr.div.i = sdiv exact i64 %sub.ptr.sub.i, 24
  call void @use(i64 %sub.ptr.div.i)
  %cmp.i.not.i.i = icmp eq i64 %sub.ptr.sub.i.i, %sub.ptr.sub.i
  ret i1 %cmp.i.not.i.i
}
```
2024-01-07 20:16:12 +08:00
Mikhail Gudim
7a581c34f1
Reland "[InstCombine] Extend foldICmpBinOp to add-like or" (#76531)
The original PR had a typo which was causing a bug.
2023-12-30 01:55:07 -05:00
Yingwei Zheng
aacff347af
[InstCombine] Simplify icmp pred (sdiv exact X, C), (sdiv exact Y, C) into icmp pred X, Y when C is positive (#76409)
Alive2: https://alive2.llvm.org/ce/z/u49dQ9
It will improve the codegen of `std::_Vector_base<T>::~_Vector_base()` when `sizeof(T)` is not a power of 2.

NOTE: We can also fold `icmp signed-pred (sdiv exact X, C), (sdiv exact Y, C)` into `icmp signed-pred (sdiv exact Y, C), (sdiv exact X, C)` when C is negative. But I don't think it enables more optimizations for real-world applications.
2023-12-27 06:06:16 +08:00
Mikhail Gudim
411cba215a
Revert "[InstCombine] Extend foldICmpBinOp to add-like or. (#71… (#76167)
…396)"

This reverts commit 8773c9be3d9868288f1f46957945d50ff58e4e91.
2023-12-21 11:41:09 -05:00
Mikhail Gudim
8773c9be3d
[InstCombine] Extend foldICmpBinOp to add-like or. (#71396)
InstCombine canonicalizes `add` to `or` when possible, but this makes
some optimizations applicable to `add` to be missed because they don't
realize that the `or` is equivalent to `add`.

In this patch we generalize `foldICmpBinOp` to handle such cases.
2023-12-20 17:28:57 -05:00
Craig Topper
7ec4f6094e
[InstCombine] Infer disjoint flag on Or instructions. (#72912)
The disjoint flag was recently added to IR in #72583

We already set it when we turn an add into an or. This patch sets it on Ors that weren't converted from an Add.
2023-12-02 14:11:12 -08:00
Nikita Popov
e4a4122eb6
[IR] Remove zext and sext constant expressions (#71040)
Remove support for zext and sext constant expressions. All places
creating them have been removed beforehand, so this just removes the
APIs and uses of these constant expressions in tests.

There is some additional cleanup that can be done on top of this, e.g.
we can remove the ZExtInst vs ZExtOperator footgun.

This is part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
2023-11-03 10:46:07 +01:00
Yingwei Zheng
a7f962c007
[InstCombine] Canonicalize and(zext(A), B) into select A, B & 1, 0 (#66740)
This patch canonicalizes the pattern `and(zext(A), B)` into `select A, B
& 1, 0`. Thus, we can reuse transforms `select B == even, B & 1, 0 -> 0`
and `select B == odd, B & 1, 0 -> zext(B == odd)` in `InstCombine`.
It is an alternative to #66676. 
Alive2: https://alive2.llvm.org/ce/z/598phE
Fixes #66733.
Fixes #66606.
Fixes #28612.
2023-09-29 02:51:58 +08:00
Nikita Popov
5de89b4d99 [ValueTracking] Support non-zero pow2 for shl with nowrap flags
If the shl has either nuw or nsw flags, then we know that bits
cannot be shifted out, so a power of two cannot become zero.

Proofs: https://alive2.llvm.org/ce/z/4QfebE
2023-08-07 17:04:55 +02:00
Nikita Popov
95cd6aedc1 [InstCombine] Add tests for non-zero pow2 of shl with nowrap flags (NFC) 2023-08-07 17:00:50 +02:00
Nikita Popov
03de1cb715 [InstCombine][CGP] Move swapMayExposeCSEOpportunities() fold
InstCombine tries to swap compare operands to match sub instructions
in order to expose "CSE opportunities". However, it doesn't really
make sense to perform this transform in the middle-end, as we cannot
actually CSE the instructions there.

The backend already performs this fold in
18f5446a45/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp (L4236)
on the SDAG level, however this only works within a single basic block.

To handle cross-BB cases, we do need to handle this in the IR layer.
This patch moves the fold from InstCombine to CGP in the backend,
while keeping the same (somewhat dubious) heuristic.

Differential Revision: https://reviews.llvm.org/D152541
2023-06-15 14:17:58 +02:00
Jun Zhang
e3175f7f1b
[InstCombine] icmp(X | OrC, C) --> icmp(X, 0)
We can eliminate the or operation based on the predicate and the
relation between OrC and C.

sge: X | OrC s>= C --> X s>= 0 iff OrC s>= C s>= 0
sgt: X | OrC s>  C --> X s>= 0 iff OrC s>  C s>= 0
sle: X | OrC s<= C --> X s<  0 iff OrC s>  C s>= 0
slt: X | OrC s<  C --> X s<  0 iff OrC s>= C s>= 0

Alive2 links:
sge: https://alive2.llvm.org/ce/z/W-6FHE
sgt: https://alive2.llvm.org/ce/z/TKK2yJ
sle: https://alive2.llvm.org/ce/z/vURQGM
slt: https://alive2.llvm.org/ce/z/JAsVfw

Related issue: https://github.com/llvm/llvm-project/issues/61538

Signed-off-by: Jun Zhang <jun@junz.org>

Differential Revision: https://reviews.llvm.org/D147597
2023-04-13 17:26:24 +08:00
Jun Zhang
a088beaef2
Add baseline tests for D147597
Signed-off-by: Jun Zhang <jun@junz.org>
2023-04-13 17:26:21 +08:00
Sanjay Patel
e44a305690 [InstCombine] invert canonicalization of sext (x > -1) --> not (ashr x)
https://alive2.llvm.org/ce/z/2iC4oB

This is similar to changes made for zext + lshr:
21d3871b7c90
6c39a3aae1dc

The existing fold did not account for extra uses, so we
see some instruction count reductions in the test diffs.

This is intended to improve analysis (icmp likely has more
transforms than any other opcode), make other transforms
more symmetric with zext/lshr, and it can be inverted
in codegen if profitable.

As with the earlier changes, there is potential to uncover
infinite combine loops, but I have not found any yet.
2023-01-24 16:44:15 -05:00
Sanjay Patel
82dcfe0dbb [InstCombine] allow matching vector types for icmp-of-mask/cast
Also use a more specific matcher to simplify the mask
compare to type size.
2023-01-23 18:23:43 -05:00
Sanjay Patel
7a9e3ad070 [InstCombine] relax one-use check for icmp with mask/cast 2023-01-23 18:23:43 -05:00
Sanjay Patel
3a295f0335 [InstCombine] add tests for masked/casted icmp; NFC 2023-01-23 18:23:43 -05:00
Sanjay Patel
3e58d94780 [InstCombine] remove dead pattern matching code; NFC
Complexity canonicalization guarantees that a binop and cast
are op0/op1 respectively. Adjusted generic test names to
show that this pattern is still useful.
2023-01-23 18:23:43 -05:00
Craig Topper
77f2f34d69 [InstCombine] Generalize (icmp sgt (1 << Y), -1) -> (icmp ne Y, BitWidth-1) to any negative constant.
Similar for the sle version which will be canonicalized to slt first.

Alive2 proof as implemented: https://alive2.llvm.org/ce/z/_YawdM

@spatel's  original Alive2: https://alive2.llvm.org/ce/z/3YB2vs

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D141773
2023-01-15 13:36:57 -08:00
Craig Topper
ff39b7ea89 [InstCombine] Optimize (icmp slt (1 << Y), 1) -> (icmp eq Y, BitWidth-1).
The code tried to do this for (icmp sle (1 << Y), 0), but that is
canonicalized to sgt before we get there.

Simplify the code by removing the unreachable SGE and SLE handling.

Also remove the (1 << Y) >=u 2147483648 and (1 << Y) <u 2147483648
handling since those are canonicalized to (1 << Y) <s 0 and
(1 << Y) >=s 0 before we get there.

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D141753
2023-01-14 11:18:47 -08:00
Craig Topper
bb83dc10f5 [InstCombine] Add test coverage for (icmp sgt/sle (1 << Y), 0). NFC"
We already optimize the sgt case to (icmp ne Y, BitWidth-1), but
we miss optimizing sle because it canonicalizes to (icmp slt (1 << X), 1)
first.
2023-01-14 00:38:07 -08:00
Craig Topper
5be3894b60 Revert "[InstCombine] Add test coverage for (icmp slt/sge (1 << Y), 0). NFC"
This reverts commit e25f2287dd7d6854b0bbfb9878fecdbbad21038d.

I messed up the predicates in the description.
2023-01-14 00:37:01 -08:00
Craig Topper
e25f2287dd [InstCombine] Add test coverage for (icmp slt/sge (1 << Y), 0). NFC
We already optimize the slt case to (icmp eq Y, BitWidth-1), but
we miss optimizing sge because it canonicalizes to (icmp sgt (1 << X), 1)
first.
2023-01-14 00:31:14 -08:00
Sanjay Patel
d4493dd1ed [InstCombine] add nuw to any (1<<x)
https://alive2.llvm.org/ce/z/9EjDKE

This was mentioned as a missing fold in D139598.

It can unlock follow-on folds in some cases.
This verifies one of the changed tests:
https://alive2.llvm.org/ce/z/B_btDM
2022-12-15 12:03:47 -05:00
Nikita Popov
de59d222ab [InstCombine] Support logical ops in foldAndOrOfICmpEqZeroAndICmp()
If the and/or is logical and one of the operands only occurs on the
RHS, we need to freeze it: https://alive2.llvm.org/ce/z/vuMuE_
2022-12-13 10:36:55 +01:00
Nikita Popov
81ac23e0c2 [InstCombine] Add additional tests for foldAndOrOfICmpEqZeroAndICmp (NFC)
Adds logical variants of the commuted tests.
2022-12-13 10:08:53 +01:00
Bjorn Pettersson
4ab40eca08 [test][InstCombine] Update some test cases to use opaque pointers
These tests cases were converted using the script at
https://gist.github.com/nikic/98357b71fd67756b0f064c9517b62a34

Differential Revision: https://reviews.llvm.org/D135094
2022-10-03 22:17:59 +02:00
Sanjay Patel
7073ec530e [InstCombine] canonicalize more zext-and-of-bool compare to narrow and
https://alive2.llvm.org/ce/z/vBNiiM

This matches variants of patterns that were folded with:
b5a9361c90ca
2022-07-30 11:22:05 -04:00
Alexander Shaposhnikov
4220ef2be1 [InstCombine] Add fold for redundant sign bits count comparison
For power-of-2 C:
((X s>> ShiftC) ^ X) u< C --> (X + C) u< (C << 1)
((X s>> ShiftC) ^ X) u> (C - 1) --> (X + C) u> ((C << 1) - 1)

(https://github.com/llvm/llvm-project/issues/56479)

Test plan:
0/ ninja check-llvm check-clang + bootstrap LLVM/Clang
1/ https://alive2.llvm.org/ce/z/eEUfx3

Differential revision: https://reviews.llvm.org/D130433
2022-07-30 09:06:53 +00:00
Alexander Shaposhnikov
f4aa08586a [InstCombine] Add baseline tests for redundant sign bits count folds
Add baseline tests. NFC.
(https://github.com/llvm/llvm-project/issues/56479)

Test plan:
ninja check-all

Differential revision: https://reviews.llvm.org/D130605
2022-07-30 08:23:54 +00:00