1075 Commits

Author SHA1 Message Date
David Majnemer
902acde341 [InstCombine] Optimize away certain additions using modular arithmetic
We can turn:
```
  %add = add i8 %arg, C1
  %and = and i8 %add, C2
  %cmp = icmp eq i1 %and, C3
```

into:
```
  %and = and i8 %arg, C2
  %cmp = icmp eq i1 %and, (C3 - C1) & C2
```

This is only worth doing if the sequence is the sole user of the addition
operation.
2024-10-28 22:51:35 +00:00
Noah Goldstein
294726d738 Reapply "[InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)" (#111236)
The underlying issue with msan was fixed by #113200
2024-10-23 09:12:08 -05:00
Kazu Hirata
8819267747
[InstCombine] Simplify code with SmallMapVector::operator[] (NFC) (#113022) 2024-10-19 14:38:40 -07:00
Jay Foad
85c17e4092
[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112706)
Convert many instances of:
  Fn = Intrinsic::getOrInsertDeclaration(...);
  CreateCall(Fn, ...)
to the equivalent CreateIntrinsic call.
2024-10-17 16:20:43 +01:00
Yingwei Zheng
095d49da76
[InstCombine] Set samesign when converting signed predicates into unsigned (#112642)
Alive2: https://alive2.llvm.org/ce/z/6cqdt-
2024-10-17 20:43:48 +08:00
Yingwei Zheng
0936195311
[InstCombine] Drop samesign in InstCombine (#112480)
Closes https://github.com/llvm/llvm-project/issues/112476.
2024-10-16 19:13:52 +08:00
Rahul Joshi
fa789dffb1
[NFC] Rename Intrinsic::getDeclaration to getOrInsertDeclaration (#111752)
Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).
2024-10-11 05:26:03 -07:00
Kazu Hirata
2d8cd32ae5
[InstCombine] Avoid repeated hash lookups (NFC) (#111618) 2024-10-08 20:37:33 -07:00
Vitaly Buka
574266ce33
Revert "[InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)" (#111236)
Reverts #110880 because of exposed issue is Msan instrumentation
#111212.

This reverts commit a64643688526114b50c25b3eda8a57855bd2be87.
2024-10-04 23:20:40 -07:00
Nikita Popov
67d247a441
[InstCombine] Decompose more icmps into masks (#110836)
Extend decomposeBitTestICmp() to handle cases where the resulting
comparison is of the form `icmp (X & Mask) pred C` with non-zero
`C`. Add a flag to allow code to opt-in to this behavior and use it in
the "log op of icmp" fold infrastructure.

This addresses regressions from #97289.

Proofs: https://alive2.llvm.org/ce/z/hUhdbU
2024-10-04 10:17:23 +02:00
Noah Goldstein
a646436885 [InstCombine] Folding (icmp eq/ne (and X, -P2), INT_MIN)
Folds to `(icmp slt/sge X, (INT_MIN + P2))`

Proofs: https://alive2.llvm.org/ce/z/vpNFY5

Closes #110880
2024-10-03 13:05:08 -05:00
Nikita Popov
7de492f90d [InstCombine] Preserve nuw flag in indexed compare fold
If all the involved GEPs have the nuw flag, also preserve it on
the resulting adds and GEPs.
2024-10-02 16:03:47 +02:00
Yingwei Zheng
2a2c35a9a6
[InstCombine] Fold icmp spred (mul nsw X, Z), (mul nsw Y, Z) into icmp spred X, Y (#110630)
```
icmp spred (mul nsw X, Z), (mul nsw Y, Z) -> icmp spred X, Y iff Z > 0
icmp spred (mul nsw X, Z), (mul nsw Y, Z) -> icmp spred Y, X iff Z < 0
```
Alive2: https://alive2.llvm.org/ce/z/9fXFfn
2024-10-01 22:16:05 +08:00
Yingwei Zheng
1efd1227b2
[InstCombine] Fold icmp eq/ne (X *nw Z), (Y *nw Z) -> icmp eq/ne Z, 0 when X != Y (#110413)
Alive2: https://alive2.llvm.org/ce/z/9oDP6K
I found this pattern in
04e75858d7/casadi/core/repmat.cpp (L70-L78).
2024-09-30 10:21:20 +08:00
Nikita Popov
b8d1bae648
[CmpInstAnalysis] Return decomposed bit test as struct (NFC) (#109819)
decomposeBitTestICmp() currently returns the result via two out
parameters plus an in-place modification of Pred. This changes it to
return an optional struct instead.

The motivation here is twofold. First, I'd like to extend this code to
handle cases where the comparison is against a value other than zero,
which would mean yet another out parameter. Second, while doing that I
was badly bitten by the in-place modification, so I'd like to get rid of
it.
2024-09-25 10:14:15 +02:00
Marina Taylor
5cd0900ef6
[InstCombine] Compare icmp inttoptr, inttoptr values directly (#107012)
InstCombine already has some rules for `icmp ptrtoint, ptrtoint` to drop
the casts and compare the source values. This change adds the same for
the reverse case with `inttoptr`.
2024-09-24 09:39:07 +02:00
Yingwei Zheng
872932b7a9
[InstCombine] Generalize icmp (shl nuw C2, Y), C -> icmp Y, C3 (#104696)
The motivation of this patch is to fold more generalized patterns like
`icmp ult (shl nuw 16, X), 64 -> icmp ult X, 2`.

Alive2: https://alive2.llvm.org/ce/z/gyqjQH
2024-09-18 19:10:41 +08:00
c8ef
86f0399c1f
[InstCombine] Fold expression using basic properties of floor and ceiling function (#107107)
alive2: ~~https://alive2.llvm.org/ce/z/Ag3Ki7~~
https://alive2.llvm.org/ce/z/ywP5t2
related: #76438

This patch adds the following foldings: `floor(x) <= x --> true` and `x
<= ceil(x) --> true`. We leverage the properties of these math functions
and ensure there is no floating point input of `nan`.

---------

Co-authored-by: Yingwei Zheng <dtcxzyw@qq.com>
2024-09-15 14:25:00 +04:00
Nikita Popov
de2b6cb6ab
[InstCombine] Fold icmp over select of cmp more aggressively (#105536)
When folding an icmp into a select, treat an icmp of a constant with a
one-use ucmp/scmp intrinsic as a simplification. These comparisons will
reduce down to an icmp.

This addresses a regression seen in Rust and also in llvm-opt-benchmark.
2024-08-22 09:47:35 +02:00
Volodymyr Vasylkun
7e23a23d5e
[InstCombine] Fold an unsigned icmp of ucmp/scmp with a constant to an icmp of the original arguments (#104471)
Proofs: https://alive2.llvm.org/ce/z/9mv8HU
2024-08-16 13:38:13 +01:00
Volodymyr Vasylkun
8320b97ab9
[InstCombine] Fold an unsigned comparison of add nsw X, C with a constant into a signed comparison (#103480)
Given an unsigned integer comparison of `add nsw X, C1` with some
constant `C2` we can fold it into a signed comparison of `X` and `C2 -
C1` under the following conditions:
  * There's a `nsw` flag on the addition
  * `C2` is non-negative
  * `X + C1` is non-negative
  * `C2 - C1` is non-negative
2024-08-14 15:31:19 +01:00
Nikita Popov
adb4cfe0b6 [InstCombine] Use getAllOnesValue()
Split off from https://github.com/llvm/llvm-project/pull/80309.
2024-08-13 15:04:23 +02:00
Noah Goldstein
b4ac7f4fc9 [InstCombine] Fold (icmp eq/ne (or (select cond, 0/NZ, 0/NZ), X), 0)
Four cases:
`(icmp eq (or (select cond, 0, NonZero), Other))`
 -> `(and cond, (icmp eq Other, 0))`
`(icmp ne (or (select cond, NonZero, 0), Other))`
 -> `(or cond, (icmp ne Other, 0))`
`(icmp ne (or (select cond, 0, NonZero), Other))`
 -> `(or (not cond), (icmp ne Other, 0))`
`(icmp eq (or (select cond, NonZero, 0), Other))`
 -> `(and (not cond), (icmp eq Other, 0))`

These cases came up in tests on: #88088

Proofs: https://alive2.llvm.org/ce/z/ojGo_J

Closes #88183
2024-08-05 23:48:49 +08:00
mskamp
533190acdb
[InstCombine] Canonicalize Bit Testing by Shifting to Bit 0 (#101838)
Implement a new transformation that fold the bit-testing expression
(icmp ne (and (lshr V B) 1) 0) to (icmp ne (and V (shl 1 B)) 0) for
constant V. This rule already existed for non-constant V and constants
other than 1; this restriction to non-constant V has been added in
commit c3b2111d975a39d19f0c5d635e2961a4449c5a71 to fix an infinite loop.
Avoid the infinite loop by allowing constant V only if the shift
instruction is an lshr and the constant is 1. Also fold the negated
variant of the LHS.
    
This transformation necessitates an adaption of existing tests in
`icmp-and-shift.ll` and `load-cmp.ll`. One test in `icmp-and-shift.ll`,
which previously was a negative test, now gets folded. Rename it to
indicate that it is a positive test.
    
Alive proof: https://alive2.llvm.org/ce/z/vcJJTx
    
Relates to issue #86813.
2024-08-04 09:32:40 +02:00
Yingwei Zheng
8bd9ade628
[InstCombine] Fold fcmp pred sqrt(X), 0.0 -> fcmp pred2 X, 0.0 (#101626)
Proof (Please run alive-tv with larger smt-to):
https://alive2.llvm.org/ce/z/-aqixk
FMF propagation: https://alive2.llvm.org/ce/z/zyKK_p

```
sqrt(X) < 0.0 --> false
sqrt(X) u>= 0.0 --> true
sqrt(X) u< 0.0 --> X u< 0.0
sqrt(X) u<= 0.0 --> X u<= 0.0
sqrt(X) > 0.0 --> X > 0.0
sqrt(X) >= 0.0 --> X >= 0.0
sqrt(X) == 0.0 --> X == 0.0
sqrt(X) u!= 0.0 --> X u!= 0.0
sqrt(X) <= 0.0 --> X == 0.0
sqrt(X) u> 0.0 --> X u!= 0.0
sqrt(X) u== 0.0 --> X u<= 0.0
sqrt(X) != 0.0 --> X > 0.0
!isnan(sqrt(X)) --> X >= 0.0
isnan(sqrt(X)) --> X u< 0.0
```

In most cases, `sqrt` cannot be eliminated since it has multiple uses.
But this patch will break data dependencies and allow optimizer to sink
expensive `sqrt` calls into successor blocks.
2024-08-03 13:35:22 +08:00
Yingwei Zheng
62e9f40949
[PatternMatch] Use m_SpecificCmp matchers. NFC. (#100878)
Compile-time improvement:
http://llvm-compile-time-tracker.com/compare.php?from=13996378d81c8fa9a364aeaafd7382abbc1db83a&to=861ffa4ec5f7bde5a194a7715593a1b5359eb581&stat=instructions:u
baseline: 803eaf29267c6aae9162d1a83a4a2ae508b440d3
```
Top 5 improvements:
  stockfish/movegen.ll 2541620819 2538599412 -0.12%
  minetest/profiler.cpp.ll 431724935 431246500 -0.11%
  abc/luckySwap.c.ll 581173720 580581935 -0.10%
  abc/kitTruth.c.ll 2521936288 2519445570 -0.10%
  abc/extraUtilTruth.c.ll 1216674614 1215495502 -0.10%
Top 5 regressions:
  openssl/libcrypto-shlib-sm4.ll 1155054721 1155943201 +0.08%
  openssl/libcrypto-lib-sm4.ll 1155054838 1155943063 +0.08%
  spike/vsm4r_vv.ll 1296430080 1297039258 +0.05%
  spike/vsm4r_vs.ll 1312496906 1313093460 +0.05%
  nuttx/lib_rand48.c.ll 126201233 126246692 +0.04%
Overall: -0.02112308%
```
2024-07-29 10:04:06 +08:00
Nikita Popov
11484cb817 [InstCombine] Pass SimplifyQuery to SimplifyDemandedBits()
This will enable calling SimplifyDemandedBits() with a SimplifyQuery
that has CondContext set in the future.

Additionally this also marginally strengthens the analysis by
retaining the original context instruction for one-use chains.
2024-07-01 12:41:21 +02:00
SahilPatidar
a7bf4124bf
[InstCombine] Fold fcmp pred (x - y), 0 into fcmp pred x, y (#85506)
Resolve #85245
Alive2: https://alive2.llvm.org/ce/z/F4rDwK

---------

Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
Co-authored-by: Jay Foad <jay.foad@gmail.com>
2024-06-29 15:37:35 +08:00
Stephen Tozer
d75f9dd1d2 Revert "[IR][NFC] Update IRBuilder to use InsertPosition (#96497)"
Reverts the above commit, as it updates a common header function and
did not update all callsites:

  https://lab.llvm.org/buildbot/#/builders/29/builds/382

This reverts commit 6481dc57612671ebe77fe9c34214fba94e1b3b27.
2024-06-24 18:00:22 +01:00
Stephen Tozer
6481dc5761
[IR][NFC] Update IRBuilder to use InsertPosition (#96497)
Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock
alongside a BasicBlock::iterator, using the fact that we can now get the
parent basic block from the iterator even if it points to the sentinel.
This patch removes the BasicBlock argument from each constructor or call
to setInsertPoint.

This has no functional effect, but later on as we look to remove the
`Instruction *InsertBefore` argument from instruction-creation
(discussed
[here](https://discourse.llvm.org/t/psa-instruction-constructors-changing-to-iterator-only-insertion/77845)),
this will simplify the process by allowing us to deprecate the
InsertPosition constructor directly and catch all the cases where we use
instructions rather than iterators.
2024-06-24 17:27:43 +01:00
Poseydon42
905e4ec747
[InstCombine] Implement folds of icmp of UCMP/SCMP call and a constant (#96118)
This patch handles various cases where an operation of the kind `icmp
(ucmp/scmp x, y), constant` folds to `icmp x, y`. Another patch with
cases where this operation folds to a constant (i.e. dumb cases like
`icmp eq (cmp x, y), 4` should be published in a couple of days.

I wasn't sure what negative tests should be added here, if any are
necessary at all. I'd love to hear your suggestions.

Proofs (ucmp): https://alive2.llvm.org/ce/z/qQ7ihz
Proofs (scmp): https://alive2.llvm.org/ce/z/cipKEn

---------

Co-authored-by: Nikita Popov <github@npopov.com>
2024-06-22 12:23:20 +08:00
Antonio Frighetto
a4b44c003e [InstCombine] Canonicalize icmp ult (add X, C2), C expressions
`icmp ult (add X, C2), C` can be folded to `icmp ne (and X, C), 2C`,
subject to `C == -C2` and C2 being a power of 2.

Proofs: https://alive2.llvm.org/ce/z/P-VVmQ.

Fixes: https://github.com/llvm/llvm-project/issues/75613.
2024-06-17 19:34:48 +02:00
Noah Goldstein
77f75b45fd [InstCombine] Extend (icmp eq/ne (and Z, X), (and Z, Y)) folds
Two ways to relaxed:
    1) Only require one of the `and` ops to be single-use if both `X`
       and `Y` are constant.
    2) Special case, if `X ^ Y` is a negative power of 2, then
       `Z ^ NegP2 ==/!= 0` will fold to `Z u</u>= -NegP2` which
       creates no additional instructions so fold irrelivant of
       use counts.

Closes #94867
2024-06-10 13:00:28 -05:00
Yingwei Zheng
e4b0655b29
[InstCombine] Fix missing argument typo in InstCombinerImpl::foldICmpShlConstant (#94899)
Closes #94897.
2024-06-10 03:17:01 +08:00
Noah Goldstein
166c1849d6 [InstCombine] Fold (icmp eq/ne (xor x, y), C1) even if multiuse
Two folds unlocked:
    `(icmp eq/ne (xor x, C0), C1)` -> `(icmp eq/ne x, C2)`
    `(icmp eq/ne (xor x, y), 0)` -> `(icmp eq/ne x, y)`

This fixes regressions assosiated with #87180

Closes #87275
2024-06-07 15:06:41 -05:00
Noah Goldstein
3716a3c1bc [InstCombine] Folding multiuse (icmp eq/ne (or X, Y), Y) for 2 uses of Y
The fold will replace 2 uses of `Y` we should also do fold if `Y` has
2 uses (not only oneuse).

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D159062
2024-06-06 12:42:07 -05:00
Noah Goldstein
c3c443bb41 [InstCombine] Add transforms (icmp spred (and X, Y), X) if X or Y are known signed/unsigned
Several transforms:
    1) If known `Y < 0`:
        - slt -> ult: https://alive2.llvm.org/ce/z/9zt2iK
        - sle -> ule: https://alive2.llvm.org/ce/z/SPoPNF
        - sgt -> ugt: https://alive2.llvm.org/ce/z/IGNxAk
        - sge -> uge: https://alive2.llvm.org/ce/z/joqTvR
    2) If known `Y >= 0`:
        - `(X & PosY) s> X --> X s< 0`
            - https://alive2.llvm.org/ce/z/7e-5BQ
        - `(X & PosY) s> X --> X s< 0`
            - https://alive2.llvm.org/ce/z/jvT4Gb
    3) If known `X < 0`:
        - `(NegX & Y) s> NegX --> Y s>= 0`
            - https://alive2.llvm.org/ce/z/ApkaEh
        - `(NegX & Y) s<= NegX --> Y s< 0`
            - https://alive2.llvm.org/ce/z/oRnfHp

Closes #94417
2024-06-06 03:05:41 -05:00
Noah Goldstein
5532ab1732 [InstCombine] Make the (icmp eq/ne (and X, Y), X) canonicalization work for non-const operands
We currently do:
    `(icmp eq/ne (and X, Y), Y)` -> `(icmp eq/ne (and ~X, Y), 0)`
if `X` is constant. We can make this more general and do it if `X` is
freely invertable (i.e say `X = ~Z`).

As well, we can also do:
    `(icmp eq/ne (and X, Y), Y)` -> `(icmp eq/ne (or X, ~Y), -1)`
If `Y` is freely invertible.

Proofs: https://alive2.llvm.org/ce/z/yeWH3E

Differential Revision: https://reviews.llvm.org/D159059

Closes #84688
2024-05-29 02:38:23 -05:00
Yingwei Zheng
5c7c1f6aba
[InstCombine] Try the flipped strictness of predicate in foldICmpShlConstant (#92773)
This patch extends the transform `(icmp pred iM (shl iM %v, N), C) ->
(icmp pred i(M-N) (trunc %v iM to i(M-N)), (trunc (C>>N))` to handle
icmps with the flipped strictness of predicate.

See the following case:
```
icmp ult i64 (shl X, 32), 8589934593 ->
icmp ule i64 (shl X, 32), 8589934592 ->
icmp ule i32 (trunc X, i32), 2 ->
icmp ult i32 (trunc X, i32), 3
```

Fixes the regression introduced by
https://github.com/llvm/llvm-project/pull/86111#issuecomment-2098203152.

Alive2 proofs: https://alive2.llvm.org/ce/z/-sp5n3

`nuw` cannot be propagated as we always use `ashr` here. I don't see the
value of fixing this (see the test `test_icmp_shl_nuw`).
2024-05-28 12:47:27 +08:00
Nikita Popov
0748a98ab0 [InstCombine] Handle ConstantFoldCompareInstOperands() failure
This function will return nullptr instead of returning a constant
expression now, so be sure to handle that.

Fixes https://github.com/llvm/llvm-project/issues/93017.
2024-05-22 14:33:50 +02:00
Noah Goldstein
23f1047daa [InstCombine] Fold (icmp pred (trunc nuw/nsw X), C) -> (icmp pred X, (zext/sext C))
This is valid as long as the sign of the wrap flag doesn't differ from
the sign of the `pred`.

Proofs: https://alive2.llvm.org/ce/z/35NsrR

NB: The online Alive2 hasn't been updated with `trunc nuw/nsw`
support, so the proofs must be reproduced locally.

Closes #87935
2024-05-16 13:03:32 -05:00
Alex Bradbury
3be8e2c95d
[InstCombine] Prefer to keep power-of-2 constants when combining ashr exact and slt/ult of a constant (#86111)
We have flexibility in what constant to use when combining an `ashr
exact` with a slt or ult of a constant, and it's not possible to revisit
this decision later in the compilation pipeline after the `ashr exact`
is removed. Keeping a constant close to power-of-2 (pow2val + 1) should
be no worse than neutral, and in some cases may allow better codegen
later on for targets that can more cheaply generate power of 2 (which
may be selectable if converting back to setle/setge) or near power of 2
constants.

Alive2 proofs:
<https://alive2.llvm.org/ce/z/2BmPnq> and
<https://alive2.llvm.org/ce/z/DtuhnR>
2024-05-10 13:50:03 +01:00
Eli Friedman
f893dccbba
Replace uses of ConstantExpr::getCompare. (#91558)
Use ICmpInst::compare() where possible, ConstantFoldCompareInstOperands
in other places. This only changes places where the either the fold is
guaranteed to succeed, or the code doesn't use the resulting compare if
we fail to fold.
2024-05-09 16:50:01 -07:00
Yingwei Zheng
d3dad7a74b
[InstCombine] Fix miscompilation caused by #90436 (#91133)
Proof: https://alive2.llvm.org/ce/z/iRnJ4i

Fixes https://github.com/llvm/llvm-project/issues/91127.
2024-05-06 16:30:07 +08:00
Noah Goldstein
f561daf989 [InstCombine] Add example usage for new Checked matcher API
There is no real motivation for this change other than to highlight a
case where the new `Checked` matcher API can handle non-splat-vecs
without increasing code complexity.

Closes #85676
2024-05-03 14:10:24 -05:00
Nikita Popov
b0eeacb203
[InstCombine] Fold icmp of trunc nuw/nsw (#90436)
Convert the existing foldICmpTruncWithTruncOrExt() fold to work with
trunc nowrap flags instead of computeKnownBits(). This also allows us to
generalize the fold to work with signed comparisons.

Interestingly, apart from the obvious combinations like signed
predicates with trunc nsw, some non-obvious ones are also legal. For
example for unsigned predicates we can do the transform for two trunc
nsw as well (rather than only trunc nuw).

Proofs: https://alive2.llvm.org/ce/z/ndewwK
2024-05-03 11:09:00 +09:00
Maciej Gabka
bfc0317153
Move several vector intrinsics out of experimental namespace (#88748)
This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from the experimental namespace.

All these intrinsics exist in LLVM for more than a year now, and are
widely used, so should not be considered as experimental.
2024-04-29 10:16:45 +01:00
Nikita Popov
873889b7fa [InstCombine] Extract logic for "emit offset and rewrite gep" (NFC) 2024-04-25 14:18:11 +09:00
Yingwei Zheng
9fb7a736f0
[InstCombine] Fold fcmp into select (#86482)
This patch simplifies `fcmp (select Cond, C1, C2), C3` patterns in
ceres:
Alive2: https://alive2.llvm.org/ce/z/fWh_sD
```
define i1 @src(double %x) {
  %cmp1 = fcmp ord double %x, 0.000000e+00
  %sel = select i1 %cmp1, double 0xFFFFFFFFFFFFFFFF, double 0.000000e+00
  %cmp2 = fcmp oeq double %sel, 0.000000e+00
  ret i1 %cmp2
}

define i1 @tgt(double %x) {
  %cmp1 = fcmp uno double %x, 0.000000e+00
  ret i1 %cmp1
}

```
2024-04-23 19:35:15 +08:00
SahilPatidar
225ae82fdb
[InstCombine] fold cond ? x : -x == 0 into x == 0 (#85673)
Resolve #85250 
Alive2: https://alive2.llvm.org/ce/z/7DMRCy
2024-04-20 10:45:14 +02:00