888 Commits

Author SHA1 Message Date
Craig Topper
2872987e5e [InstCombine] Fix InstCombinerImpl::foldICmpMulConstant for nsw and nuw mul with unsigned compare.
If we have both an nsw and nuw flag, we would see the nsw flag
first and only handle signed comparisons.

This patch ignores the nsw flag if the comparison isn't signed.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D143766
2023-02-14 23:43:17 -08:00
chenglin.bi
dd31a3b3a5 [InstCombine] fold icmp of the sum of ext bool based on limited range
For the pattern `(zext i1 X) + (sext i1 Y)`, the constant range is [-1, 1].
We can simplify the pattern by logical operations. Like:

```
    (zext i1 X) + (sext i1 Y) == -1 -->  ~X & Y
    (zext i1 X) + (sext i1 Y) == 0  --> ~(X ^ Y)
    (zext i1 X) + (sext i1 Y) == 1 --> X & ~Y
```
And other predicates can the combination of these results:

```
    (zext i1 X) + (sext i1 Y)) != -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) s> -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) u< -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) s> 0 --> X & ~Y
    (zext i1 X) + (sext i1 Y)) s< 0 --> ~X & Y
    (zext i1 X) + (sext i1 Y)) != 1 --> ~X | Y
    (zext i1 X) + (sext i1 Y)) s< 1 --> ~X | Y
    (zext i1 X) + (sext i1 Y)) u> 1 --> ~X & Y
```

All alive proofs:
https://alive2.llvm.org/ce/z/KmgDpF
https://alive2.llvm.org/ce/z/fLwWa9
https://alive2.llvm.org/ce/z/ZKQn2P

Fix: https://github.com/llvm/llvm-project/issues/59666

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143373
2023-02-15 10:34:00 +08:00
chenglin.bi
6f149a17d4 [InstCombine] Look through truncate to fold icmp with intrinsics
The output of intrinsic functions like ctpop, cttz, ctlz have limited range from 0 to bitwidth. So if the truncate destination type can hold the source bitwidth size, we can just ignore the truncate and use the truncate src to do combination.

Alive2 proofs:
https://alive2.llvm.org/ce/z/9D_-qP

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143368
2023-02-10 10:33:07 +08:00
Noah Goldstein
abbd256a81 Improve transforms for (icmp uPred X * Z, Y * Z) -> (icmp uPred X, Y)
Several cases where missing.

1. `(icmp eq/ne X*Z, Y*Z) [if Z % 2 != 0] -> (icmp eq/ne X, Y)`
    EQ: https://alive2.llvm.org/ce/z/6_HPZ5
    NE: https://alive2.llvm.org/ce/z/c34qSU

    There was previously an implementation of this that work of `Y`
    was non-constant, but it was missing if `Y*Z` evaluated to a
    constant and/or `nsw`/`nuw` where both false. As well it only
    worked if `Z` was a constant but we can check 1s bit of
    `KnownBits` to cover more cases.

2. `(icmp eq/ne X*Z, Y*Z) [if Z != 0 and nsw(X*Y) and nsw(Y*Z)] -> (icmp eq/ne X, Y)`
    EQ: https://alive2.llvm.org/ce/z/6SdAG6
    NE: https://alive2.llvm.org/ce/z/fjsq_b

    This was previously implemented only to work if `Z` was constant,
    but we can use `isKnownNonZero` to cover more cases.

3. `(icmp uPred X*Y, Y*Z) [if Z != 0 and nuw(X*Y) and nuw(X*Y)] -> (icmp uPred X, Y)`
    EQ:  https://alive2.llvm.org/ce/z/FqWQLX
    NE:  https://alive2.llvm.org/ce/z/2gHrd2
    ULT: https://alive2.llvm.org/ce/z/MUAWgZ
    ULE: https://alive2.llvm.org/ce/z/szQQ2L
    UGT: https://alive2.llvm.org/ce/z/McVUdu
    UGE: https://alive2.llvm.org/ce/z/95uyC8

    This was previously implemented only for `eq/ne` cases. As well
    only if `Z` was constant, but again we can use `isKnownNonZero` to
    cover more cases.

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D142786
2023-02-06 14:09:18 -06:00
Noah Goldstein
2a3732f934 Add transform for (mul X, OddC) eq/ne N * C --> X eq/ne N
We previously only did this if the `mul` was `nuw`, but it works for
any odd value.

Alive2 Links:
EQ: https://alive2.llvm.org/ce/z/6_HPZ5
NE: https://alive2.llvm.org/ce/z/c34qSU

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143026
2023-02-06 14:09:18 -06:00
Sanjay Patel
eb6f5f1ada [InstCombine] reduce icmp_ne0-of-and-of-select-of-constants
Follow-up to:
D142847 / 98855059674c

This handles the 'ne' variant by inverting the result.

https://alive2.llvm.org/ce/z/D229jS
2023-01-31 14:59:55 -05:00
Sanjay Patel
9885505967 [InstCombine] reduce icmp_eq0-of-and-of-select-of-constants
This is the most basic patch to handle fixing issue #57666.

D133919 proposes to handle much more than this in a single patch,
but I've used 10 regression tests just to make sure this part is
doing what I expected and nothing more, and it already shows even
more potential TODO items.

The more general proofs from D133919 are correct, but I want to
enable this in smaller steps to reduce risk:
https://alive2.llvm.org/ce/z/RrVEyX

Differential Revision: https://reviews.llvm.org/D142847
2023-01-30 16:17:15 -05:00
Kazu Hirata
55e2cd1609 Use llvm::count{lr}_{zero,one} (NFC) 2023-01-28 12:41:20 -08:00
Noah Goldstein
baf7f7e575 Recommit "Add optimizations for icmp eq/ne (mul(X, Y), 0)" 2nd Try
First time caused build failure:
    https://lab.llvm.org/buildbot/#/builders/183/builds/10447
but after investigating it seems to be unrelated. The same
test/build passed later with the original commit here:
    https://lab.llvm.org/buildbot/#/builders/183/builds/10448

1. Add checks if X and/or Y are odd. The Odd values are unnecessary to
   the icmp: isZero(Odd * N) == isZero(N)

2. If neither X nor Y is known odd, then if X * Y cannot overflow AND
   if X and/or Y is non-zero, the non-zero values are unnecessary to the
   icmp.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D140850
2023-01-27 20:35:40 -06:00
Noah Goldstein
e7e3d723a5 Revert "Add optimizations for icmp eq/ne (mul(X, Y), 0)"
This reverts commit aa250ceb3ff15d55bb506b4bc8196f143133d8b5.

Caused test failures in Clangd: https://lab.llvm.org/buildbot/#/builders/183/builds/10447
reverting while investigating.
2023-01-27 18:44:02 -06:00
Noah Goldstein
aa250ceb3f Add optimizations for icmp eq/ne (mul(X, Y), 0)
1. Add checks if X and/or Y are odd. The Odd values are unnecessary to
   the icmp: isZero(Odd * N) == isZero(N)

2. If neither X nor Y is known odd, then if X * Y cannot overflow AND
   if X and/or Y is non-zero, the non-zero values are unnecessary to the
   icmp.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D140850
2023-01-27 17:45:35 -06:00
Sanjay Patel
0ef7cbc319 [InstCombine] reduce compare of signbits of 2 values, signed variant
(X s>> BitWidth - 1) == sext (Y s> -1) --> (X ^ Y) < 0
(X s>> BitWidth - 1) != sext (Y s> -1) --> (X ^ Y) > -1

This is the same logic as:
7cbfc39c77ca
...extended to deal with "signed" cast+shift instructions.

https://alive2.llvm.org/ce/z/LLidya
2023-01-26 08:58:45 -05:00
Sanjay Patel
7cbfc39c77 [InstCombine] reduce compare of signbits of 2 values
Test if 2 values have different or same signbits:
(X u>> BitWidth - 1) == zext (Y s> -1) --> (X ^ Y) < 0
(X u>> BitWidth - 1) != zext (Y s> -1) --> (X ^ Y) > -1

https://alive2.llvm.org/ce/z/qMwMhj

As noted in #60242, these patterns regressed between the
14.0 and 15.0 releases - probably due to a change in
canonicalization of related patterns.

The related patterns for testing if 2 values are both
pos/neg appear to be handled already.
2023-01-24 08:47:41 -05:00
Sanjay Patel
82dcfe0dbb [InstCombine] allow matching vector types for icmp-of-mask/cast
Also use a more specific matcher to simplify the mask
compare to type size.
2023-01-23 18:23:43 -05:00
Sanjay Patel
7a9e3ad070 [InstCombine] relax one-use check for icmp with mask/cast 2023-01-23 18:23:43 -05:00
Sanjay Patel
3e58d94780 [InstCombine] remove dead pattern matching code; NFC
Complexity canonicalization guarantees that a binop and cast
are op0/op1 respectively. Adjusted generic test names to
show that this pattern is still useful.
2023-01-23 18:23:43 -05:00
luxufan
0ad5909958 [InstCombine] Don't combine smul of i1 type constant one
Fixes: https://github.com/llvm/llvm-project/issues/59876

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D141214
2023-01-17 22:04:48 +08:00
Craig Topper
8e317e693a [InstCombine] Remove dead code from foldICmpShlOne. NFC
This code handles (icmp eq/ne (1 << Y), C) if C is a power of 2.

This case is also handled by the more general foldICmpShlConstConst
which is called before we reach foldICmpShlOne.
2023-01-15 19:10:17 -08:00
Craig Topper
77f2f34d69 [InstCombine] Generalize (icmp sgt (1 << Y), -1) -> (icmp ne Y, BitWidth-1) to any negative constant.
Similar for the sle version which will be canonicalized to slt first.

Alive2 proof as implemented: https://alive2.llvm.org/ce/z/_YawdM

@spatel's  original Alive2: https://alive2.llvm.org/ce/z/3YB2vs

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D141773
2023-01-15 13:36:57 -08:00
Craig Topper
ff39b7ea89 [InstCombine] Optimize (icmp slt (1 << Y), 1) -> (icmp eq Y, BitWidth-1).
The code tried to do this for (icmp sle (1 << Y), 0), but that is
canonicalized to sgt before we get there.

Simplify the code by removing the unreachable SGE and SLE handling.

Also remove the (1 << Y) >=u 2147483648 and (1 << Y) <u 2147483648
handling since those are canonicalized to (1 << Y) <s 0 and
(1 << Y) >=s 0 before we get there.

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D141753
2023-01-14 11:18:47 -08:00
Sanjay Patel
062415d3c8 [InstCombine] improve description of fold and add TODO; NFC
D58633
2023-01-13 10:34:55 -05:00
Guillaume Chatelet
8fd5558b29 [NFC] Use TypeSize::geFixedValue() instead of TypeSize::getFixedSize()
This change is one of a series to implement the discussion from
https://reviews.llvm.org/D141134.
2023-01-11 16:49:38 +00:00
Noah Goldstein
6d839621da [InstCombine] Canonicalize (A & B_Pow2) eq/ne B_Pow2 patterns
1. A & B_Pow2 != B_Pow2 -> A & B_Pow2 == 0
   https://alive2.llvm.org/ce/z/KVUej4

2. A & B_Pow2 == B_Pow2 -> A & B_Pow2 != 0
   https://alive2.llvm.org/ce/z/PVv9FR

This allows the patterns to more easily be analyzed elsewhere.

Differential Revision: https://reviews.llvm.org/D141090
2023-01-09 12:48:28 +01:00
Noah Goldstein
e6375ca6dc [InstCombine] Fix potentially buggy code in ((%x & C) == 0) --> %x u< (-C) transform
While demanded bits constant shrinking appears to prevent this in
practice right now, it is principally possible for C2 to have
set bits that are known not-needed (zeroable). See: D140858

`+` will overflow here, `|` will get the right logic.

Differential Revision: https://reviews.llvm.org/D141089
2023-01-09 11:44:11 +01:00
luxufan
eda8e999dd [InstCombine] Combine (zext a) mul (zext b) to llvm.umul.with.overflow only if mul has NUW flag
Fixes: https://github.com/llvm/llvm-project/issues/59836

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D141031
2023-01-08 14:41:59 +08:00
serge-sans-paille
38818b60c5
Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part
Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955
2023-01-05 14:11:08 +01:00
Nikita Popov
81ac46445b [InstCombine] Support vectors in icmp of GEP fold
EmitGEPOffset() supports vector GEPs nowadays, so we don't need
any further code changes.

compare_gep_with_base_vector1 shows a weakness in folding the
resulting comparison if an index splat has to be performed.
2023-01-02 15:29:13 +01:00
Nikita Popov
f7bc8e035d [InstCombine] Remove redundant evaluateGEPOffsetExpression() fold (NFCI)
If we go through the generic EmitGEPOffset code, the resulting
expression can be (and is) reduced in the same way this code did
manually. There are no changes in lit tests or llvm-test-suite.

This fold predates the time where we started adding nsw to the adds
created by EmitGEPOffset, so it was likely needed back then.

This might not actually be NFC due to worklist order changes etc.
2022-12-27 17:17:21 +01:00
Paul Walker
362c52ad5a [InstCombine] Bubble vector.reverse of compare operands to their result.
This mirrors a similar shufflevector transformation so the same
effect is obtained for scalable vectors. The transformation is
only performed when it can be proven the number of resulting
reversals is not increased. By bubbling the reversals from operand
to result this should typically be the case and ideally leads to
back-back shuffles that can be elimitated entirely.

Differential Revision: https://reviews.llvm.org/D139340
2022-12-21 15:53:14 +00:00
Matt Arsenault
191c1d95e8 APFloat: Add isSmallestNormalized predicate function
It was annoying to write the check for this in the one case I added,
and I'm planning on adding another, so add a convenient PatternMatch
like for other special case values.

I have no idea what is going on in the DoubleAPFloat case, I reversed
this from the makeSmallestNormalized test. Also could implement this
as *this == getSmallestNormalized() for less code, but this avoids the
construction of a temporary APFloat copy and follows the style of the
other functions.
2022-12-15 14:04:26 -05:00
Fangrui Song
d4b6fcb32e [Analysis] llvm::Optional => std::optional 2022-12-14 07:32:24 +00:00
Matt Arsenault
e9d298d7c6 InstCombine: Perform simpler test of APFloat first 2022-12-13 10:51:41 -05:00
Fangrui Song
21cd58baa1 [Transforms/InstCombine] llvm::Optional => std::optional 2022-12-13 08:26:08 +00:00
Kazu Hirata
343de6856e [Transforms] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 21:11:37 -08:00
Matt Arsenault
6463961941 InstCombine: Fold some identities for canonicalize
Equality is directly stated as true in the LangRef,
and I believe this works for every compare type.
2022-11-22 21:42:44 -05:00
Matt Arsenault
0f68ffe1e2 InstCombine: Fold compare with smallest normal if input denormals are flushed
Try to simplify comparisons with the smallest normalized value. If
denormals will be treated as 0, we can simplify by using an equality
comparison with 0.

fcmp olt fabs(x), smallest_normalized_number -> fcmp oeq x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0
fcmp oge fabs(x), smallest_normalized_number -> fcmp one x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0

The device libraries have a few range checks that look like
this for denormal handling paths.
2022-11-07 07:16:47 -08:00
Simon Pilgrim
09cb9fdef9 [InstCombine] Fold ult(add(x,-1),c) -> ule(x,c) iff x != 0 (PR57635)
Alive2: https://alive2.llvm.org/ce/z/sZ6wwS

As detailed on Issue #57635 and #37628 - for unsigned comparisons, we can compare prior to a decrement iff the value is known never to be zero.

Differential Revision: https://reviews.llvm.org/D134172
2022-09-20 16:44:41 +01:00
Sanjay Patel
6174da2299 [InstCombine] reduce code duplication in foldICmpMulConstant(); NFC 2022-09-16 10:39:54 -04:00
Sanjay Patel
02a27b3890 [InstCombine] fold X*X == 0 --> X == 0
This is safe when the mul does not overflow:
https://alive2.llvm.org/ce/z/LedVVP

This could be extended to handle non-zero compare constants
and non-squared multiplies.
2022-09-15 12:02:50 -04:00
Sanjay Patel
444f08c832 [InstCombine] fold icmp of truncated left shift, part 2
(trunc (1 << Y) to iN) == 2**C --> Y == C
(trunc (1 << Y) to iN) != 2**C --> Y != C
https://alive2.llvm.org/ce/z/xnFPo5

Follow-up to d9e1f9d7591b0d3e4d. This was a suggested
enhancement mentioned in issue #51889.
2022-09-08 12:44:02 -04:00
Sanjay Patel
d9e1f9d759 [InstCombine] Fold icmp of truncated left shift
(trunc (1 << Y) to iN) == 0 --> Y u>= N
(trunc (1 << Y) to iN) != 0 --> Y u<  N

These can be generalized in several ways as noted by the TODO
items, but this handles the pattern in the motivating bug report.

Fixes #51889

Differential Revision: https://reviews.llvm.org/D115480
2022-09-08 10:48:14 -04:00
Sanjay Patel
dd6eb4d67f [InstCombine] reduce code duplication; NFC 2022-09-06 08:19:30 -04:00
Tian Zhou
8fa432be4f [InstCombine] reduce test-for-overflow of shifted value
Fixes #57338.

The added code makes the following transformations:

For unsigned predicates / eq / ne:
icmp pred (x << 1), x --> icmp getSignedPredicate(pred) x, 0
icmp pred x, (x << 1) --> icmp getSignedPredicate(pred) 0, x

Some examples:
https://alive2.llvm.org/ce/z/ckn4cj
https://alive2.llvm.org/ce/z/h-4bAQ

Differential Revision: https://reviews.llvm.org/D132888
2022-09-05 09:51:51 -04:00
Sanjay Patel
c3d1504d63 [InstCombine] fix crash on type mismatch with fcmp fold
The existing predicate doesn't work for a single-element
vector, so make sure we are not crossing scalar/vector types.

Test (was crashing) based on the post-commit example for:
482777123427
2022-09-01 08:57:55 -04:00
Sanjay Patel
4827771234 [InstCombine] fold test of equality to 0.0 with bitcast operand
fcmp oeq/une (bitcast X), 0.0 --> (and X, SignMaskC) ==/!= 0
https://alive2.llvm.org/ce/z/ZKATGN
2022-08-26 13:46:11 -04:00
Jakub Kuderski
6fa87ec10f [ADT] Deprecate is_splat and replace all uses with all_equal
See the discussion thread for more details:
https://discourse.llvm.org/t/adt-is-splat-and-empty-ranges/64692

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D132335
2022-08-23 11:36:27 -04:00
Sanjay Patel
e5748c6e73 [InstCombine] reduce sub-with-overflow ==/!= 0
The basic patterns look like this:
https://alive2.llvm.org/ce/z/MDj9EC

The tests have a use of the overflow value too.
Otherwise, existing folds should reduce already.

This was noted as a missing IR fold in:
926e7312b2f20f2f7b

Hopefully, this makes it easier to implement a backend
fix because we should get the same IR regardless of
whether the source used builtins or inline code.
2022-08-15 13:03:51 -04:00
Fangrui Song
de9d80c1c5 [llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-08 11:24:15 -07:00
Sanjay Patel
7073ec530e [InstCombine] canonicalize more zext-and-of-bool compare to narrow and
https://alive2.llvm.org/ce/z/vBNiiM

This matches variants of patterns that were folded with:
b5a9361c90ca
2022-07-30 11:22:05 -04:00
Alexander Shaposhnikov
4220ef2be1 [InstCombine] Add fold for redundant sign bits count comparison
For power-of-2 C:
((X s>> ShiftC) ^ X) u< C --> (X + C) u< (C << 1)
((X s>> ShiftC) ^ X) u> (C - 1) --> (X + C) u> ((C << 1) - 1)

(https://github.com/llvm/llvm-project/issues/56479)

Test plan:
0/ ninja check-llvm check-clang + bootstrap LLVM/Clang
1/ https://alive2.llvm.org/ce/z/eEUfx3

Differential revision: https://reviews.llvm.org/D130433
2022-07-30 09:06:53 +00:00