411 Commits

Author SHA1 Message Date
Noah Goldstein
453d983d56 [InstCombine] Add transforms for (rem (shl Y, X), (shl Z, X))
This is just filling in a missing case from D144225.

We treat `(shl Y, X)` and `(shl Z, X)` as `(mul Z, 1 << X)` and `(mul
Y, 1 << X)` then reuse the same transformations that already exist.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D147108
2023-07-06 14:46:34 -05:00
Noah Goldstein
2cb6b06c89 [InstCombine] Add constant combines for (urem/srem (shl X, Y), (shl X, Z))
Forked from D142901 to deduce more `nsw`/`nuw` flag for the output
`shl`.

We can handle the following cases + some `nsw`/`nuw` flags:

The rationale for doing this all in `InstCombine` rather than handling
the constant `shl` cases in `InstSimplify` is we often create a new
instruction because we are able to deduce more `nsw`/`nuw` flags than
the original instruction had.

Differential Revision: https://reviews.llvm.org/D144225
2023-07-06 14:46:34 -05:00
Zhongyunde
90d30fde12 [InstCombine] Add frozen for the condition value of SelectInst
If the condition value of SelectInst may be a poison or undef value,
infer constant range at SelectInst use is incorrect, similar to D143883.
Fixes https://github.com/llvm/llvm-project/issues/62401

Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D149339
2023-04-27 21:35:54 +08:00
Noah Goldstein
bfe2f5d38b [InstCombine] Fix buggy (mul X, Y) -> (shl X, Log2(Y)) transform PR62175
Bug was because we recognized patterns like `(shl 4, Z)` as a power of
2 we could take Log2 of (`2 + Z`), but doing `(shl X, (2 + Z))` can
cause a poison shift.
    https://alive2.llvm.org/ce/z/yuJm_k

The fix is to verify that `Log2(Y)` will be a non-poisonous shift
amount. We can do this with:
    `nsw` flag:
        - https://alive2.llvm.org/ce/z/yyyJBr
        - https://alive2.llvm.org/ce/z/YgubD_
    `nuw` flag:
        - https://alive2.llvm.org/ce/z/-4mpyV
        - https://alive2.llvm.org/ce/z/a6ik6r
    Prove `Y != 0`:
        - https://alive2.llvm.org/ce/z/ced4su
        - https://alive2.llvm.org/ce/z/X-JJHb

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D148609
2023-04-18 17:17:48 -05:00
Noah Goldstein
513251b765 [InstCombine] Improve transforms for (mul X, Y) -> (shl X, log2(Y)
Using the more robust log2 search allows us to fold more cases (same
logic as exists for idiv/irem).

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D146347
2023-04-07 14:58:20 -05:00
Serguei Katkov
f38365aef4 [InstCombine] Add support for maximum(a,b) + minimum(a,b) => a + b
Unfortunately alive2 cannot prove the correctness due to fails by timeout even for
float type half.

However it should be correct. If a and b are not NaN, maximum and minimum will just
return different values (a and b) and take into account a + b == b + a this is the same.
If a or b is NaN, than maximum and minimum are equal to NaN and NaN + NaN is NaN.
a + b is also a NaN.

In terms of preserving fast flags, we cannot preserve ninf due to
minimum(NaN, Infinity) == maximum(NaN, Infinity) == NaN,
minimum(NaN, Infinity) +ninf maximum(NaN, Infinity) == NaN +ninf NaN = NaN
However transformation will change
minimum(NaN, Infinity) + maximum(NaN, Infinity) to NaN +ninf Infinity == poison.

But if fadd is marked as nnan, we can preserve because NaN +ninf/nnan NaN = poison as well.

The same optimization for
  maximum(a,b) * minimum(a,b) => a * b
is added.
All said above for fadd is correct for fmul.

Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D147299
2023-04-07 12:38:04 +07:00
Serguei Katkov
624973806c [InstCombine] Add support for max(a,b) + min(a,b) => a + b. Re-land.
The same optimization for
  max(a,b) * min(a,b) => a * b
is added.

Correctness check:
uadd: https://alive2.llvm.org/ce/z/2rXDek
sadd: https://alive2.llvm.org/ce/z/zNu_er
uadd + nuw/nsw: https://alive2.llvm.org/ce/z/EaiNjB
sadd + nuw/nsw: https://alive2.llvm.org/ce/z/w_2Nrs

umul: https://alive2.llvm.org/ce/z/dgXRLr
smul: https://alive2.llvm.org/ce/z/hBjGzz
umul + nuw/nsw: https://alive2.llvm.org/ce/z/EaiNjB
smul + nuw/nsw: https://alive2.llvm.org/ce/z/87MNeS

Reviewed By: goldstein.w.n
Differential Revision: https://reviews.llvm.org/D147296
2023-04-07 11:56:05 +07:00
Serguei Katkov
4665f3c838 Revert "[InstCombine] Add support for max(a,b) + min(a,b) => a + b."
Revert commit due to failure on buildbot:
error: 'match_combine_or' may not intend to support class template argument deduction

This reverts commit b86a06ef284f2637bef89bf5bb20157a8b195568.
2023-04-07 11:14:28 +07:00
Serguei Katkov
b86a06ef28 [InstCombine] Add support for max(a,b) + min(a,b) => a + b.
The same optimization for
  max(a,b) * min(a,b) => a * b
is added.

Correctness check:
uadd: https://alive2.llvm.org/ce/z/2rXDek
sadd: https://alive2.llvm.org/ce/z/zNu_er
uadd + nuw/nsw: https://alive2.llvm.org/ce/z/EaiNjB
sadd + nuw/nsw: https://alive2.llvm.org/ce/z/w_2Nrs

umul: https://alive2.llvm.org/ce/z/dgXRLr
smul: https://alive2.llvm.org/ce/z/hBjGzz
umul + nuw/nsw: https://alive2.llvm.org/ce/z/EaiNjB
smul + nuw/nsw: https://alive2.llvm.org/ce/z/87MNeS

Reviewed By: goldstein.w.n
Differential Revision: https://reviews.llvm.org/D147296
2023-04-07 10:24:07 +07:00
Noah Goldstein
aba71f37d0 [InstCombine] Add constant combines for (urem/srem (mul X, Y), (mul X, Z))
We can handle the following cases + some `nsw`/`nuw` flags:

`(srem (mul X, Y), (mul X, Z))`
    [If `srem(Y, Z) == 0`]
        -> 0
            - https://alive2.llvm.org/ce/z/PW4XZ-
    [If `srem(Y, Z) == Y`]
        -> `(mul nuw nsw X, Y)`
            - https://alive2.llvm.org/ce/z/DQe9Ek
        -> `(mul nsw X, Y)`
            - https://alive2.llvm.org/ce/z/Nr_MdH

    [If `Y`/`Z` are constant]
        -> `(mul/shl nuw nsw X, (srem Y, Z))`
            - https://alive2.llvm.org/ce/z/ccTFj2
            - https://alive2.llvm.org/ce/z/i_UQ5A
        -> `(mul/shl nsw X, (srem Y, Z))`
            - https://alive2.llvm.org/ce/z/mQKc63
            - https://alive2.llvm.org/ce/z/uERkKH

`(urem (mul X, Y), (mul X, Z))`
    [If `urem(Y, Z) == 0`]
        -> 0
            - https://alive2.llvm.org/ce/z/LL7UVR
    [If `srem(Y, Z) == Y`]
        -> `(mul nuw nsw X, Y)`
            - https://alive2.llvm.org/ce/z/9Kgs_i
        -> `(mul nuw X, Y)`
            - https://alive2.llvm.org/ce/z/ow9i8u

    [If `Y`/`Z` are constant]
        -> `(mul nuw nsw X, (srem Y, Z))`
            - https://alive2.llvm.org/ce/z/mNnQqJ
            - https://alive2.llvm.org/ce/z/Bj_DR-
            - https://alive2.llvm.org/ce/z/X6ZEtQ
        -> `(mul nuw X, (srem Y, Z))`
            - https://alive2.llvm.org/ce/z/SJYtUV

The rationale for doing this all in `InstCombine` rather than handling
the constant `mul` cases in `InstSimplify` is we often create a new
instruction because we are able to deduce more `nsw`/`nuw` flags than
the original instruction had.

Reviewed By: MattDevereau, sdesmalen

Differential Revision: https://reviews.llvm.org/D143014
2023-03-16 13:01:46 -05:00
Sanjay Patel
703423c269 [InstCombine] relax constraint on udiv fold
The pair of div folds was just added with:
4966d8ebe1bbe5bd6a4d28

But as noted in the post-commit review, we don't actually need
the no-remainder requirement for an unsigned division (still
need the no-unsigned-wrap though):
https://alive2.llvm.org/ce/z/qHjK3Q
2023-02-20 15:08:32 -05:00
Sanjay Patel
4966d8ebe1 [InstCombine] distribute div over add with matching mul-by-constant
((X * C2) + C1) / C2 --> X + C1/C2
https://alive2.llvm.org/ce/z/P66io8
https://alive2.llvm.org/ce/z/vghegw

This could be made more general -- the multiplier could be a
multiple of the divisor -- but this is the pattern from
issue #60754.
2023-02-20 13:45:06 -05:00
zhongyunde
ee9a0f30ca [InstCombine] canonicalize urem as cmp+select
Fix https://github.com/llvm/llvm-project/issues/60546

Reviewed By: nikic, efriedma, RKSimon, spatel

Differential Revision: https://reviews.llvm.org/D143883
2023-02-20 23:52:10 +08:00
Kazu Hirata
a28b252d85 Use APInt::getSignificantBits instead of APInt::getMinSignedBits (NFC)
Note that getMinSignedBits has been soft-deprecated in favor of
getSignificantBits.
2023-02-19 23:56:52 -08:00
Kazu Hirata
f8f3db2756 Use APInt::count{l,r}_{zero,one} (NFC) 2023-02-19 22:04:47 -08:00
Sanjay Patel
1378e7d8b8 [InstSimplify] add no-wrap parameters to simplifyMul and add more tests; NFC
This gives mul the same capabilities as add/sub.
A potential improvement with nsw was noted in:
1720ec6da040729f17
2023-01-18 13:29:30 -05:00
Sanjay Patel
072b03c471 [InstCombine] fold pow(X,Y) / X -> pow(X, Y-1)
This is one of the patterns suggested in issue #34943.
2023-01-13 17:13:46 -05:00
Sanjay Patel
61af2ab681 [InstCombine] fold pow(X,Y) * X -> pow(X, Y+1) (with fast-math)
This is one of the patterns suggested in issue #34943.
2023-01-13 17:13:46 -05:00
Sanjay Patel
914576c1f0 [InstCombine] fold pow(X,Y) * pow(Z,Y) -> pow(X*Z, Y) (with fast-math)
This is one of the patterns suggested in issue #34943.
2023-01-13 13:26:10 -05:00
Sanjay Patel
f0faea5714 [InstSimplify] fold exact divide to poison if it is known to not divide evenly
This is related to the discussion in D140665. I was looking over the demanded
bits implementation in IR and noticed that we just bail out of a potential
fold if a udiv is exact:
82be8a1d2b/llvm/lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp (L799)

Also, see tests added with 7f0c11509e8f.

Then, I saw that we could lose a fold to poison if we zap the exact with that
transform, so this patch tries to catch that as a preliminary step.

Alive2 proofs:
https://alive2.llvm.org/ce/z/zCjKM7
https://alive2.llvm.org/ce/z/-tz_RK (trailing zeros must be "less-than")
https://alive2.llvm.org/ce/z/c9CMsJ (general proof and specific example)

Differential Revision: https://reviews.llvm.org/D140733
2022-12-29 10:26:50 -05:00
Chenbing Zheng
1f84e72b7b [InstCombine] Fold (X << Z) / (X * Y) -> (1 << Z) / Y
Alive2: https://alive2.llvm.org/ce/z/CBJLeP
2022-12-29 17:30:49 +08:00
Chenbing Zheng
bff1f8c79b [InstCombine] complete (X << Z) / (Y << Z) --> X / Y
Add one more situations for this fold.
For unsigned div, 'nsw' on both shifts + 'nuw' on the dividend.

Alive2: https://alive2.llvm.org/ce/z/sELF76

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D139997
2022-12-23 11:56:52 +08:00
Matt Arsenault
e661185fb3 InstCombine: Fold fdiv nnan x, 0 -> copysign(inf, x)
https://alive2.llvm.org/ce/z/gLBFKB
2022-11-07 22:00:15 -08:00
Sanjay Patel
f03b069c5b [InstCombine] fold mul with decremented "shl -1" factor (2nd try)
This is a corrected version of:
bc886e9b587b

I made a copy-paste error that created an "add" instead of the
intended "sub" on that attempt. The regression tests showed the
bug, but I overlooked that.

As I said in a comment on issue #58717, the bug reports resulting
from the botched patch confirm that the pattern does occur in
many real-world applications, so hopefully eliminating the multiply
results in better code.

I added one more regression test in this version of the patch,
and here's an Alive2 proof to show that exact example:
https://alive2.llvm.org/ce/z/dge7VC

Original commit message:

This is a sibling to:
6064e92b0a84
...but we canonicalize the shl+add to shl+xor,
so the pattern is different than I expected:
https://alive2.llvm.org/ce/z/8CX16e

I have not found any patterns that are safe
to propagate no-wrap, so that is not included
here.

Differential Revision: https://reviews.llvm.org/D137157
2022-11-02 09:30:01 -04:00
Florian Mayer
e1de7ac20f Revert "[InstCombine] fold mul with decremented "shl -1" factor"
This reverts commit bc886e9b587b9d009f49b12eaaa9ebc1c71905a1.

Broke MSAN bootstrap buildbots with Assertion `RangeAfterCopy % ExtraScale == 0 && "Extra instruction requires immediate to be aligned"' failed.
2022-10-31 17:39:05 -07:00
Sanjay Patel
bc886e9b58 [InstCombine] fold mul with decremented "shl -1" factor
This is a sibling to:
6064e92b0a84
...but we canonicalize the shl+add to shl+xor,
so the pattern is different than I expected:
https://alive2.llvm.org/ce/z/8CX16e

I have not found any patterns that are safe
to propagate no-wrap, so that is not included
here.
2022-10-31 09:06:55 -04:00
zhongyunde
f58311796c [InstCombine] refactor the SimplifyUsingDistributiveLaws NFC
Precommit for D136015
Reviewed By: spatel
Differential Revision: https://reviews.llvm.org/D137019
2022-10-30 21:04:06 +08:00
Sanjay Patel
6064e92b0a [InstCombine] fold mul with incremented "shl 1" factor
X * ((1 << Z) + 1) --> (X << Z) + X

https://alive2.llvm.org/ce/z/P-7WK9

It's possible that we could do better with propagating
no-wrap, but this carries over the existing logic and
appears to be correct.

The naming differences on the existing folds are a result
of using getName() to set the final value via Builder.
That makes it easier to transfer no-wrap rather than the
gymnastics required from the raw create instruction APIs.
2022-10-29 12:50:19 -04:00
Sanjay Patel
50000ec2cb [InstCombine] create helper function for mul patterns with 1<<X; NFC
There are at least 2 other potential patterns that could go here.
2022-10-29 12:50:19 -04:00
Sanjay Patel
d344146857 [InstCombine] reduce code duplication in visitMul(); NFC 2022-10-29 09:26:02 -04:00
Sanjay Patel
44b7da89d7 [InstCombine] fmul nnan X, 0.0 --> copysign(0.0, X)
https://alive2.llvm.org/ce/z/ybgM5F

Differential Revision: https://reviews.llvm.org/D136166
2022-10-18 11:34:02 -04:00
Sanjay Patel
e5ee0b06d6 [InstCombine] try to determine "exact" for sdiv
If the divisor is a power-of-2 or negative-power-of-2 and the dividend
is known to have >= trailing zeros than the divisor, the division is exact:
https://alive2.llvm.org/ce/z/UGBksM (general proof)
https://alive2.llvm.org/ce/z/D4yPS- (examples based on regression tests)

This isn't the most direct optimization (we could create ashr in these
examples instead of relying on existing folds for exact divides), but
it's possible that there's a more general constraint than just a pow2
divisor, so this might be extended in the future.

This should solve issue #58348.

Differential Revision: https://reviews.llvm.org/D135970
2022-10-16 10:59:56 -04:00
Sanjay Patel
340ae45be0 [InstCombine] use isKnownNonNegative() for readability; NFCI
This should be functionally equivalent - both calls are thin
wrappers around computeKnownBits(). We'll probably want to use
known-bits directly in follow-up patches because that could
determine "exact" for example (see issue #58348).
2022-10-16 10:59:56 -04:00
Sanjay Patel
7b9482df3d [InstCombine] fold sdiv with common shl amount in operands
(X << Z) / (Y << Z) --> X / Y

https://alive2.llvm.org/ce/z/CLKzqT

This requires a surprising "nuw" constraint because we have
to guard against immediate UB via signed-div overflow with
-1 divisor.

This extends 008a89037a49ca0d9 and is another transform
derived from issue #58137.
2022-10-12 11:32:15 -04:00
Sanjay Patel
008a89037a [InstCombine] fold udiv with common shl amount in operands
(X << Z) / (Y << Z) --> X / Y

https://alive2.llvm.org/ce/z/E5eaxU

This fixes the motivating example from issue #58137,
but it is not the most general transform. We should
probably also convert left-shift in the divisor to
right-shift in the dividend for that, but that exposes
another missed canonicalization for shifts and adds.
2022-10-12 11:12:26 -04:00
Sanjay Patel
fe97f95036 [InstCombine] propagate "exact" through folds of div
These folds were added recently with:
6b869be8100d
8da2fa856f1b
...but they didn't account for the "exact" attribute,
and that can be safely propagated:
https://alive2.llvm.org/ce/z/F_WhnR
https://alive2.llvm.org/ce/z/ft9Cgr
2022-10-12 09:25:05 -04:00
Sanjay Patel
d117ee25b8 [InstCombine] add helper function for div+shl folds; NFC
There are at least 2 similar patterns that could be added here,
and the existing fold can be improved because it fails to
propagate "exact".
2022-10-12 09:25:04 -04:00
Sanjay Patel
7ec604a317 [InstCombine] try harder to cancel out mul/div
((Op1 * X) / Y) / Op1 --> X / Y
https://alive2.llvm.org/ce/z/JYxWjA

InstSimplify handles the more basic mul+div pattern with
shared operand, but we don't seem to have any reassociation
folds to handle cases where the common op is further away.

This is a generalization of 9cff4711ac72 and another
transform derived from issue #58137.
2022-10-11 09:51:51 -04:00
Sanjay Patel
9cff4711ac [InstCombine] fold udiv with common factor
((X *nuw Y) >> Z) / X --> Y >> Z

https://alive2.llvm.org/ce/z/x3kKnq

This is similar to 6b869be8100d / 8da2fa856f1b, but I have
not found a signed equivalent, so it's just an unsigned
match for now.
2022-10-10 08:12:06 -04:00
Sanjay Patel
eccb9a77c6 [InstCombine] fold exact sdiv to ashr (2nd try)
The 1st attempt failed to updated the test checks as expected.

Original commit message:

sdiv exact X, (1<<ShAmt) --> ashr exact X, ShAmt (if shl is non-negative)

https://alive2.llvm.org/ce/z/kB6VF7

It would probably be better to use ValueTracking to replace this
and the existing transform above it, but the analysis does not
account for the no-wrap properly, and it's not immediately clear
to me how to fix it.
2022-10-08 10:09:44 -04:00
Sanjay Patel
68d4dbc2c1 Revert "[InstCombine] fold exact sdiv to ashr"
This reverts commit fe15290e0cf5d2bcdefca2e81ef6ff8155a2f7a8.
The test checks were not updated as expected.
2022-10-08 10:02:03 -04:00
Sanjay Patel
fe15290e0c [InstCombine] fold exact sdiv to ashr
sdiv exact X, (1<<ShAmt) --> ashr exact X, ShAmt (if shl is non-negative)

https://alive2.llvm.org/ce/z/kB6VF7

It would probably be better to use ValueTracking to replace this
and the existing transform above it, but the analysis does not
account for the no-wrap properly, and it's not immediately clear
to me how to fix it.
2022-10-08 09:23:46 -04:00
Sanjay Patel
bdfefac9a4 [InstCombine] refactor sdiv by (negative) power-of-2 folds; NFCI
It's probably better to try harder on this kind of
pattern by using ValueTracking.
2022-10-07 11:35:17 -04:00
Sanjay Patel
8da2fa856f [InstCombine] fold sdiv with hidden common factor
(X * Y) s/ (X << Z) --> Y s/ (1 << Z)

https://alive2.llvm.org/ce/z/yRSddG

issue #58137
2022-10-06 13:11:50 -04:00
Sanjay Patel
6b869be810 [InstCombine] fold udiv with hidden common factor
(X * Y) u/ (X << Z) --> Y u>> Z

https://alive2.llvm.org/ce/z/4G9D_W
2022-10-06 11:35:27 -04:00
Sanjay Patel
2e87333bfe [InstCombine] convert mul by negative-pow2 to negate and shift
This is an unusual canonicalization because we create an extra instruction,
but it's likely better for analysis and codegen (similar reasoning as D133399).

InstCombine::Negator may create this kind of multiply from negate and shift,
but this should not conflict because of the narrow negation.

I don't know how to create a fully general proof for this kind of transform in
Alive2, but here's an example with bitwidths similar to one of the regression
tests:
https://alive2.llvm.org/ce/z/J3jTjR

Differential Revision: https://reviews.llvm.org/D133667
2022-10-02 12:22:25 -04:00
zhongyunde
23a5de4294 [InstCombine] Distributive or+mul with const operand
We aleady support the transform: `(X+C1)*CI -> X*CI+C1*CI`
Here the case is a little special as the form of `(X+C1)*CI` is transformed into `(X|C1)*CI`,
so we should also support the transform: `(X|C1)*CI -> X*CI+C1*CI`
Fixes https://github.com/llvm/llvm-project/issues/57278

Reviewed By: bcl5980, spatel, RKSimon
Differential Revision: https://reviews.llvm.org/D132658
2022-08-30 20:36:52 +08:00
zhongyunde
84d6966e4d [InstCombine] Propagate the nuw for combine of add+mul
As the commit of D132658, make the 'nuw' change separately.
Reviewed By: spatel
Differential Revision: https://reviews.llvm.org/D132777
2022-08-28 23:01:11 +08:00
Zain Jaffal
f61f99a105
[instcombine] Optimise for zero initialisation of product given fast flags are enabled
Currently, clang ignores the 0 initialisation in finite math
For example:

```
double f_prod = 0;
double arr[1000];
for (size_t i = 0; i < 1000; i++) {
  f_prod *= arr[i];
 }
```
Clang will ignore that `f_prod` is set to zero and it will generate assembly to iterate over the loop.

Reviewed By: fhahn, spatel

Differential Revision: https://reviews.llvm.org/D131672
2022-08-17 11:12:15 +01:00
Sanjay Patel
f95a6aea1b [InstCombine] avoid splitting a constant expression with div/rem fold
Follow-up to d4940c0f3d43 to further limit the transform
to avoid an unintended pattern/fold of a constant expression.
2022-07-30 09:45:25 -04:00