7633 Commits

Author SHA1 Message Date
Noah Goldstein
9a8f517f57 [ValueTracking] Add KnownBits patterns xor(x, x - 1) and and(x, -x) for knowing upper bits to be zero
These two BMI pattern will clear the upper bits of result past the
first set bit. So if we know a single bit in `x` is set, we know that
`results[bitwidth - 1, log2(x) + 1] = 0`.

Alive2:
blsmsk: https://alive2.llvm.org/ce/z/a397BS
blsi: https://alive2.llvm.org/ce/z/tsbQhC

Differential Revision: https://reviews.llvm.org/D142271
2023-02-18 13:31:17 -06:00
Sanjay Patel
a8831631c7 [InstCombine] add tests for 1<<cttz(x); NFC
issue #60799
issue #60801
2023-02-18 08:34:55 -05:00
Sanjay Patel
4ecc6af813 [InstCombine] create a pass options container and add "use-loop-info" argument
This is a cleanup/modernization patch requested in D144045 to make loop
analysis a proper optional parameter to the pass rather than a
semi-arbitrary value inherited via the pass pipeline.

It's a bit more complicated than the recent patch I started copying from
(D143980) because InstCombine already has an option for MaxIterations
(added with D71145).

I debated just deleting that option, but it was used by a pair of existing
tests, so I put it into a struct (code largely copied from SimplifyCFG's
implementation) to make the code more flexible for future options
enhancements.

I didn't alter the pass manager invocations of InstCombine in this patch
because the patch was already getting big, but that will be a small
follow-up as noted with the TODO comment.

Differential Revision: https://reviews.llvm.org/D144199
2023-02-17 10:30:15 -05:00
Nick Desaulniers
45a291b5f6 [Dominators] check indirect branches of callbr
This will be necessary to support outputs from asm goto along indirect
edges.

Test via:
  $ pushd llvm/build; ninja IRTests; popd
  $ ./llvm/build/unittests/IR/IRTests \
    --gtest_filter=DominatorTree.CallBrDomination

Also, return nullptr in Instruction::getInsertionPointAfterDef for
CallBrInst as was recommened in
https://reviews.llvm.org/D135997#3991427.  The following phab review was
folded into this commit: https://reviews.llvm.org/D140166

Link: Link: https://discourse.llvm.org/t/rfc-syncing-asm-goto-with-outputs-with-gcc/65453/8

Reviewed By: void, efriedma, ChuanqiXu, MaskRay

Differential Revision: https://reviews.llvm.org/D135997
2023-02-16 17:58:33 -08:00
Nikita Popov
c9fad20f6a [InstCombine] Call simplifyInsertValueInst()
InstCombine is supposed to be a superset of InstSimplify, but we
were not attempting simplification of insertvalue instructions.
As the test change illustrates, we failed to remove some aggregate
construction patterns because of that.
2023-02-16 09:51:40 +01:00
Nikita Popov
6bec2c3077 [InstCombine] Add additional aggregate reconstruction test (NFC)
This is like test2 from the same file, but using poison instead of
undef as base, which matches the IR we use nowadays.
2023-02-16 09:47:44 +01:00
Sanjay Patel
50ef8673a3 [InstCombine] remove stale test comment; NFC 2023-02-15 11:43:35 -05:00
Matt Arsenault
458ad6900e Revert "InstCombine: Fold is.fpclass(x, fcZero) to fcmp oeq 0"
This reverts commit df78976d023a6b7fcf64bc695261b7b402fcede0.

I pushed the wrong patch
2023-02-15 05:27:26 -04:00
Matt Arsenault
df78976d02 InstCombine: Fold is.fpclass(x, fcZero) to fcmp oeq 0
This requires the denormal mode to definitively be IEEE handling.
2023-02-15 05:21:00 -04:00
Craig Topper
2872987e5e [InstCombine] Fix InstCombinerImpl::foldICmpMulConstant for nsw and nuw mul with unsigned compare.
If we have both an nsw and nuw flag, we would see the nsw flag
first and only handle signed comparisons.

This patch ignores the nsw flag if the comparison isn't signed.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D143766
2023-02-14 23:43:17 -08:00
chenglin.bi
dd31a3b3a5 [InstCombine] fold icmp of the sum of ext bool based on limited range
For the pattern `(zext i1 X) + (sext i1 Y)`, the constant range is [-1, 1].
We can simplify the pattern by logical operations. Like:

```
    (zext i1 X) + (sext i1 Y) == -1 -->  ~X & Y
    (zext i1 X) + (sext i1 Y) == 0  --> ~(X ^ Y)
    (zext i1 X) + (sext i1 Y) == 1 --> X & ~Y
```
And other predicates can the combination of these results:

```
    (zext i1 X) + (sext i1 Y)) != -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) s> -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) u< -1 --> X | ~Y
    (zext i1 X) + (sext i1 Y)) s> 0 --> X & ~Y
    (zext i1 X) + (sext i1 Y)) s< 0 --> ~X & Y
    (zext i1 X) + (sext i1 Y)) != 1 --> ~X | Y
    (zext i1 X) + (sext i1 Y)) s< 1 --> ~X | Y
    (zext i1 X) + (sext i1 Y)) u> 1 --> ~X & Y
```

All alive proofs:
https://alive2.llvm.org/ce/z/KmgDpF
https://alive2.llvm.org/ce/z/fLwWa9
https://alive2.llvm.org/ce/z/ZKQn2P

Fix: https://github.com/llvm/llvm-project/issues/59666

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143373
2023-02-15 10:34:00 +08:00
chenglin.bi
f93da3962e [Instcombine] Precommit tests for icmp range; NFC 2023-02-15 10:31:34 +08:00
Sanjay Patel
4a4e44e378 [InstCombine] add test for loop-invariant fdiv; NFC
Adding baseline test based on D87479, so we know
current state and limitations of a potential fix.
2023-02-14 17:51:06 -05:00
Sanjay Patel
f48f178717 [InstCombine] canonicalize cmp+select as smin/smax
(V == SMIN) ? SMIN+1 : V --> smax(V, SMIN+1)
(V == SMAX) ? SMAX-1 : V --> smin(V, SMAX-1)

https://alive2.llvm.org/ce/z/d5bqjy

Follow-up for the unsigned variants added with:
86b4d8645fc1b866

issue #60374
2023-02-12 07:54:43 -05:00
Sanjay Patel
14411adc3d [InstCombine] add tests for disguised smin/smax; NFC
Adapted from the existing umin/umax tests.
2023-02-12 07:54:43 -05:00
Craig Topper
db89896bbb [InstCombine] Add tests for D143766. NFC 2023-02-10 12:43:31 -08:00
ManuelJBrito
83e84cdbe4 [InstCombine] precommit tests for D143593 2023-02-10 19:09:52 +00:00
Sanjay Patel
78056e2f2d [InstCombine] propagate FMF in exp2->ldexp fold 2023-02-10 10:02:25 -05:00
Sanjay Patel
3abea2b544 [InstCombine] copy tail markings in exp2->ldexp fold 2023-02-10 10:02:25 -05:00
Sanjay Patel
e89180801c [InstCombine] add 'tail' to exp2 call in test; NFC
Tail markings are not propagated if the transform succeeds.
2023-02-10 10:02:14 -05:00
Sanjay Patel
9dcd7195a2 [InstCombine] avoid crashing in pow->ldexp
Similar to 62a0a1b9eea7788c1f9dbae -

We have pow math intrinsics in IR, but no ldexp intrinsics
to handle vector types.

A patch for that was proposed in D14327, but it was not completed.

Issue #60605
2023-02-10 08:03:13 -05:00
Sanjay Patel
62a0a1b9ee [InstCombine] avoid crashing in exp2->ldexp
We have exp2 math intrinsics in IR, but no ldexp intrinsics
to handle vector types.

A patch for that was proposed in D14327, but it was not completed.

Issue #60605
2023-02-10 07:35:39 -05:00
Sanjay Patel
c2b7085d5c [InstCombine] add test for exp2->ldexp; NFC 2023-02-10 07:35:25 -05:00
Sanjay Patel
55c0719557 [InstCombine] auto-generate check lines in test file; NFC 2023-02-10 07:35:25 -05:00
Yingchi Long
68adc028b4 [InstCombine][NFC] regenerate tests for simple_phi_condition.ll 2023-02-10 19:29:17 +08:00
Yingchi Long
f9e2fb9d8e [InstCombine] combine intersection for inequality icmps
```
define i1 @src(i32 %A) {
  %mask1 = and i32 %A, 15 ; 0x0f
  %tst1 = icmp eq i32 %mask1, 3 ; 0x03
  %mask2 = and i32 %A, 255 ; 0xff
  %tst2 = icmp eq i32 %mask2, 243; 0xf3
  %res = or i1 %tst1, %tst2
  ret i1 %res
}
```

->

```
define i1 @tgt(i32 %A) {
  %1 = and i32 %A, 15
  %res = icmp eq i32 %1, 3
  ret i1 %res
}
```

Proof: https://alive2.llvm.org/ce/z/4AyvcE

Assume that `(B & D) & (C ^ E) == 0`, and `(B & D) == D || (B & D) == B`,
transforms:

```
(icmp ne (A & B), C) & (icmp ne (A & D), E)
-> (icmp ne (A & (B&D)), (C&E))
```

Fixes: https://github.com/llvm/llvm-project/issues/59680

Reviewed By: spatel, bcl5980

Differential Revision: https://reviews.llvm.org/D140666
2023-02-10 12:50:39 +08:00
Yingchi Long
942c4cdac4 [InstCombine] precommit tests for D140666
Differential Revision: https://reviews.llvm.org/D142090
2023-02-10 12:48:32 +08:00
chenglin.bi
6f149a17d4 [InstCombine] Look through truncate to fold icmp with intrinsics
The output of intrinsic functions like ctpop, cttz, ctlz have limited range from 0 to bitwidth. So if the truncate destination type can hold the source bitwidth size, we can just ignore the truncate and use the truncate src to do combination.

Alive2 proofs:
https://alive2.llvm.org/ce/z/9D_-qP

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143368
2023-02-10 10:33:07 +08:00
chenglin.bi
e09c76fe2a [Instcombine] Precommit tests update for icmp(trunc cttz/ctlz(x), C); NFC 2023-02-09 13:03:50 +08:00
Matt Arsenault
0b5c51b040 InstCombine: Add some additional is.fpclass tests
Test some more cases related to compare with 0 and inf.
2023-02-08 22:19:20 -04:00
Matt Arsenault
9ad6bdd747 InstCombine: Fold and (fcmp), (is.fpclass) into is.fpclass
Fold class test performed by an fcmp into another class. For now this
avoids introducing new class calls then there isn't one that already
exists.
2023-02-08 21:40:20 -04:00
Sanjay Patel
86b4d8645f [InstCombine] canonicalize cmp+select as umin/umax
(V == 0) ? 1 : V --> umax(V, 1)
(V == UMAX) ? UMAX-1 : V --> umin(V, UMAX-1)

https://alive2.llvm.org/ce/z/pfDBAf

This is one pair of the variants discussed in issue #60374.

Enhancements for the other end of the constant range and
signed variants are potential follow-ups, but that may
require more work because we canonicalize at least one
min/max like that to icmp+zext.
2023-02-08 17:25:58 -05:00
Sanjay Patel
b98813fbb6 [InstCombine] add tests for cmp+select; NFC 2023-02-08 17:25:58 -05:00
Krzysztof Drewniak
2d279c0d95 [llvm] Add tests for upcoming fixes to pointer/index type confusion.
Various parts of the codebase are using getIntPtrType() and its
relatives when getting the type of the offset argument to GEP. Most
such code has been updated to use the pointer index type field from
the data layout, but there is code that still assumes these two types
are the same in certain optimizaiton passes.

This commit adds regression tests to capture the old behavior.

Reviewed By: #amdgpu, arsenm

Differential Revision: https://reviews.llvm.org/D143435
2023-02-07 16:06:58 +00:00
chenglin.bi
1c345c1873 [Instcombine] Precommit tests for D143373; NFC 2023-02-07 12:08:43 +08:00
Noah Goldstein
abbd256a81 Improve transforms for (icmp uPred X * Z, Y * Z) -> (icmp uPred X, Y)
Several cases where missing.

1. `(icmp eq/ne X*Z, Y*Z) [if Z % 2 != 0] -> (icmp eq/ne X, Y)`
    EQ: https://alive2.llvm.org/ce/z/6_HPZ5
    NE: https://alive2.llvm.org/ce/z/c34qSU

    There was previously an implementation of this that work of `Y`
    was non-constant, but it was missing if `Y*Z` evaluated to a
    constant and/or `nsw`/`nuw` where both false. As well it only
    worked if `Z` was a constant but we can check 1s bit of
    `KnownBits` to cover more cases.

2. `(icmp eq/ne X*Z, Y*Z) [if Z != 0 and nsw(X*Y) and nsw(Y*Z)] -> (icmp eq/ne X, Y)`
    EQ: https://alive2.llvm.org/ce/z/6SdAG6
    NE: https://alive2.llvm.org/ce/z/fjsq_b

    This was previously implemented only to work if `Z` was constant,
    but we can use `isKnownNonZero` to cover more cases.

3. `(icmp uPred X*Y, Y*Z) [if Z != 0 and nuw(X*Y) and nuw(X*Y)] -> (icmp uPred X, Y)`
    EQ:  https://alive2.llvm.org/ce/z/FqWQLX
    NE:  https://alive2.llvm.org/ce/z/2gHrd2
    ULT: https://alive2.llvm.org/ce/z/MUAWgZ
    ULE: https://alive2.llvm.org/ce/z/szQQ2L
    UGT: https://alive2.llvm.org/ce/z/McVUdu
    UGE: https://alive2.llvm.org/ce/z/95uyC8

    This was previously implemented only for `eq/ne` cases. As well
    only if `Z` was constant, but again we can use `isKnownNonZero` to
    cover more cases.

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D142786
2023-02-06 14:09:18 -06:00
Noah Goldstein
2a3732f934 Add transform for (mul X, OddC) eq/ne N * C --> X eq/ne N
We previously only did this if the `mul` was `nuw`, but it works for
any odd value.

Alive2 Links:
EQ: https://alive2.llvm.org/ce/z/6_HPZ5
NE: https://alive2.llvm.org/ce/z/c34qSU

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D143026
2023-02-06 14:09:18 -06:00
Noah Goldstein
19c766f742 Add tests for folding (icmp UnsignedPred X * Z, Y * Z) -> (icmp UnsignedPred X, Y); NFC
Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D142785
2023-02-06 14:09:18 -06:00
chenglin.bi
defa9b8d5a [Instcombine] precommit tests for icmp with intrinsic look through trunc; NFC 2023-02-06 17:23:02 +08:00
chenglin.bi
322a2ed355 [InstCombine] precommit tests for icmp with bool range; NFC 2023-02-06 17:16:30 +08:00
Matt Arsenault
43b5008e85 InstCombine: Fold is.fpclass (fabs x), mask -> is.fpclass x, (fabs mask) 2023-02-05 09:11:39 -04:00
Matt Arsenault
a520851d12 InstCombine: Fold is.fpclass (fneg x) into the test mask 2023-02-05 09:08:41 -04:00
Matt Arsenault
5ccbd07dd6 InstCombine: Fold out is_fpclass inf checks from test mask for known finite sources
Make the partner transform already done for nnan sources.
2023-02-05 09:07:53 -04:00
Matt Arsenault
e9f3034feb InstCombine: Perform basic isnan combines on llvm.is.fpclass
is.fpclass(x, qnan|snan) -> fcmp uno x, 0.0
is.fpclass(nnan x, qnan|snan|other) -> is.fpclass(x, other)

Start porting the existing combines from llvm.amdgcn.class to the
generic intrinsic. Start with the ones which aren't dependent on the
FP mode.
2023-02-05 08:36:09 -04:00
Craig Topper
2919ec041f [RISCV] Remove side effects from vsetvli intrinsics.
Delete the opt intrinsics since they are now identical.

I left the side effects due to user expectations about how these
interact with things like inline assembly or function calls. Or
that they wouldn't be hoisted. I think we should look at other
ways to address thoughs.

If I could, I'd rename them these somehow to distance them from
the vsetvli instruction. In some sense they only query the VL for
a particular SEW and LMUL. They don't guarantee a vsetvli
instruction will be emitted.

Fixes https://github.com/llvm/llvm-project/issues/59359

Reviewed By: rogfer01, kito-cheng

Differential Revision: https://reviews.llvm.org/D143220
2023-02-03 13:03:56 -08:00
Sanjay Patel
a8f13dbdeb [InstCombine] fold shuffle of fabs
shuffle (fabs X), Mask --> fabs (shuffle X, Mask)
shuffle (fabs X), (fabs Y), Mask --> fabs (shuf X, Y, Mask)

https://alive2.llvm.org/ce/z/JH2nkf

This generalizes the existing fneg transforms to also work with fabs.

A likely follow-up would generalize this further to move any unary
intrinsic op.
2023-02-03 14:23:17 -05:00
Sanjay Patel
0f4d5925b9 [InstCombine] add tests for shuffle-of-fabs; NFC
Adapted from the existing shuffle-of-fneg tests.
2023-02-03 09:26:58 -05:00
chenglin.bi
c0fc1a118c [InstCombine] typo fix in the test xor-and-or.ll; NFC 2023-02-03 17:15:47 +08:00
chenglin.bi
4a66b3b20e [InstCombine] Fold pattern xor(and, or) to select
(A & B) ^ (A | C) --> A ? ~B : C

https://alive2.llvm.org/ce/z/KCBfXr
https://alive2.llvm.org/ce/z/Pm-zJN
https://alive2.llvm.org/ce/z/VT8uC2

Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D141660
2023-02-03 17:12:16 +08:00
Sander de Smalen
da4a5a46b3 [InstCombine] Promote expression tree with @llvm.vscale when zero-extending result.
The LoopVectorizer emits the (scaled) element count as i32, which for
scalable VFs results in calls to @llvm.vscale.i32(). This value is scaled
and further zero-extended to i64.

The zero-extend can be folded away by executing the whole expression in i64
type using @llvm.vscale.i64(). Any logical `and` that would needed to mask
the result can be further folded away by KnownBits analysis when
vscale_range is set.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D143016
2023-02-02 11:18:16 +00:00