8645 Commits

Author SHA1 Message Date
Matt Arsenault
f5c8242042
SimplifyLibCalls: Prefer to emit intrinsic in pow(2, x) -> ldexp(1, x) (#92363) 2024-05-17 14:28:03 +02:00
David Sherwood
0ad275c158
[InstCombine] Fold vector.reduce.op(vector.reverse(X)) -> vector.reduce.op(X) (#91743)
For all of the following reductions:

vector.reduce.or
vector.reduce.and
vector.reduce.xor
vector.reduce.add
vector.reduce.mul
vector.reduce.umin
vector.reduce.umax
vector.reduce.smin
vector.reduce.smax
vector.reduce.fmin
vector.reduce.fmax

if the input operand is the result of a vector.reverse then we can
perform a reduction on the vector.reverse input instead since the answer
is the same. If the reassociation is permitted we can also do the same
folds for these:

vector.reduce.fadd
vector.reduce.fmul
2024-05-17 12:58:14 +01:00
Noah Goldstein
23f1047daa [InstCombine] Fold (icmp pred (trunc nuw/nsw X), C) -> (icmp pred X, (zext/sext C))
This is valid as long as the sign of the wrap flag doesn't differ from
the sign of the `pred`.

Proofs: https://alive2.llvm.org/ce/z/35NsrR

NB: The online Alive2 hasn't been updated with `trunc nuw/nsw`
support, so the proofs must be reproduced locally.

Closes #87935
2024-05-16 13:03:32 -05:00
Noah Goldstein
ebbf484e1c [InstCombine] Add tests for folding (icmp pred (trunc nuw/nsw X), C); NFC 2024-05-16 13:03:32 -05:00
Noah Goldstein
05347f8c2f [ValueTracking] Compute knownbits from (icmp upred (add/sub nuw X, Y), C)
`(icmp ule/ult (add nuw X, Y), C)` implies both `(icmp ule/ult X, C)` and
`(icmp ule/ult Y, C)`. We can use this to deduce leading zeros in `X`/`Y`.

`(icmp uge/ugt (sub nuw X, Y), C)` implies `(icmp uge/uge X, C)` . We
can use this to deduce leading ones in `X`.

Proofs: https://alive2.llvm.org/ce/z/sc5k22

Closes #87180
2024-05-16 13:03:32 -05:00
Noah Goldstein
80a0a067fb [ValueTracking] Add tests for computing knownbits from (icmp upred (add/sub nuw X, Y), C); NFC 2024-05-16 13:03:32 -05:00
Yingwei Zheng
46bc54f4e6
[ValueTracking] Fix assertion failure when computeKnownFPClass returns fcNone (#92355)
Fixes
https://github.com/llvm/llvm-project/pull/92084#issuecomment-2114083188.
2024-05-16 19:14:30 +08:00
Matt Arsenault
0ea178b085
SimplifyLibCalls: Emit vector ldexp intrinsics in exp2->ldexp combine (#92219)
Co-authored-by: Nikita Popov <github@npopov.com>
2024-05-16 10:24:56 +02:00
Matt Arsenault
ce1ce5d30c
InstCombine: Try to use exp10 intrinsic instead of libcall (#92287)
Addresses old TODO about the exp10 intrinsic not existing.
2024-05-16 09:09:02 +02:00
Jay Foad
1650f1b3d7
Fix typo "indicies" (#92232) 2024-05-15 13:10:16 +01:00
Matt Arsenault
d7bb0723fe
InstCombine: Emit ldexp intrinsic in exp2->ldexp combine (#92039)
Prefer to emit the intrinsic over a libcall in the
intrinsic or no-math-errno case.
2024-05-15 07:41:28 +02:00
Matt Arsenault
847c83f7cc
InstCombine: Process addrspacecast uses in PointerReplacer (#91953)
This was looking through an addrspacecast, and not finding a later
unfoldable cast to another address space. Fixes improperly deleting
a required alloca + memcpy and introducing an illegal addrspacecast.
    
This also required fixing some worklist management issues with
addrspacecast, and assuming that only memcpy sources could need
replacement.
    
Regresses one test function, but this looks like it optimized
before by accident. It never saw the pointer use by the call
to readonly_callee, which should require insertion of a new cast.
    
Fixes #68120
2024-05-15 07:02:31 +02:00
Yingwei Zheng
3aae916ff7
Reland "[ValueTracking] Compute knownbits from known fp classes" (#92084)
This patch relands https://github.com/llvm/llvm-project/pull/86409.

I mistakenly thought that `Known.makeNegative()` clears the sign bit of
`Known.Zero`. This patch fixes the assertion failure by explicitly
clearing the sign bit.
2024-05-14 18:10:28 +08:00
Martin Storsjö
2e165a2c4b Revert "[ValueTracking] Compute knownbits from known fp classes (#86409)"
This reverts commit d03a1a6e5838c7c2c0836d71507dfdf7840ade49.

This change caused failed assertions, see
https://github.com/llvm/llvm-project/pull/86409#issuecomment-2109469845
for details.
2024-05-14 10:37:23 +03:00
Yingwei Zheng
d03a1a6e58
[ValueTracking] Compute knownbits from known fp classes (#86409)
This patch calculates knownbits from fp instructions/dominating fcmp
conditions. It will enable more optimizations with signbit idioms.
2024-05-14 01:58:45 +08:00
Matt Arsenault
c5b0da9d83
InstCombine: Preserve inbounds in PointerReplacer (#91735)
This avoids spurious test changes in a future commit.
2024-05-13 13:49:09 +02:00
Yingwei Zheng
b5f4210e9f
[InstCombine] Drop nuw flag when CtlzOp is a sub nuw (#91776)
See the following case:
```
define i32 @src1(i32 %x) {
  %dec = sub nuw i32 -2, %x
  %ctlz = tail call i32 @llvm.ctlz.i32(i32 %dec, i1 false)
  %sub = sub nsw i32 32, %ctlz
  %shl = shl i32 1, %sub
  %ugt = icmp ult i32 %x, -2
  %sel = select i1 %ugt, i32 %shl, i32 1
  ret i32 %sel
}

define i32 @tgt1(i32 %x) {
  %dec = sub nuw i32 -2, %x
  %ctlz = tail call i32 @llvm.ctlz.i32(i32 %dec, i1 false)
  %sub = sub nsw i32 32, %ctlz
  %and = and i32 %sub, 31
  %shl = shl nuw i32 1, %and
  ret i32 %shl
}
```
`nuw` in `%dec` should be dropped after the select instruction is
eliminated.

Alive2: https://alive2.llvm.org/ce/z/7S9529

Fixes https://github.com/llvm/llvm-project/issues/91691.
2024-05-13 14:27:59 +08:00
Alex Bradbury
3be8e2c95d
[InstCombine] Prefer to keep power-of-2 constants when combining ashr exact and slt/ult of a constant (#86111)
We have flexibility in what constant to use when combining an `ashr
exact` with a slt or ult of a constant, and it's not possible to revisit
this decision later in the compilation pipeline after the `ashr exact`
is removed. Keeping a constant close to power-of-2 (pow2val + 1) should
be no worse than neutral, and in some cases may allow better codegen
later on for targets that can more cheaply generate power of 2 (which
may be selectable if converting back to setle/setge) or near power of 2
constants.

Alive2 proofs:
<https://alive2.llvm.org/ce/z/2BmPnq> and
<https://alive2.llvm.org/ce/z/DtuhnR>
2024-05-10 13:50:03 +01:00
Alex Bradbury
452f4393c7 [InstCombine] Precommit tests for #86111
The upcoming patch adds logic to prefer to use constants close to
power-of-two in these ashr exact + slt/ult patterns when it has a choice
on which constant can be used.
2024-05-10 13:47:57 +01:00
Nikita Popov
534701d5f9 [InstCombine] Handle commuted variants in or of xor pattern
This pattern only handled commutation in the "or", while all
involved operations are commutative. Make sure we handle all
sixteen patterns.
2024-05-09 15:16:25 +09:00
Nikita Popov
90ffaa6ccc [InstCombine] Add proper test coverage for or of xors pattern (NFC)
Test all commuted variants of the pattern, most of which currently
fail to fold.
2024-05-09 15:16:25 +09:00
Nikita Popov
73d423319c [InstCombine] Regenerate test checks (NFC) 2024-05-09 13:25:37 +09:00
Nikita Popov
f958a7348f [InstCombine] Fix name clashes in check lines (NFC)
These used both lower and upper case variants of the same name,
resulting in malformed check lines when regenerated.
2024-05-09 12:43:00 +09:00
Nikita Popov
0d335f78e4 [InstCombine] Handle more commuted cases in matchesSquareSum() 2024-05-09 12:35:16 +09:00
Nikita Popov
a39a382755 [InstCombine] Thwart complexity-based canonicalization (NFC)
These tests did not test what they were supposed to. The transform
fails to actually handle the commuted cases.
2024-05-09 12:29:01 +09:00
AtariDreams
ba5170f430
[InstCombine] Thwart complexity-based canonicalization in shl-add test (NFC) (#91413)
Fixed test for #88193
2024-05-09 09:29:14 +09:00
Teresa Johnson
cec6665f2b
[MemProf] Optionally update hints on existing hot/cold new calls (#91047)
If directed by an option, update hints on calls to new that already
provide a hot/cold hint.
2024-05-08 13:41:29 -07:00
Yingwei Zheng
d085b42cbb
[InstSimplify] Do not simplify freeze in simplifyWithOpReplaced (#91215)
See the LangRef:
> All uses of a value returned by the same ‘freeze’ instruction are
guaranteed to always observe the same value, while different ‘freeze’
instructions may yield different values.

It is incorrect to replace freezes with the simplified value.

Proof:
https://alive2.llvm.org/ce/z/3Dn9Cd
https://alive2.llvm.org/ce/z/Qyh5h6

Fixes https://github.com/llvm/llvm-project/issues/91178
2024-05-08 10:04:09 +08:00
AtariDreams
1e36c96dc0
[InstCombine] Fold ((X << nuw Z) binop nuw Y) >>u Z --> X binop nuw (Y >>u Z) (#88193)
Proofs:
https://alive2.llvm.org/ce/z/N9dRzP
https://alive2.llvm.org/ce/z/Xrpc-Y
https://alive2.llvm.org/ce/z/BagBM6
2024-05-07 21:17:56 +02:00
Monad
0ab4458df0
[InstCombine] Fold cttz(lshr(-1, x) + 1) to width - x (#91244)
Fold
``` llvm
define i64 @src(i64 %50) {
  %52 = lshr i64 -1, %50
  %53 = add i64 %52, 1
  %54 = call i64 @llvm.cttz.i64(i64 %53, i1 false)
  ret i64 %54
}
```
to
``` llvm
define i64 @tgt(i64 %50) {
  %52 = sub i64 64, %50
  ret i64 %52
}
```

as
https://github.com/llvm/llvm-project/pull/91171#pullrequestreview-2040663002
pointed out.

Alive2 proof: https://alive2.llvm.org/ce/z/2aHfYa

Note: the `ctlz` version of this pattern seems not exist in dtcxzyw's
benchmark, so put it aside for now.
2024-05-07 11:06:52 +09:00
Yingwei Zheng
d3dad7a74b
[InstCombine] Fix miscompilation caused by #90436 (#91133)
Proof: https://alive2.llvm.org/ce/z/iRnJ4i

Fixes https://github.com/llvm/llvm-project/issues/91127.
2024-05-06 16:30:07 +08:00
Noah Goldstein
f561daf989 [InstCombine] Add example usage for new Checked matcher API
There is no real motivation for this change other than to highlight a
case where the new `Checked` matcher API can handle non-splat-vecs
without increasing code complexity.

Closes #85676
2024-05-03 14:10:24 -05:00
Noah Goldstein
1708788d2d [InstCombine] Add non-splat test for (icmp (lshr x, y), x); NFC 2024-05-03 14:10:24 -05:00
Fangrui Song
b9ae06ba15 [test] Convert text files from CRLF to LF
Skip *.pdb, *.rc, *crlf*, and FileCheck/dos-style-eol.txt
2024-05-03 10:09:52 -07:00
Nikita Popov
b0eeacb203
[InstCombine] Fold icmp of trunc nuw/nsw (#90436)
Convert the existing foldICmpTruncWithTruncOrExt() fold to work with
trunc nowrap flags instead of computeKnownBits(). This also allows us to
generalize the fold to work with signed comparisons.

Interestingly, apart from the obvious combinations like signed
predicates with trunc nsw, some non-obvious ones are also legal. For
example for unsigned predicates we can do the transform for two trunc
nsw as well (rather than only trunc nuw).

Proofs: https://alive2.llvm.org/ce/z/ndewwK
2024-05-03 11:09:00 +09:00
Nikita Popov
74aa1abfae
[InstCombine] Canonicalize scalable GEPs to use llvm.vscale intrinsic (#90569)
Canonicalize getelementptr instructions for scalable vector types into
ptradd representation with an explicit llvm.vscale call. This
representation has better support in BasicAA, which can reason about
llvm.vscale, but not plain scalable GEPs.
2024-05-01 14:53:43 +09:00
Monad
34c89eff64
[InstCombine] Fold trunc nuw/nsw (x xor y) to i1 to x != y (#90408)
Fold:
``` llvm
define i1 @src(i8 %x, i8 %y) {
  %xor = xor i8 %x, %y
  %r = trunc nuw/nsw i8 %xor to i1
  ret i1 %r
}

define i1 @tgt(i8 %x, i8 %y) {
  %r = icmp ne i8 %x, %y
  ret i1 %r
}
```

Proof: https://alive2.llvm.org/ce/z/dcuHmn
2024-04-30 18:07:40 +08:00
Maciej Gabka
bfc0317153
Move several vector intrinsics out of experimental namespace (#88748)
This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from the experimental namespace.

All these intrinsics exist in LLVM for more than a year now, and are
widely used, so should not be considered as experimental.
2024-04-29 10:16:45 +01:00
Nikita Popov
e1622e189e [InstCombine] Add tests for trunc nuw/nsw in icmp (NFC) 2024-04-29 16:17:51 +09:00
Nikita Popov
3c553fc9e0
[InstCombine] Infer nuw on mul nsw with non-negative operands (#90170)
If a mul nsw has non-negative operands, it's also nuw.

Proof: https://alive2.llvm.org/ce/z/2Dz9Uu

Fixes https://github.com/llvm/llvm-project/issues/90020.
2024-04-29 09:53:09 +09:00
Nikita Popov
feaddc1019
[InstCombine] Preserve inbounds when canonicalizing gep+add (#90160)
When canonicalizing gep+add into gep+gep we can preserve inbounds if the
add is also nsw and both add operands are non-negative (or both
negative, but I don't think that's practically relevant).

Proof: https://alive2.llvm.org/ce/z/tJLBta
2024-04-29 09:44:45 +09:00
Nikita Popov
82d87600e5 [InstCombine] Add extra tests for mul nuw inference (NFC) 2024-04-26 14:45:46 +09:00
Nikita Popov
c8edf11dc9 [InstCombine] Add tests for inbounds preservation in gep+add fold (NFC) 2024-04-26 11:38:36 +09:00
Nikita Popov
cbe1760f02
[InstCombine] Allow multi-use OptimizePointerDifference() with two GEPs (#90017)
Currently, the OptimizePointerDifference fold does not trigger when
working on the sub of two geps where one of the geps has multiple uses,
to avoid duplicating the offset arithmetic too much.

However, there are cases where performing it would still be
clearly profitable, e.g. test_sub_ptradd_multiuse.

This patch drops the one-use restriction using the same strategy we use
in GEP comparison folds: If there are multiple uses, we rewrite the GEP
to use the expanded offset arithmetic instead (effectively
canonicalizing it into ptradd representation).

Fixes https://github.com/llvm/llvm-project/issues/88231.
2024-04-26 10:53:03 +09:00
Yingwei Zheng
5a1d85051f
[InstCombine] Canonicalize gep T, (gep i8, base, C1), (Index + C2) into gep T, (gep i8, base, C1 + C2 * sizeof(T)), Index (#76177)
This patch tries to canonicalize `gep T, (gep i8, base, C1), (Index +
C2)` into `gep T, (gep i8, base, C1 + C2 * sizeof(T)), Index`.

Alive2: https://alive2.llvm.org/ce/z/dxShKF
Fixes regressions found in
https://github.com/llvm/llvm-project/pull/68882.
2024-04-26 01:42:10 +08:00
Yingwei Zheng
cbb0477e9a
[InstCombine] Fold fneg over select (#89947)
As we folds fabs over select in
https://github.com/llvm/llvm-project/pull/86390, this patch folds fneg
over select to make sure nabs idioms are generated.
Addresses
https://github.com/llvm/llvm-project/pull/86390#discussion_r1568862289.

Alive2 for FMF propagation: https://alive2.llvm.org/ce/z/-h6Vuo
2024-04-25 23:14:37 +08:00
Nikita Popov
7a77b763af [InstCombine] Add additional multi-use test for sub of gep (NFC)
A case where still performing the fold is clearly profitable.
2024-04-25 14:24:55 +09:00
Nikita Popov
865b73ae45 [InstCombine] Add multi-use tests for shift-of-shift transform (NFC)
Also drop irrelevant function attributes from tests.
2024-04-25 13:45:23 +09:00
Andreas Jonson
b8f3024a31
[InstCombine] Swap out range metadata to range attribute for cttz/ctlz/ctpop (#88776)
Since all optimizations that use range metadata now also handle range attribute, this patch replaces writes of
range metadata for call instructions to range attributes.
2024-04-25 01:45:50 +08:00
Yingwei Zheng
945eeb2d92
[InstCombine] Simplify (X / C0) * C1 + (X % C0) * C2 to (X / C0) * (C1 - C2 * C0) + X * C2 (#76285)
Since `DivRemPairPass` runs after `ReassociatePass` in the optimization
pipeline, I decided to do this simplification in `InstCombine`.

Alive2: https://alive2.llvm.org/ce/z/Jgsiqf
Fixes #76128.
2024-04-24 17:01:49 +08:00