1404 Commits

Author SHA1 Message Date
Matt Arsenault
6cfd3439d4
APFloat: Fix signed zero handling in minnum/maxnum (#83376)
Follow the 2019 rules and order -0 as less than +0 and +0 as greater
than -0. As currently defined this isn't required for the intrinsics,
but is a better QoI.

This will avoid the workaround in libc added by #83158
2024-02-29 16:51:33 +05:30
Paul Walker
6a17929e9f [LLVM][tests/Transforms/InstSimplify] Convert instances of ConstantExpr based splats to use splat().
This is mostly NFC but some output does change due to consistently
inserting into poison rather than undef and using i64 as the index
type for inserts.
2024-02-27 13:37:23 +00:00
Björn Pettersson
7677453886
[ConstantFolding] Do not consider padded-in-memory types as uniform (#81854)
Teaching ConstantFoldLoadFromUniformValue that types that are padded in
memory can't be considered as uniform.

Using the big hammer to prevent optimizations when loading from a
constant for which DataLayout::typeSizeEqualsStoreSize would return
false.

Main problem solved would be something like this:
  store i17 -1, ptr %p, align 4
  %v = load i8, ptr %p, align 1
If for example the i17 occupies 32 bits in memory, then LLVM IR doesn't
really tell where the padding goes. And even if we assume that the 15
most significant bits are padding, then they should be considered as
undefined (even if LLVM backend typically would pad with zeroes).
Anyway, for a big-endian target the load would read those most
significant bits, which aren't guaranteed to be one's. So it would be
wrong to constant fold the load as returning -1.

If LLVM IR had been more explicit about the placement of padding, then
we could allow the constant fold of the load in the example, but only
for little-endian.

Fixes: https://github.com/llvm/llvm-project/issues/81793
2024-02-15 15:40:21 +01:00
Yingwei Zheng
470c5b8011
[InstSimplify][InstCombine] Remove unnecessary m_c_* matchers. (#81712)
This patch removes unnecessary `m_c_*` matchers since we always
canonicalize `commutive_op Cst, X` into `commutive_op X, Cst`.

Compile-time impact:
https://llvm-compile-time-tracker.com/compare.php?from=bfc0b7c6891896ee8e9818f22800472510093864&to=d27b058bb9acaa43d3cadbf3cd889e8f79e5c634&stat=instructions:u
2024-02-14 16:40:36 +08:00
Danila Malyutin
cb1a9f70ec
[InstSimplify] Add trivial simplifications for gc.relocate intrinsic (#81639)
Fold gc.relocate of undef and null to undef and null respectively.

Similar transform is currently done by instcombine, but there is no
reason to not include it here as well.
2024-02-14 02:16:32 +03:00
Yingwei Zheng
e17dded8d7
[InstSimplify] Generalize simplifyAndOrOfFCmps (#81027)
This patch generalizes `simplifyAndOrOfFCmps` to simplify patterns like:
```
define i1 @src(float %x, float %y) {
  %or.cond.i = fcmp ord float %x, 0.000000e+00
  %cmp.i.i34 = fcmp olt float %x, %y
  %cmp.i2.sink.i = and i1 %or.cond.i, %cmp.i.i34
  ret i1 %cmp.i2.sink.i
}

define i1 @tgt(float %x, float %y) {
  %cmp.i.i34 = fcmp olt float %x, %y
  ret i1 %cmp.i.i34
}
```
Alive2: https://alive2.llvm.org/ce/z/9rydcx

This patch and #80986 will fix the regression introduced by #80941.
See also the IR diff
https://github.com/dtcxzyw/llvm-opt-benchmark/pull/199#discussion_r1480974120.
2024-02-08 15:07:35 +08:00
Yingwei Zheng
f37d81f8a3
[PatternMatch] Add a matching helper m_ElementWiseBitCast. NFC. (#80764)
This patch introduces a matching helper `m_ElementWiseBitCast`, which is
used for matching element-wise int <-> fp casts.
The motivation of this patch is to avoid duplicating checks in
https://github.com/llvm/llvm-project/pull/80740 and
https://github.com/llvm/llvm-project/pull/80414.
2024-02-07 21:02:13 +08:00
Yingwei Zheng
50e80e06d1
[ValueTracking] Merge cannotBeOrderedLessThanZeroImpl into computeKnownFPClass (#76360)
This patch merges the logic of `cannotBeOrderedLessThanZeroImpl` into
`computeKnownFPClass` to improve the signbit inference.

---------

Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
2024-01-31 18:26:50 +08:00
Matt Arsenault
a46422a776 Reapply "ValueTracking: Identify implied fp classes by general fcmp (#66505)"
This reverts commit 0d0c2298552222b049fa3b8db5efef4b161e51e9.

Includes a bug fix for fcmp one handling, as well as for positive constants.
2024-01-25 13:38:23 +05:30
Nikita Popov
49e3e75143 [ConstantFold] Clean up binop identity folding
Resolve the two FIXMEs: Perform the binop identitiy fold with
AllowRHSConstant, and remove redundant folds later in the code.
2024-01-18 10:37:48 +01:00
Nikita Popov
97e3220d63 [InstSimplify] Consider bitcast as potential cross-lane operation
The bitcast might change the number of vector lanes, in which case
it will be a cross-lane operation.

Fixes https://github.com/llvm/llvm-project/issues/77320.
2024-01-08 15:52:58 +01:00
Nikita Popov
ade7ae4760 [InstSimplify] Add test for #77320 (NFC) 2024-01-08 15:52:58 +01:00
Yingwei Zheng
554feb0058
[InstSimplify] Simplify select cond, undef, val to val if val = poison implies cond = poison (#76465)
This patch folds:
```
select cond, undef, val -> val
select cond, val, undef -> val
```
iff `impliesPoison(val, cond)` returns true.

Example:
```
define i32 @src1(i32 %retval.0.i.i) {
  %cmp.i = icmp sgt i32 %retval.0.i.i, -1
  %spec.select.i = select i1 %cmp.i, i32 %retval.0.i.i, i32 undef
  ret i32 %spec.select.i
}

define i32 @tgt1(i32 %retval.0.i.i) {
  ret i32 %retval.0.i.i
}
```
Alive2: https://alive2.llvm.org/ce/z/okJW3G

Compile-time impact:
http://llvm-compile-time-tracker.com/compare.php?from=38c9390b59c4d2b9181614d6a909887497d3692f&to=e146f51ba278aa3bb6879a9ec651831ac8938e91&stat=instructions%3Au
2023-12-28 23:37:19 +08:00
Yingwei Zheng
8a4266a626
[InstSimplify] Fold u/sdiv exact (mul nsw/nuw X, C), C --> X when C is not a power of 2 (#76445)
Alive2: https://alive2.llvm.org/ce/z/3D9R7d
2023-12-28 17:36:25 +08:00
Yingwei Zheng
1fea712cd1
[ValueTracking] Infer X u<= X +nuw Y for any Y (#75524)
Alive2: https://alive2.llvm.org/ce/z/kiGxCf
Fixes #70374.
2023-12-15 16:33:39 +08:00
Nikita Popov
7686d49517 [ValueTracking] Handle returned attribute with mismatched type
The returned attribute can be used when it is possible to
"losslessly bitcast" between the argument and return type,
including between two vector types.

computeKnownBits() would crash in this case, isKnownNonZero()
would potentially produce a miscompile.

Fixes https://github.com/llvm/llvm-project/issues/74722.
2023-12-08 17:05:13 +01:00
Mikhail Goncharov
0d0c229855 Revert "Reapply "ValueTracking: Identify implied fp classes by general fcmp (#66505)""
This reverts commit d55692d60d218f402ce107520daabed15f2d9ef6.

See discussion in #66505: assertion fires in OSS build of TensorFlow.
2023-12-05 11:10:24 +01:00
Nikita Popov
460faa0c87 [InstSimplify] Check common operand with constant earlier
If both icmps have the same operands and the RHS is constant, we
would currently go into the isImpliedCondMatchingOperands() code
path, instead of the isImpliedCondCommonOperandWithConstants()
path. Both are correct, but the latter can produce more accurate
results if the implication is dependent on the sign.
2023-12-01 12:18:59 +01:00
Nikita Popov
89b0044ca9 [InstSimplify] Add test for implied cond with equal ops and constant (NFC) 2023-12-01 12:18:27 +01:00
Nikita Popov
cd31cf5989 [InstSimplify] Fix or disjoint miscompile with op replacement
Make sure %x does not get folded to "or disjoint %x, %x" without
dropping the flag, as this would be a derefinement.
2023-12-01 11:45:09 +01:00
Nikita Popov
5a1020bb00 [InstSimplify] Add test for disjoint or miscompile (NFC)
The absorption case is already handled correctly, but the
idempentence case is not.
2023-12-01 11:45:09 +01:00
Matt Arsenault
d55692d60d Reapply "ValueTracking: Identify implied fp classes by general fcmp (#66505)"
This reverts commit 96a0d714d58e48c363ee6abbbcdfd7a6ce646ac1.

Avoid assert with dynamic denormal-fp-math We don't recognize compares
with 0 as an exact class test if we don't know the denormal mode. We could
try to do better here, but it's probably not worth it.

Fixes asserts reported after 1adce7d8e47e2438f99f91607760b825e5e3cc37
2023-12-01 17:51:46 +09:00
Nikita Popov
ea602cb806 [IR] Support or disjoint in hasPoisonGeneratingFlags()
This fixed incorrect removal of freeze instructions.
2023-11-30 17:26:23 +01:00
Nikita Popov
ca5a01d8e4 [InstSimplify] Add test for incorrect freeze of or disjoint (NFC) 2023-11-30 17:26:23 +01:00
Nikita Popov
07c18a05e2 [InstSimplify] Fix select bit test miscompile with disjoint
The select condition ensures the disjointness here. The transform
is not valid without dropping the flag, which InstSimplify can't
do.
2023-11-30 16:55:32 +01:00
Nikita Popov
c89553ae82 [InstSimplify] Add test for or disjoint miscompile (NFC) 2023-11-30 16:55:32 +01:00
Graham Hunter
4028dd2e93
[InstSimplify] Fold converted urem to 0 if there's no overlapping bits (#71528)
When folding urem instructions we can end up not recognizing that
the output will always be 0 due to Value*s being different, despite
generating the same data (in this case, 2 different calls to vscale).

This patch recognizes the (x << N) & (add (x << M), -1) pattern that
instcombine replaces urem with after the two vscale calls have been
reduced to one via CSE, then replaces with 0 when x is a power of 2
and N >= M.
2023-11-20 10:27:16 +00:00
Nikita Popov
56c1d30183
[IR] Remove support for lshr/ashr constant expressions (#71955)
Remove support for the lshr and ashr constant expressions. All places
creating them have been removed beforehand, so this just removes the
APIs and uses of these constant expressions in tests.

This is part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
2023-11-14 09:25:14 +01:00
Matt Arsenault
0e1a52f556
ValueTracking: Handle compare gt to -inf in class identification (#72086)
This apparently shows up somewhere in chromium. We also are missing a
canonicalization to an equality compare with inf.
2023-11-14 10:05:38 +09:00
Matt Arsenault
d4912e8050 ValueTracking: Add some tests to cover asserts in fcmpImpliesClass
Catch asserts hit after 1adce7d8e47e2438f99f91607760b825e5e3cc37
2023-11-13 15:05:52 +09:00
Hans Wennborg
96a0d714d5 Revert "ValueTracking: Identify implied fp classes by general fcmp (#66505)"
This causes asserts to fire:

  llvm/lib/Analysis/ValueTracking.cpp:4262:
  std::tuple<Value *, FPClassTest, FPClassTest> llvm::fcmpImpliesClass(CmpInst::Predicate, const Function &, Value *, const APFloat *, bool):
  Assertion `(RHSClass == fcPosNormal || RHSClass == fcNegNormal || RHSClass == fcPosSubnormal || RHSClass == fcNegSubnormal) && "should have been recognized as an exact class test"' failed.

See comments on the PR.

> Previously we could recognize exact class tests performed by
> an fcmp with special values (0s, infs and smallest normal).
> Expand this to recognize the implied classes by a compare with a general
> constant. e.g. fcmp ogt x, 1 implies positive and non-0.
>
> The API should be better merged with fcmpToClassTest but that
> made the diff way bigger, will try to do that in a future
> patch.

This reverts commit dc3faf0ed0e3f1ea9e435a006167d9649f865da1.
2023-11-10 14:45:52 +01:00
Matt Arsenault
dc3faf0ed0
ValueTracking: Identify implied fp classes by general fcmp (#66505)
Previously we could recognize exact class tests performed by
an fcmp with special values (0s, infs and smallest normal).
Expand this to recognize the implied classes by a compare with a general
constant. e.g. fcmp ogt x, 1 implies positive and non-0.
    
The API should be better merged with fcmpToClassTest but that
made the diff way bigger, will try to do that in a future
patch.
2023-11-10 11:39:19 +09:00
Graham Hunter
34f83e86b4 [InstSimplify] Precommit extra tests for PR71528 2023-11-08 17:02:10 +00:00
Graham Hunter
e397a465a8 [InstSimplify] Precommit test for PR71528 2023-11-08 12:17:09 +00:00
Nikita Popov
17764d2c87
[IR] Remove FP cast constant expressions (#71408)
Remove support for the fptrunc, fpext, fptoui, fptosi, uitofp and sitofp
constant expressions. All places creating them have been removed
beforehand, so this just removes the APIs and uses of these constant
expressions in tests.

With this, the only remaining FP operation that still has constant
expression support is fcmp.

This is part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
2023-11-07 09:34:16 +01:00
Nikita Popov
05a4770647 [ConstantFold] Fix incorrect type assumptions
If a pointer isn't a constant expression, global or block address,
it's not guaranteed to be a null pointer. It can also be a no_cfi
or dso_local_equivalent constant.
2023-11-03 13:12:39 +01:00
Nikita Popov
e4a4122eb6
[IR] Remove zext and sext constant expressions (#71040)
Remove support for zext and sext constant expressions. All places
creating them have been removed beforehand, so this just removes the
APIs and uses of these constant expressions in tests.

There is some additional cleanup that can be done on top of this, e.g.
we can remove the ZExtInst vs ZExtOperator footgun.

This is part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
2023-11-03 10:46:07 +01:00
Nikita Popov
060de415af Reapply [InstCombine] Simplify and/or of icmp eq with op replacement (#70335)
Relative to the first attempt, this contains two changes:

First, we only handle the case where one side simplifies to true or
false, instead of calling simplification recursively. The previous
approach would return poison if one operand simplified to poison
(under the equality assumption), which is incorrect.

Second, we do not fold llvm.is.constant in simplifyWithOpReplaced().
We may be assuming that a value is constant, if the equality holds,
but it may not actually be constant. This is nominally just a QoI
issue, but the std::list implementation in libstdc++ relies on the
precise behavior in a way that causes miscompiles.

-----

and/or in logical (select) form benefit from generic simplifications via
simplifyWithOpReplaced(). However, the corresponding fold for plain
and/or currently does not exist.

Similar to selects, there are two general cases for this fold
(illustrated with `and`, but there are `or` conjugates).

The basic case is something like `(a == b) & c`, where the replacement
of a with b or b with a inside c allows it to fold to true or false.
Then the whole operation will fold to either false or `a == b`.

The second case is something like `(a != b) & c`, where the replacement
inside c allows it to fold to false. In that case, the operand can be
replaced with c, because in the case where a == b (and thus the icmp is
false), c itself will already be false.

As the test diffs show, this catches quite a lot of patterns in existing
test coverage. This also obsoletes quite a few existing special-case
and/or of icmp folds we have (e.g. simplifyAndOrOfICmpsWithLimitConst),
but I haven't removed anything as part of this patch in the interest of
risk mitigation.

Fixes #69050.
Fixes #69091.
2023-11-03 10:16:15 +01:00
Nikita Popov
2a1821766c [InstSimplify] Add additional test for #70335 (NFC)
Test the interaction with llvm.is.constant.
2023-11-02 16:15:23 +01:00
Nikita Popov
e331ca5076 [InstSimplify] Add extra tests for #70335 (NFC)
These were miscompiled with the initial version of the patch.
2023-11-02 11:41:47 +01:00
Noah Goldstein
8c2fcf5b77 [InstSimplify] Add some basic simplifications for llvm.ptrmask
Mostly the same as `and`. We also have a check for a useless
`llvm.ptrmask` if the ptr is already known aligned.

Differential Revision: https://reviews.llvm.org/D156633
2023-11-01 23:50:35 -05:00
Noah Goldstein
a37861d707 [InstSimplify] Add tests for simplify llvm.ptrmask; NFC
Differential Revision: https://reviews.llvm.org/D156632
2023-11-01 23:50:35 -05:00
Nikita Popov
e46dd6fbc0 Revert "[InstCombine] Simplify and/or of icmp eq with op replacement (#70335)"
This reverts commit 1770a2e325192f1665018e21200596da1904a330.

Stage 2 llvm-tblgen crashes when generating X86GenAsmWriter.inc and
other files.
2023-10-30 18:33:03 +01:00
Nikita Popov
1770a2e325
[InstCombine] Simplify and/or of icmp eq with op replacement (#70335)
and/or in logical (select) form benefit from generic simplifications via
simplifyWithOpReplaced(). However, the corresponding fold for plain
and/or currently does not exist.

Similar to selects, there are two general cases for this fold
(illustrated with `and`, but there are `or` conjugates).

The basic case is something like `(a == b) & c`, where the replacement
of a with b or b with a inside c allows it to fold to true or false.
Then the whole operation will fold to either false or `a == b`.

The second case is something like `(a != b) & c`, where the replacement
inside c allows it to fold to false. In that case, the operand can be
replaced with c, because in the case where a == b (and thus the icmp is
false), c itself will already be false.

As the test diffs show, this catches quite a lot of patterns in existing
test coverage. This also obsoletes quite a few existing special-case
and/or of icmp folds we have (e.g. simplifyAndOrOfICmpsWithLimitConst),
but I haven't removed anything as part of this patch in the interest of
risk mitigation.

Fixes #69050.
Fixes #69091.
2023-10-30 10:05:39 +01:00
Pierre van Houtryve
4fc1e7db27
[InstSimplify] Fold (a != 0) ? abs(a) : 0 (#70305)
Solves #70204
2023-10-27 14:52:09 +02:00
Nikita Popov
ea1909f82c [InstSimplify] Add tests for #69050 and #69091 (NFC) 2023-10-27 09:50:35 +02:00
Alex Richardson
e39f6c1844 [opt] Infer DataLayout from triple if not specified
There are many tests that specify a target triple/CPU flags but no
DataLayout which can lead to IR being generated that has unusual
behaviour. This commit attempts to use the default DataLayout based
on the relevant flags if there is no explicit override on the command
line or in the IR file.

One thing that is not currently possible to differentiate from a missing
datalayout `target datalayout = ""` in the IR file since the current
APIs don't allow detecting this case. If it is considered useful to
support this case (instead of passing "-data-layout=" on the command
line), I can change IR parsers to track whether they have seen such a
directive and change the callback type.

Differential Revision: https://reviews.llvm.org/D141060
2023-10-26 12:07:37 -07:00
Noah Goldstein
444383e0d0 [ValueTracking] Do more thorough non-zero check in isKnownToBePowerOfTwo when OrZero is no set.
We can cover more cases by directly checking if the result is
known-nonzero for common patterns when they are missing `OrZero`.

This patch add `isKnownNonZero` checks for `shl`, `lshr`, `and`, and `mul`.

Differential Revision: https://reviews.llvm.org/D157309
2023-10-12 16:05:19 -05:00
Nikita Popov
b35f2940e9 [InstSimplify] Avoid use of ConstantExpr::getCast()
Use the constant folding API instead.

One of these uses actually improves results, because the bitcast
expression gets folded away.
2023-09-29 10:23:40 +02:00
Yingwei Zheng
9f2fc88b23
[ValueTracking] Simplify uaddo pattern (#65910)
This patch simplifies the overflow check of unsigned addition.

`a + b <u a` implies `a + b <u b`
`a + b >=u a` implies `a + b >=u b`

Alive2: https://alive2.llvm.org/ce/z/H8oK8n
Fixes #65863.
2023-09-29 01:59:17 +08:00