1142 Commits

Author SHA1 Message Date
Nikita Popov
9df71d7673
[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)
Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, replacing the
current `getParent()->getDataLayout()` pattern.
2024-06-28 08:36:49 +02:00
Nikita Popov
2d209d964a
[IR] Add getDataLayout() helpers to BasicBlock and Instruction (#96902)
This is a helper to avoid writing `getModule()->getDataLayout()`. I
regularly try to use this method only to remember it doesn't exist...

`getModule()->getDataLayout()` is also a common (the most common?)
reason why code has to include the Module.h header.
2024-06-27 16:38:15 +02:00
Nikita Popov
c5aa983f91 [InstSimplify] Fold all poison phi to poison instead of undef 2024-06-25 14:28:13 +02:00
Nikita Popov
9b8c3c6871 [InstSimplify] Use poison instead of undef for unreachable inst 2024-06-24 16:25:40 +02:00
Poseydon42
b7b3d1798d
[InstSimplify] Implement simple folds for ucmp/scmp intrinsics (#95601)
This patch adds folds for the cases where both operands are the same or
where it can be established that the first operand is less than, equal
to, or greater than the second operand.
2024-06-17 13:10:57 +08:00
AtariDreams
fbac697782
[Transforms] Replace incorrect uses of m_Deferred with m_Specific (#95719)
The values have been bound already, so use m_Specific.
2024-06-17 05:50:39 +08:00
Nikita Popov
cc2dc0916a Reapply [ConstantFold] Drop gep of gep fold entirely (#95126)
Reapplying without changes. The flang+openmp buildbot failure
should be addressed by https://github.com/llvm/llvm-project/pull/94541.

-----

This is a followup to https://github.com/llvm/llvm-project/pull/93823
and drops the DataLayout-unaware GEP of GEP fold entirely. All cases are
now left to the DataLayout-aware constant folder, which will fold
everything to a single i8 GEP.

We didn't have any test coverage for this fold in LLVM, but some Clang
tests change.
2024-06-13 17:03:35 +02:00
Nikita Popov
cece0a105b Revert "[ConstantFold] Drop gep of gep fold entirely (#95126)"
This reverts commit 3b3b839c66dc49674fd6646650525a2173030690.

This broke the flang+openmp+offload buildbot, as reported in
https://github.com/llvm/llvm-project/pull/95126#issuecomment-2162424019.
2024-06-12 11:52:12 +02:00
Nikita Popov
3b3b839c66
[ConstantFold] Drop gep of gep fold entirely (#95126)
This is a followup to https://github.com/llvm/llvm-project/pull/93823
and drops the DataLayout-unaware GEP of GEP fold entirely. All cases are
now left to the DataLayout-aware constant folder, which will fold
everything to a single i8 GEP.

We didn't have any test coverage for this fold in LLVM, but some Clang
tests change.
2024-06-12 09:50:14 +02:00
Nikita Popov
f98be870e4 [InstSimplify] Accept GEPNoWrapFlags instead of only InBounds flag
This preserves the flags if a constexpr GEP is created (at least
as long as they don't get dropped later -- the test cases uses a
constexpr index to avoid that).
2024-06-04 11:15:06 +02:00
Nikita Popov
d094bb660d [InstSimplify] Avoid use of ConstantExpr::getICmp() (NFC)
Use ConstantFoldCompareInstOperands() instead.
2024-05-21 09:26:51 +02:00
Ramkumar Ramachandra
c4d6867b78
InstSimplify: strip bad TODO (NFC) (#92754)
foldIdentityShuffles requires two sets of canceling shuffles. If there
are any intervening instructions, they are feeding in the result of the
first set of shuffles. To eliminate the two sets of shuffles, you'd have
to rewrite the head of the intervening instructions to feed in the
operand of the first set of shuffles. Since modifying the IR in any way
is disallowed by an analysis, strip this bad TODO.
2024-05-21 08:02:10 +01:00
Yingwei Zheng
d085b42cbb
[InstSimplify] Do not simplify freeze in simplifyWithOpReplaced (#91215)
See the LangRef:
> All uses of a value returned by the same ‘freeze’ instruction are
guaranteed to always observe the same value, while different ‘freeze’
instructions may yield different values.

It is incorrect to replace freezes with the simplified value.

Proof:
https://alive2.llvm.org/ce/z/3Dn9Cd
https://alive2.llvm.org/ce/z/Qyh5h6

Fixes https://github.com/llvm/llvm-project/issues/91178
2024-05-08 10:04:09 +08:00
Maciej Gabka
bfc0317153
Move several vector intrinsics out of experimental namespace (#88748)
This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from the experimental namespace.

All these intrinsics exist in LLVM for more than a year now, and are
widely used, so should not be considered as experimental.
2024-04-29 10:16:45 +01:00
Nikita Popov
f8a19a8f74 [SimplifyQuery] Avoid PatternMatch.h include (NFC)
Move the one method that uses it out of line. This is primarily to
reduce the number of files to rebuild when changing PatternMatch.h.
2024-04-23 12:18:07 +09:00
Nikita Popov
1baa385065
[IR][PatternMatch] Only accept poison in getSplatValue() (#89159)
In #88217 a large set of matchers was changed to only accept poison
values in splats, but not undef values. This is because we now use
poison for non-demanded vector elements, and allowing undef can cause
correctness issues.

This patch covers the remaining matchers by changing the AllowUndef
parameter of getSplatValue() to AllowPoison instead. We also carry out
corresponding renames in matchers.

As a followup, we may want to change the default for things like m_APInt
to m_APIntAllowPoison (as this is much less risky when only allowing
poison), but this change doesn't do that.

There is one caveat here: We have a single place
(X86FixupVectorConstants) which does require handling of vector splats
with undefs. This is because this works on backend constant pool
entries, which currently still use undef instead of poison for
non-demanded elements (because SDAG as a whole does not have an explicit
poison representation). As it's just the single use, I've open-coded a
getSplatValueAllowUndef() helper there, to discourage use in any other
places.
2024-04-18 15:44:12 +09:00
Andreas Jonson
ff3523f67b
[IR] Drop poison-generating return attributes when necessary (#89138)
Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.
2024-04-18 15:27:36 +09:00
Nikita Popov
d9a5aa8e2d
[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)
Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elements inside vector
splats, not undef elements.

Historically, we used to represent non-demanded elements in vectors
using undef. Nowadays, we use poison instead. As such, I believe that
support for undef in vector splats is no longer useful.

At the same time, while poison splat elements are pretty much always
safe to ignore, this is not generally the case for undef elements. We
have existing miscompiles in our tests due to this (see the
masked-merge-*.ll tests changed here) and it's easy to miss such cases
in the future, now that we write tests using poison instead of undef
elements.

I think overall, keeping support for undef elements no longer makes
sense, and we should drop it. Once this is done consistently, I think we
may also consider allowing poison in m_APInt by default, as doing that
change is much less risky than doing the same with undef.

This change involves a substantial amount of test changes. For most
tests, I've just replaced undef with poison, as I don't think there is
value in retaining both. For some tests (where the distinction between
undef and poison is important), I've duplicated tests.
2024-04-17 18:22:05 +09:00
Harald van Dijk
60de56c743
[ValueTracking] Restore isKnownNonZero parameter order. (#88873)
Prior to #85863, the required parameters of llvm::isKnownNonZero were
Value and DataLayout. After, they are Value, Depth, and SimplifyQuery,
where SimplifyQuery is implicitly constructible from DataLayout. The
change to move Depth before SimplifyQuery needed callers to be updated
unnecessarily, and as commented in #85863, we actually want Depth to be
after SimplifyQuery anyway so that it can be defaulted and the caller
does not need to specify it.
2024-04-16 15:21:09 +01:00
Jordan Rupprecht
37575f5262 [NFC][ValueTracking] Fix Wunused-variable
For e0a628715a8464e220c8ba9e9aaaf2561139198a
2024-04-12 16:27:53 +00:00
Yingwei Zheng
e0a628715a
[ValueTracking] Convert isKnownNonZero to use SimplifyQuery (#85863)
This patch converts `isKnownNonZero` to use SimplifyQuery. Then we can
use the context information from `DomCondCache`.

Fixes https://github.com/llvm/llvm-project/issues/85823.
Alive2: https://alive2.llvm.org/ce/z/QUvHVj
2024-04-12 23:47:20 +08:00
AtariDreams
5d6b00929b
[NFC] Replace m_Sub(m_Zero(), X) with m_Neg(X) (#88461) 2024-04-12 18:24:03 +09:00
Yingwei Zheng
3197f9d8b0
[InstSimplify] Make sure the simplified value doesn't generate poison in threadBinOpOverSelect (#87075)
Alive2: https://alive2.llvm.org/ce/z/y_Jmdn
Fix https://github.com/llvm/llvm-project/issues/87042.
2024-04-11 12:48:52 +08:00
Yingwei Zheng
2f1f6b704d
[LLVM] Use std::move for APInt. NFC. (#86257)
This patch adjusts argument passing for `APInt` to improve the
compile-time.
Compile-time improvement:
https://llvm-compile-time-tracker.com/compare.php?from=d1f182c895728d89c5c3d198b133e212a5d9d4a3&to=ba3e326def3a6e5cd6d72ff5a49c74fba18de1df&stat=instructions:u
2024-03-23 14:58:25 +08:00
Andreas Jonson
e66cfebb04
[ValueTracking] Handle range attributes (#85143)
Handle the range attribute in ValueTracking.
2024-03-20 12:43:00 +01:00
Noah Goldstein
5265be11b1 [InstSimply] Simplify (fmul -x, +/-0) -> -/+0
We already handle the `+x` case, and noticed it was missing in the bug
affecting #82555

Proofs: https://alive2.llvm.org/ce/z/WUSvmV

Closes #85345
2024-03-18 15:11:55 -05:00
Artem Tyurin
141145232f
[IRBuilder] Fold binary intrinsics (#80743)
Fixes https://github.com/llvm/llvm-project/issues/61240.
2024-03-15 09:58:25 +01:00
Andreas Jonson
a3b52509d5
[InstSimpliy] Use range attribute to simplify comparisons (#84627)
Use the new range attribute from https://github.com/llvm/llvm-project/pull/84617
to simplify comparisons where both sides have range information.
2024-03-12 10:39:37 +01:00
Andreas Jonson
54bb4be018
[InstSimplify] Handle vec values when simplifying comparisons using range metadata (#84673)
Found that this failed with an assertion when vec was used in this
optimization while working on https://github.com/llvm/llvm-project/pull/84627.
2024-03-10 12:54:37 +01:00
Björn Pettersson
7677453886
[ConstantFolding] Do not consider padded-in-memory types as uniform (#81854)
Teaching ConstantFoldLoadFromUniformValue that types that are padded in
memory can't be considered as uniform.

Using the big hammer to prevent optimizations when loading from a
constant for which DataLayout::typeSizeEqualsStoreSize would return
false.

Main problem solved would be something like this:
  store i17 -1, ptr %p, align 4
  %v = load i8, ptr %p, align 1
If for example the i17 occupies 32 bits in memory, then LLVM IR doesn't
really tell where the padding goes. And even if we assume that the 15
most significant bits are padding, then they should be considered as
undefined (even if LLVM backend typically would pad with zeroes).
Anyway, for a big-endian target the load would read those most
significant bits, which aren't guaranteed to be one's. So it would be
wrong to constant fold the load as returning -1.

If LLVM IR had been more explicit about the placement of padding, then
we could allow the constant fold of the load in the example, but only
for little-endian.

Fixes: https://github.com/llvm/llvm-project/issues/81793
2024-02-15 15:40:21 +01:00
Yingwei Zheng
470c5b8011
[InstSimplify][InstCombine] Remove unnecessary m_c_* matchers. (#81712)
This patch removes unnecessary `m_c_*` matchers since we always
canonicalize `commutive_op Cst, X` into `commutive_op X, Cst`.

Compile-time impact:
https://llvm-compile-time-tracker.com/compare.php?from=bfc0b7c6891896ee8e9818f22800472510093864&to=d27b058bb9acaa43d3cadbf3cd889e8f79e5c634&stat=instructions:u
2024-02-14 16:40:36 +08:00
Yingwei Zheng
dc866ae49e
[ValueTracking] Move the isSignBitCheck helper into ValueTracking. NFC. (#81704)
This patch moves the `isSignBitCheck` helper into ValueTracking to reuse
the logic in ValueTracking/InstSimplify.

Addresses the comment
https://github.com/llvm/llvm-project/pull/80740#discussion_r1488440050.
2024-02-14 15:33:08 +08:00
Danila Malyutin
cb1a9f70ec
[InstSimplify] Add trivial simplifications for gc.relocate intrinsic (#81639)
Fold gc.relocate of undef and null to undef and null respectively.

Similar transform is currently done by instcombine, but there is no
reason to not include it here as well.
2024-02-14 02:16:32 +03:00
Yingwei Zheng
e17dded8d7
[InstSimplify] Generalize simplifyAndOrOfFCmps (#81027)
This patch generalizes `simplifyAndOrOfFCmps` to simplify patterns like:
```
define i1 @src(float %x, float %y) {
  %or.cond.i = fcmp ord float %x, 0.000000e+00
  %cmp.i.i34 = fcmp olt float %x, %y
  %cmp.i2.sink.i = and i1 %or.cond.i, %cmp.i.i34
  ret i1 %cmp.i2.sink.i
}

define i1 @tgt(float %x, float %y) {
  %cmp.i.i34 = fcmp olt float %x, %y
  ret i1 %cmp.i.i34
}
```
Alive2: https://alive2.llvm.org/ce/z/9rydcx

This patch and #80986 will fix the regression introduced by #80941.
See also the IR diff
https://github.com/dtcxzyw/llvm-opt-benchmark/pull/199#discussion_r1480974120.
2024-02-08 15:07:35 +08:00
Yingwei Zheng
f37d81f8a3
[PatternMatch] Add a matching helper m_ElementWiseBitCast. NFC. (#80764)
This patch introduces a matching helper `m_ElementWiseBitCast`, which is
used for matching element-wise int <-> fp casts.
The motivation of this patch is to avoid duplicating checks in
https://github.com/llvm/llvm-project/pull/80740 and
https://github.com/llvm/llvm-project/pull/80414.
2024-02-07 21:02:13 +08:00
Yingwei Zheng
930996e9e4
[ValueTracking][NFC] Pass SimplifyQuery to computeKnownFPClass family (#80657)
This patch refactors the interface of the `computeKnownFPClass` family
to pass `SimplifyQuery` directly.
The motivation of this patch is to compute known fpclass with
`DomConditionCache`, which was introduced by
https://github.com/llvm/llvm-project/pull/73662. With
`DomConditionCache`, we can do more optimization with context-sensitive
information.

Example (extracted from
[fmt/format.h](e17bc67547/include/fmt/format.h (L3555-L3566))):
```
define float @test(float %x, i1 %cond) {
  %i32 = bitcast float %x to i32
  %cmp = icmp slt i32 %i32, 0
  br i1 %cmp, label %if.then1, label %if.else

if.then1:
  %fneg = fneg float %x
  br label %if.end

if.else:
  br i1 %cond, label %if.then2, label %if.end

if.then2:
  br label %if.end

if.end:
  %value = phi float [ %fneg, %if.then1 ], [ %x, %if.then2 ], [ %x, %if.else ]
  %ret = call float @llvm.fabs.f32(float %value)
  ret float %ret
}
```
We can prove the signbit of `%value` is always zero. Then the fabs can
be eliminated.
2024-02-06 02:30:12 +08:00
Yingwei Zheng
50e80e06d1
[ValueTracking] Merge cannotBeOrderedLessThanZeroImpl into computeKnownFPClass (#76360)
This patch merges the logic of `cannotBeOrderedLessThanZeroImpl` into
`computeKnownFPClass` to improve the signbit inference.

---------

Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
2024-01-31 18:26:50 +08:00
Nikita Popov
97e3220d63 [InstSimplify] Consider bitcast as potential cross-lane operation
The bitcast might change the number of vector lanes, in which case
it will be a cross-lane operation.

Fixes https://github.com/llvm/llvm-project/issues/77320.
2024-01-08 15:52:58 +01:00
ChipsSpectre
4444a7e89a
[InstSimplify] Simplify the expression (a^c)&(a^~c) to zero and (a^c) | (a^~c) to minus one (#76637)
Changes the InstSimplify pass of the LLVM optimizer, such that the
aforementioned expression is reduced to zero if c2==~c1.
Alive2: https://alive2.llvm.org/ce/z/xkQiid
Fixes https://github.com/llvm/llvm-project/issues/75692.
2024-01-03 12:01:02 +01:00
Yingwei Zheng
554feb0058
[InstSimplify] Simplify select cond, undef, val to val if val = poison implies cond = poison (#76465)
This patch folds:
```
select cond, undef, val -> val
select cond, val, undef -> val
```
iff `impliesPoison(val, cond)` returns true.

Example:
```
define i32 @src1(i32 %retval.0.i.i) {
  %cmp.i = icmp sgt i32 %retval.0.i.i, -1
  %spec.select.i = select i1 %cmp.i, i32 %retval.0.i.i, i32 undef
  ret i32 %spec.select.i
}

define i32 @tgt1(i32 %retval.0.i.i) {
  ret i32 %retval.0.i.i
}
```
Alive2: https://alive2.llvm.org/ce/z/okJW3G

Compile-time impact:
http://llvm-compile-time-tracker.com/compare.php?from=38c9390b59c4d2b9181614d6a909887497d3692f&to=e146f51ba278aa3bb6879a9ec651831ac8938e91&stat=instructions%3Au
2023-12-28 23:37:19 +08:00
Yingwei Zheng
8a4266a626
[InstSimplify] Fold u/sdiv exact (mul nsw/nuw X, C), C --> X when C is not a power of 2 (#76445)
Alive2: https://alive2.llvm.org/ce/z/3D9R7d
2023-12-28 17:36:25 +08:00
Paul Walker
dea16ebd26
[LLVM][IR] Replace ConstantInt's specialisation of getType() with getIntegerType(). (#75217)
The specialisation will not be valid when ConstantInt gains native
support for vector types.

This is largely a mechanical change but with extra attention paid to constant
folding, InstCombineVectorOps.cpp, LoopFlatten.cpp and Verifier.cpp to
remove the need to call `getIntegerType()`.

Co-authored-by: Nikita Popov <github@npopov.com>
2023-12-18 11:58:42 +00:00
Yingwei Zheng
741975df92
[InstCombine][InstSimplify] Pass SimplifyQuery to computeKnownBits directly. NFC. (#74246)
This patch passes `SimplifyQuery` to `computeKnownBits` directly in
`InstSimplify` and `InstCombine`.
As the `DomConditionCache` in #73662 is only used in `InstCombine`, it
is inconvenient to introduce a new argument `DC` to `computeKnownBits`.
2023-12-04 02:26:39 +08:00
Nikita Popov
cd31cf5989 [InstSimplify] Fix or disjoint miscompile with op replacement
Make sure %x does not get folded to "or disjoint %x, %x" without
dropping the flag, as this would be a derefinement.
2023-12-01 11:45:09 +01:00
Nikita Popov
07c18a05e2 [InstSimplify] Fix select bit test miscompile with disjoint
The select condition ensures the disjointness here. The transform
is not valid without dropping the flag, which InstSimplify can't
do.
2023-11-30 16:55:32 +01:00
Nikita Popov
d9e8ae7d2f [ValueTracking] Convert MaskedValueIsZero() to use SimplifyQuery (NFC) 2023-11-29 11:18:42 +01:00
Nikita Popov
9ca9c2cf7e [InstSimplify] Remove redundant gep zero fold (NFC)
We already higher the all zero indices case above, no need to
also handle the special case of a single zero index.
2023-11-20 16:25:48 +01:00
Graham Hunter
4028dd2e93
[InstSimplify] Fold converted urem to 0 if there's no overlapping bits (#71528)
When folding urem instructions we can end up not recognizing that
the output will always be 0 due to Value*s being different, despite
generating the same data (in this case, 2 different calls to vscale).

This patch recognizes the (x << N) & (add (x << M), -1) pattern that
instcombine replaces urem with after the two vscale calls have been
reduced to one via CSE, then replaces with 0 when x is a power of 2
and N >= M.
2023-11-20 10:27:16 +00:00
Nikita Popov
2310066faa [InstSimplify] Simplify calculation of GEP result pointer type (NFC)
The result type is the same as the input pointer type, except for
splat geps.
2023-11-17 17:14:07 +01:00
Nikita Popov
ebb8ffde94 [InstSimplify] Extract commutative and folds into helper (NFCI)
There are a number of and folds that are repeated for both
operand orders. Move these into a helper that is invoked with
both orders.

This is conceptually NFC, but may not be entirely so, as the order
of folds may change.
2023-11-15 16:31:55 +01:00