6022 Commits

Author SHA1 Message Date
Noah Goldstein
5ca325e49c [InstCombine] Detect (x ^ -x) as a ~Mask
Proof: https://alive2.llvm.org/ce/z/TAFmPw

This is a lemma for clearing up some of the regressions that #84688
causes.

Closes #84868
2024-03-12 13:26:18 -05:00
Stephen Tozer
15f3f446c5
[RemoveDIs][NFC] Rename common interface functions for DPValues->DbgRecords (#84793)
As part of the effort to rename the DbgRecord classes, this patch
renames the widely-used functions that operate on DbgRecords but refer
to DbgValues or DPValues in their names to refer to DbgRecords instead;
all such functions are defined in one of `BasicBlock.h`,
`Instruction.h`, and `DebugProgramInstruction.h`.

This patch explicitly does not change the names of any comments or
variables, except for where they use the exact name of one of the
renamed functions. The reason for this is reviewability; this patch can
be trivially examined to determine that the only changes are direct
string substitutions and any results from clang-format responding to the
changed line lengths. Future patches will cover renaming variables and
comments, and then renaming the classes themselves.
2024-03-12 14:53:13 +00:00
elhewaty
3f302eaca4
[InstCombine] Fold usub_sat((sub nuw C1, A), C2) to usub_sat(C1 - C2, A) or 0 (#82280)
- Fixes: https://github.com/llvm/llvm-project/issues/82177
- Alive2: https://alive2.llvm.org/ce/z/Q7mMC3
2024-03-11 15:10:40 +01:00
Noah Goldstein
60dda1fc6e [InstCombine] fold (icmp eq/ne (and (shl -1, X), Y), 0) -> (icmp eq/ne (lshr Y, X), 0)
Proofs: https://alive2.llvm.org/ce/z/oSRGBt

Closes #84691
2024-03-10 18:16:00 -05:00
Noah Goldstein
193b3d6733 [InstCombine] Recognize (icmp eq/ne (and X, ~Mask), 0) pattern in foldICmpWithLowBitMaskedVal
`(icmp eq/ne (and X, ~Mask), 0)` is equivilent to `(icmp eq/ne (and X,
Mask), X` and we sometimes generate the former pattern intentionally
to reduce number of uses of `X`.
Proof: https://alive2.llvm.org/ce/z/3u-usC

Differential Revision: https://reviews.llvm.org/D159329

Closes #81562
2024-03-10 14:33:34 -05:00
Noah Goldstein
d77eb9ea59 [InstCombine] Improve mask detection in foldICmpWithLowBitMaskedVal
Make recursive matcher that is able to detect a lot more patterns.
Proofs for all supported patterns: https://alive2.llvm.org/ce/z/fSQ3nZ

Differential Revision: https://reviews.llvm.org/D159058
2024-03-10 14:33:34 -05:00
Noah Goldstein
f89e4e339f [InstCombine] Move foldICmpWithLowBitMaskedVal to foldICmpCommutative; NFC 2024-03-10 14:33:34 -05:00
Zain Jaffal
f5811494b0
check if operand is div in fold FDivSqrtDivisor (#81970)
This patch fixes the issues introduced in
bb5c3899d1.

I moved the check for the instruction to be div before I check for the
fast math flags which resolves the crash in

```
float a, b;
double sqrt();
void c() { b = a / sqrt(a); }
```

---------

Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
2024-03-09 17:15:14 +00:00
Noah Goldstein
8d976c7f20 [InstCombine] Make (binop ({s|u}itofp),({s|u}itofp)) transform more flexible to mismatched signs
Instead of taking the sign of the cast operation as the required since
for the transform, only force a sign if an operation is maybe
negative.

This gives us more flexability when checking if the floats are safely
converable to integers.

Closes #84389
2024-03-09 11:06:02 -06:00
Jon Chesterfield
6157538d9e
[InstCombine] ptrmask of gep for dynamic pointer aligment (#80002)
Targets the dynamic realignment pattern of `(Ptr + Align - 1) & -Align;`
as implemented by gep then ptrmask.

Specifically, when the pointer already has alignment information,
dynamically realigning it to less than is already known should be a
no-op. Discovered while writing test cases for another patch.

For the zero low bits of a known aligned pointer, adding the gep index
then removing it with a mask is a no-op. Folding the ptrmask effect
entirely into the gep is the ideal result as that unblocks other
optimisations that are not aware of ptrmask.

In some other cases the gep is known to be dead and is removed without
changing the ptrmask.

In the least effective case, this transform creates a new gep with a
rounded-down index and still leaves the ptrmask unchanged. That
simplified gep is still a minor improvement, geps are cheap and ptrmask
occurs in address calculation contexts so I don't think it's worth
special casing to avoid the extra instruction.
2024-03-07 17:36:28 +00:00
Noah Goldstein
946ea4e3ca [InstCombine] Add folds for (fp_binop ({s|u}itofp x), ({s|u}itofp y))
The full fold is one of the following:
1) `(fp_binop ({s|u}itofp x), ({s|u}itofp y))`
    -> `({s|u}itofp (int_binop x, y))`
2) `(fp_binop ({s|u}itofp x), FpC)`
    -> `({s|u}itofp (int_binop x, (fpto{s|u}i FpC)))`

And support the following binops:
    `fmul` -> `mul`
    `fadd` -> `add`
    `fsub` -> `sub`

Proofs: https://alive2.llvm.org/ce/z/zuacA8

The proofs timeout, so they must be reproduced locally.

Closes #82555
2024-03-06 13:28:04 -06:00
Noah Goldstein
0f5849eeee [InstCombine] Move folding (add (sitofp x), (sitofp y)) impl to InstructionCombiner; NFC 2024-03-06 13:28:04 -06:00
hanbeom
6cdf596c52
[InstCombine] If inst in unreachable refers to an inst change it to poison (#78444)
Instructions in unreachable basic blocks are removed, but terminators
are not. In this case, even instructions that are only referenced by
a terminator, such as a return instruction, cannot be processed
properly.

This patch changes the operand of a return instruction in an
unreachable basic block to poison if it refers to the instruction,
allowing the instruction to be properly processed.

Fixes #65107.
2024-03-06 09:42:33 +01:00
Nikita Popov
9f45c5e1a6
[InstCombine] Fix infinite loop in select equivalence fold (#84036)
When replacing with a non-constant, it's possible that the result of the
simplification is actually more complicated than the original, and may
result in an infinite combine loop.

Mitigate the issue by requiring that either the replacement or
simplification result is constant, which should ensure that it's
simpler. While this check is crude, it does not appear to cause
optimization regressions in real-world code in practice.

Fixes https://github.com/llvm/llvm-project/issues/83127.
2024-03-06 09:33:51 +01:00
Quentin Dian
e96c0c1d5e
[InstCombine] Fix shift calculation in InstCombineCasts (#84027)
Fixes #84025.
2024-03-06 06:16:28 +08:00
Noah Goldstein
61c06775c9 [KnownBits] Add API for nuw flag in computeForAddSub; NFC 2024-03-05 12:59:58 -06:00
Jeremy Morse
2fe81edef6 [NFC][RemoveDIs] Insert instruction using iterators in Transforms/
As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.

There are two general flavours of update:
 * Almost all call-sites just call getIterator on an instruction
 * Several make use of an existing iterator (scenarios where the code is
   actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.

Noteworthy changes:
 * FindInsertedValue now takes an optional iterator rather than an
   instruction pointer, as we need to always insert with iterators,
 * I've added a few iterator-taking versions of some value-tracking and
   DomTree methods -- they just unwrap the iterator. These are purely
   convenience methods to avoid extra syntax in some passes.
 * A few calls to getNextNode become std::next instead (to keep in the
   theme of using iterators for positions),
 * SeparateConstOffsetFromGEP has it's insertion-position field changed.
   Noteworthy because it's not a purely localised spelling change.

All this should be NFC.
2024-03-05 15:12:22 +00:00
Yingwei Zheng
a1a590ef12
[InstCombine] Fix miscompilation in PR83947 (#83993)
762f762504/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp (L394-L407)

Comment from @topperc:
> This transforms assumes the mask is a non-zero splat. We only know its
a splat and not provably all 0s. The mask is a constexpr that includes
the address of the global variable. We can't resolve the constant
expression to an exact value.

Fixes #83947.
2024-03-05 22:34:04 +08:00
Yingwei Zheng
d51fcd4ed8
[InstCombine] Handle scalable splat in getFlippedStrictnessPredicateAndConstant (#83980)
This patch adds support for canonicalization of icmp with a scalable
splat. Some optimizations assume that `icmp pred X, APInt C` is in
canonical form.

Fixes https://github.com/llvm/llvm-project/issues/83931.
2024-03-05 21:08:15 +08:00
SahilPatidar
94a0dd5a19
[InstCombine] Fix Failure to convert vector fp comparisons that can be represented as integers #82241 (#83274)
Resolve #82241

---------

Co-authored-by: SahilPatidar <patidarsahil@2001gmail.com>
2024-03-05 17:19:01 +05:30
Yingwei Zheng
0c47363385
[InstCombine] Simplify nested selects with implied condition (#83739)
This patch does the following simplification:
```
sel1 = select cond1, X, Y 
sel2 = select cond2, sel1, Z
-->
sel2 = select cond2, X, Z if cond2 implies cond1
sel2 = select cond2, Y, Z if cond2 implies !cond1
```
Alive2: https://alive2.llvm.org/ce/z/9A_arU

It cannot be done in CVP/SCCP since we should guarantee that `cond2` is
not an undef.
2024-03-05 14:11:37 +08:00
Yingwei Zheng
abe4677d9f
[InstCombine] Fix infinite loop due to incorrect DoesConsume (#82973)
When a call to `getFreelyInvertedImpl` with a select/phi node fails,
`DoesConsume` should not be changed.

Fixes https://github.com/llvm/llvm-project/issues/82877.
2024-03-05 03:51:06 +08:00
AtariDreams
081882eb9c
[InstCombine] Remove m_OneUse requirement for max, but not min (#81505)
If it is ever determined that min doesn't need one-use, then we can
remove the one-use requirement entirely.
2024-03-05 02:45:45 +08:00
Kai Luo
0f02431273
[InstCombine] Fold (sub (xor X, (sext C)), (sext C)) => (select C (neg X), X) (#79417)
This is useful when computing absdiff.

Correctness prove: https://alive2.llvm.org/ce/z/eMbxps,
https://alive2.llvm.org/ce/z/SNCWJe.

---------

Co-authored-by: Yingwei Zheng <dtcxzyw@qq.com>
2024-02-26 09:43:20 +08:00
Yingwei Zheng
641d160ad2
[InstCombine] Fold umax(smax)/smin(umin) with non-negative constants (#82929)
This patch extends `reassociateMinMaxWithConstants` to fold the
following patterns:
```
umax (smax X, nneg C0), nneg C1 --> smax X, (umax C0, C1)
smin (umin X, nneg C0), nneg C1 --> umin X, (smin/umin C0, C1)
```
Alive2: https://alive2.llvm.org/ce/z/wfEj-e

Address the comment
https://github.com/llvm/llvm-project/pull/82472#pullrequestreview-1896922897.
2024-02-26 03:26:55 +08:00
Yingwei Zheng
e88c255313
[InstCombine] Add support for cast instructions in getFreelyInvertedImpl (#82451)
This patch adds support for cast instructions in `getFreelyInvertedImpl`
to enable more optimizations.
Alive2: https://alive2.llvm.org/ce/z/F6maEE
2024-02-23 03:40:39 +08:00
Benjamin Kramer
d3f6dd6585
[InstCombine] Pick bfloat over half when shrinking ops that started with an fpext from bfloat (#82493)
This fixes the case where we would shrink an frem to half and then
bitcast to bfloat, producing invalid results. The transformation was
written under the assumption that there is only one type with a given
bit width.

Also add a strategic assert to CastInst::CreateFPCast to turn this
miscompilation into a crash.
2024-02-22 15:25:17 +01:00
Paul Walker
28fb2b33c2
[LLVM][SelectionDAG] Reduce number of ComputeValueVTs variants. (#75614)
This is another step in the direction of fixing the `Fixed(0) !=
Scalable(0)` bugbear, although whilst weird I don't believe it's causing
us any real issues.
2024-02-21 13:03:24 +00:00
Nikita Popov
7ce1a11f7f
[InstCombine] Fold dependent IVs (#81151)
Fold `iv = phi(start, iv.next = iv2.next + start)` where `iv2 =
phi(iv2.start, iv2.next = iv2 + iv2.step)`
to  `iv = iv2 + start` removing one induction variable from the loop.

Proof: https://alive2.llvm.org/ce/z/hfmwgf

Fixes https://github.com/llvm/llvm-project/issues/77108.
2024-02-21 09:14:48 +01:00
Simon Pilgrim
d2173d8f53 [InstCombine] foldFCmpIntToFPConst - simplify repeated calls to getBitWidth/getScalarSizeInBits. NFC.
Noticed on #82241 - we don't need to use the IntegerType just for the scalar width, and we were calling it 3 times in different forms - we can just call Type::getScalarSizeInBits once and reuse.
2024-02-20 18:10:25 +00:00
Orlando Cazalet-Hyams
ababa96475
[RemoveDIs][NFC] Introduce DbgRecord base class [1/3] (#78252)
Patch 1 of 3 to add llvm.dbg.label support to the RemoveDIs project. The
patch stack adds a new base class

    -> 1. Add DbgRecord base class for DPValue and the not-yet-added
          DPLabel class.
       2. Add the DPLabel class.
       3. Enable dbg.label conversion and add support to passes.

Patches 1 and 2 are NFC.

In the near future we also will rename DPValue to DbgVariableRecord and
DPLabel to DbgLabelRecord, at which point we'll overhaul the function
names too. The name DPLabel keeps things consistent for now.
2024-02-20 16:00:55 +00:00
Yingwei Zheng
9eb399b854
[InstCombine] Support zext nneg in foldLogicCastConstant (#82355)
This patch extends [D36234](https://reviews.llvm.org/D36234) to handle
`zext nneg` instructions.
I found this while adding support for cast instructions in
`getFreelyInvertedImpl`.
2024-02-20 23:09:00 +08:00
Nikita Popov
26d4afc3de
[InstCombine] Fold gep of exact unsigned division (#82334)
Extend the transform added in
https://github.com/llvm/llvm-project/pull/76458 to also handle unsigned
division. X exact/ Y * Y == X holds independently of whether the
division is signed or unsigned.

Proofs: https://alive2.llvm.org/ce/z/wFd5Ec
2024-02-20 12:48:13 +01:00
Antonio Frighetto
d2a26a7bd5 [InstCombine] Do not perform binop-of-shuffle when mask is poison
A miscompilation issue has been addressed with refined checking.
Shuffle masks operand may be turned into `poison` if this does
not lead to observable changes. This however may not guarantee
binop to binop-of-shuffle replacement to be sound anymore.

Fixes: https://github.com/llvm/llvm-project/issues/82052.
2024-02-20 08:37:51 +01:00
elhewaty
69d4890f80
[InstCombine] Fold abs(a * abs(b)) --> abs(a * b) (#78110)
Proof: https://alive2.llvm.org/ce/z/hfbEra
Fixes: https://github.com/llvm/llvm-project/issues/73211
2024-02-19 11:54:51 +01:00
SahilPatidar
4b483ecd55
[InstCombine] Fix failure to fold (and %x, (sext i1 %m)) -> (select %m, %x, 0) with multiple uses of %m (#81409)
Resolves #81288.
2024-02-19 11:07:16 +01:00
Eikansh Gupta
3363d23bd3 [InstCombine] Do not simplify lshr/shl arg if it is part of a rotate pattern
fshl/fshr having first two arguments as same gets lowered to target
specific rotate. But based on the uses, one of the arguments can get
simplified resulting in different arguments performing equivalent
operation.

This patch prevents the simplification of the arguments of lshr/shl if
they are part of fshl pattern.

Closes https://github.com/llvm/llvm-project/pull/73441.
2024-02-16 16:52:00 +01:00
Eikansh Gupta
db870cfc9e [InstCombine] Extract helper from matchFunnelShift (NFC)
The matchFunnelShift function was doing pattern matching and creating
the fshl/fshr instruction if needed. Moved the pattern matching code to
function convertOrOfShiftsToFunnelShift. It can be reused for other
optimizations.
2024-02-16 16:46:41 +01:00
Yingwei Zheng
470c5b8011
[InstSimplify][InstCombine] Remove unnecessary m_c_* matchers. (#81712)
This patch removes unnecessary `m_c_*` matchers since we always
canonicalize `commutive_op Cst, X` into `commutive_op X, Cst`.

Compile-time impact:
https://llvm-compile-time-tracker.com/compare.php?from=bfc0b7c6891896ee8e9818f22800472510093864&to=d27b058bb9acaa43d3cadbf3cd889e8f79e5c634&stat=instructions:u
2024-02-14 16:40:36 +08:00
Yingwei Zheng
dc866ae49e
[ValueTracking] Move the isSignBitCheck helper into ValueTracking. NFC. (#81704)
This patch moves the `isSignBitCheck` helper into ValueTracking to reuse
the logic in ValueTracking/InstSimplify.

Addresses the comment
https://github.com/llvm/llvm-project/pull/80740#discussion_r1488440050.
2024-02-14 15:33:08 +08:00
Matt Arsenault
9dd2c59312
InstCombine: Enable SimplifyDemandedUseFPClass and remove flag (#81108)
This completes the unrevert of ef388334ee5a3584255b9ef5b3fefdb244fa3fd7.
2024-02-14 00:28:46 +05:30
Noah Goldstein
79ce933114 [InstCombine] Extend (lshr/shl (shl/lshr -1, x), x) -> (lshr/shl -1, x) for multi-use
We previously did this iff the inner `(shl/lshr -1, x)` was
one-use. No instructions are added even if the inner `(shl/lshr -1,
x)` is multi-use and this canonicalization both makes the resulting
instruction easier to analyze and shrinks its dependency chain.

Closes #81576
2024-02-13 12:53:16 -06:00
ostannard
44706bd4f0
[InstCombine] Don't add fcmp instructions to strictfp functions (#81498)
The strictfp attribute has the requirement that "LLVM will not introduce
any new floating-point instructions that may trap". The llvm.is.fpclass
intrinsic is documented as "The function never raises floating-point
exceptions", and the fcmp instruction may raise one, so we can't
transform the former into the latter in functions with the strictfp
attribute.
2024-02-13 09:13:22 +00:00
Florian Hahn
c609846155
[TBAA] Extract logic to use TBAA tag for field of !tbaa.struct (NFC). (#81284) 2024-02-12 11:19:02 +00:00
Nikita Popov
f7f947e620 [InstCombine] Remove some uninteresting FIXMEs (NFC)
If there are two undef operands, the select would get folded away
entirely. One undef operand can occur if the other two operands
do not satisfy the poison implication check. However, I don't think
that handling this edge case is worthwhile in this fold. If we
wanted to handle this, it would be more natural to do so in the
simplifyValueKnownNonZero() fold (as this is actually the property
we would be exploiting -- this doesn't really have any relation
to taking the log2).
2024-02-12 10:36:29 +01:00
Nikita Popov
074f7c2235 [InstCombine] Remove redundant fold (NFCI)
This has been subsumed by simplifyAndOrWithOpReplaced().
2024-02-12 09:54:32 +01:00
Martin Storsjö
f022aaf4e7 Revert "[InstCombine] Optimise x / sqrt(y / z) with fast-math pattern. (#76737)"
This reverts commit bb5c3899d1936ebdf7ebf5ca4347ee2e057bee7f.

That commit caused failed asserts like this:

$ cat repro.c
float a, b;
double sqrt();
void c() { b = a / sqrt(a); }
$ clang -target x86_64-linux-gnu -c -O2 -ffast-math repro.c
clang: ../lib/IR/Instruction.cpp:522: bool llvm::Instruction::hasAllowReassoc() const: Assertion `isa<FPMathOperator>(this) && "getting fast-math flag on invalid op"' failed.
2024-02-10 11:54:31 +02:00
Zain Jaffal
bb5c3899d1
[InstCombine] Optimise x / sqrt(y / z) with fast-math pattern. (#76737)
Replace the pattern with
x * sqrt(z/y)

---------

Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
2024-02-09 17:24:41 +00:00
Nikita Popov
b1b8a383fc
[InstCombine] Remove one-use restriction on icmp of gep fold (#76730)
The fold for icmp (gep (p, i1), gep (p, i2)) to icmp (i1, i2) is
currently limited to one of the GEPs either having one use or a constant
offset. I believe this is to avoid duplicating complex arithmetic both
in the GEP and the offset comparison.

This patch instead does the same thing that the indexed compare fold
does, which is to rewrite the GEP into i8 form if necessary, so that the
offset arithmetic is not repeated after the transform.

I ran into this problem in a case where there are multiple conditions on
the same pointer, which prevents them from getting folded.
2024-02-09 15:25:24 +01:00
Nikita Popov
35d6ae8110
[InstCombine] Handle multi-use in simplifyAndOrWithOpReplaced() (#81006)
Slightly generalize simplifyAndOrWithOpReplaced() by allowing it to
perform simplifications (without creating new instructions) in multi-use
cases. This way we can remove existing patterns without worrying about
multi-use edge cases.

I've opted to change the general way the implementation works to be more
similar to the standard simplifyWithOpReplaced(). We perform the operand
replacement generically, and then try to simplify the result or create a
new instruction if we're allowed to do so.
2024-02-08 09:44:51 +01:00