991 Commits

Author SHA1 Message Date
Kazu Hirata
92f4001906
[Transforms] Use range-based for loops (NFC) (#97576) 2024-07-03 12:53:19 -07:00
Nikita Popov
99d8bc9e76 [InstCombine] Preserve all gep nowrap flags in ptradd canonicalization 2024-07-01 16:46:00 +02:00
Noah Goldstein
2632680006 [InstCombine] Canonicalize (gep <not i8> p, (div exact X, C))
If C % sizeof(gep_element_type) is zero, we can canonicalize to `i8` via:
    `(gep i8 p, (div exact X, C / (sizeof(gep_element_type))))`

Closes #96898
2024-07-01 22:22:35 +08:00
Youngsuk Kim
2051736f7b [llvm][Transforms] Avoid 'raw_string_ostream::str' (NFC)
Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.

Work towards TODO comment to remove `raw_string_ostream::str()`.
2024-06-30 09:03:29 -05:00
Nikita Popov
9df71d7673
[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)
Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, replacing the
current `getParent()->getDataLayout()` pattern.
2024-06-28 08:36:49 +02:00
Nikita Popov
2d209d964a
[IR] Add getDataLayout() helpers to BasicBlock and Instruction (#96902)
This is a helper to avoid writing `getModule()->getDataLayout()`. I
regularly try to use this method only to remember it doesn't exist...

`getModule()->getDataLayout()` is also a common (the most common?)
reason why code has to include the Module.h header.
2024-06-27 16:38:15 +02:00
David Green
352a836176
[InstCombine] Canonicalize non-i8 gep of mul to i8 (#96606)
This is a small canonicalization for `gep i32, p, (mul x, C)` -> `gep
i8, p, (mul x, C*4)`, so that the mul can combine both of the constant
multiplications, and we take a small step towards canonicalizing more
geps to i8.

It currently doesn't attempt to check for multiple uses on the mul, but
that should be possible if it sounds better. Let me know what you think
of the idea in general.
2024-06-26 14:25:54 +01:00
Nikita Popov
60984f5be9 [InstCombine] Preserve all gep flags in gep of exact div fold 2024-06-19 14:33:43 +02:00
Nikita Popov
2a1169088c [InstCombine] Preserve all gep flags when emitting offset 2024-06-19 12:36:36 +02:00
Nikita Popov
807222245e [InstCombine] Preserve all gep flags in gep of select fold 2024-06-19 12:31:53 +02:00
Nikita Popov
534e3ad08b [InstCombine] Avoid use of ConstantExpr::getShl()
Use the constant folding API instead. Use ImmConstant to ensure
folding succeeds.
2024-06-18 16:58:11 +02:00
Nikita Popov
9a86d0a6b5 [InstCombine] Prefer source over result element type (NFC)
For single-index GEPs the source and result element types are the
same, but using the source type is semantically more correct.
2024-06-17 11:03:00 +02:00
Nikita Popov
e57308b063 [IR] Accept GEPNoWrapFlags in creation APIs
Add overloads of GetElementPtrInst::Create() that accept
GEPNoWrapFlags, and switch the bool parameters in IRBuilder to
accept it instead as well.

As a sample use, switch GEP i8 canonicalization in InstCombine to
preserve the original flags.
2024-06-04 14:08:33 +02:00
Nikita Popov
f98be870e4 [InstSimplify] Accept GEPNoWrapFlags instead of only InBounds flag
This preserves the flags if a constexpr GEP is created (at least
as long as they don't get dropped later -- the test cases uses a
constexpr index to avoid that).
2024-06-04 11:15:06 +02:00
Nikita Popov
34b4112c90 [InstCombine] Simplify isMergedGEPInBounds() (NFCI)
Since the switch to opaque pointers, zero-index GEPs will be
optimized away anyway, so there is no need to explicitly handle
them here.
2024-06-04 09:37:10 +02:00
Mingming Liu
56c5ca8f66
[nfc][InstCombine]Find PHI incoming block by operand number (#93249) 2024-05-24 14:19:02 -07:00
Eli Friedman
f893dccbba
Replace uses of ConstantExpr::getCompare. (#91558)
Use ICmpInst::compare() where possible, ConstantFoldCompareInstOperands
in other places. This only changes places where the either the fold is
guaranteed to succeed, or the code doesn't use the resulting compare if
we fail to fold.
2024-05-09 16:50:01 -07:00
Nikita Popov
74aa1abfae
[InstCombine] Canonicalize scalable GEPs to use llvm.vscale intrinsic (#90569)
Canonicalize getelementptr instructions for scalable vector types into
ptradd representation with an explicit llvm.vscale call. This
representation has better support in BasicAA, which can reason about
llvm.vscale, but not plain scalable GEPs.
2024-05-01 14:53:43 +09:00
Maciej Gabka
bfc0317153
Move several vector intrinsics out of experimental namespace (#88748)
This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from the experimental namespace.

All these intrinsics exist in LLVM for more than a year now, and are
widely used, so should not be considered as experimental.
2024-04-29 10:16:45 +01:00
Nikita Popov
feaddc1019
[InstCombine] Preserve inbounds when canonicalizing gep+add (#90160)
When canonicalizing gep+add into gep+gep we can preserve inbounds if the
add is also nsw and both add operands are non-negative (or both
negative, but I don't think that's practically relevant).

Proof: https://alive2.llvm.org/ce/z/tJLBta
2024-04-29 09:44:45 +09:00
Yingwei Zheng
5a1d85051f
[InstCombine] Canonicalize gep T, (gep i8, base, C1), (Index + C2) into gep T, (gep i8, base, C1 + C2 * sizeof(T)), Index (#76177)
This patch tries to canonicalize `gep T, (gep i8, base, C1), (Index +
C2)` into `gep T, (gep i8, base, C1 + C2 * sizeof(T)), Index`.

Alive2: https://alive2.llvm.org/ce/z/dxShKF
Fixes regressions found in
https://github.com/llvm/llvm-project/pull/68882.
2024-04-26 01:42:10 +08:00
Nikita Popov
873889b7fa [InstCombine] Extract logic for "emit offset and rewrite gep" (NFC) 2024-04-25 14:18:11 +09:00
Kai Nacke
d5022d9ad4
[SystemZ][z/OS] Make z/OS personality function known (#89679)
This change adds the z/OS personality function to the list of known EH
personality functions. It enables removing of the EH data/labels if the
personality function is not invoked.
2024-04-23 10:39:03 -04:00
Haohai Wen
d8503a38b9
[InstCombine] Update BranchProbabilityAnalysis cache result (#86470)
InstCombine may invert branch condition and profile metadata. In such
case, BranchProbabilityAnalysis should also be updated.
2024-04-20 22:07:41 +08:00
Nikita Popov
1baa385065
[IR][PatternMatch] Only accept poison in getSplatValue() (#89159)
In #88217 a large set of matchers was changed to only accept poison
values in splats, but not undef values. This is because we now use
poison for non-demanded vector elements, and allowing undef can cause
correctness issues.

This patch covers the remaining matchers by changing the AllowUndef
parameter of getSplatValue() to AllowPoison instead. We also carry out
corresponding renames in matchers.

As a followup, we may want to change the default for things like m_APInt
to m_APIntAllowPoison (as this is much less risky when only allowing
poison), but this change doesn't do that.

There is one caveat here: We have a single place
(X86FixupVectorConstants) which does require handling of vector splats
with undefs. This is because this works on backend constant pool
entries, which currently still use undef instead of poison for
non-demanded elements (because SDAG as a whole does not have an explicit
poison representation). As it's just the single use, I've open-coded a
getSplatValueAllowUndef() helper there, to discourage use in any other
places.
2024-04-18 15:44:12 +09:00
Andreas Jonson
ff3523f67b
[IR] Drop poison-generating return attributes when necessary (#89138)
Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.
2024-04-18 15:27:36 +09:00
Harald van Dijk
60de56c743
[ValueTracking] Restore isKnownNonZero parameter order. (#88873)
Prior to #85863, the required parameters of llvm::isKnownNonZero were
Value and DataLayout. After, they are Value, Depth, and SimplifyQuery,
where SimplifyQuery is implicitly constructible from DataLayout. The
change to move Depth before SimplifyQuery needed callers to be updated
unnecessarily, and as commented in #85863, we actually want Depth to be
after SimplifyQuery anyway so that it can be defaulted and the caller
does not need to specify it.
2024-04-16 15:21:09 +01:00
Yingwei Zheng
5fe146672d
[InstCombine] Simplify switch with selects (#84143)
An example from https://github.com/image-rs/image:
```
define void @test_ult_rhsc(i8 %x) {
  %val = add nsw i8 %x, -2
  %cmp = icmp ult i8 %val, 11
  %cond = select i1 %cmp, i8 %val, i8 6
  switch i8 %cond, label %bb1 [
  i8 0, label %bb2
  i8 10, label %bb3
  ]

bb1:
  call void @func1()
  unreachable
bb2:
  call void @func2()
  unreachable
bb3:
  call void @func3()
  unreachable
}
```

When `%cmp` evaluates to false, we can prove that the range of `%val` is
[11, umax]. Thus we can safely replace `%cond` with `%val` since both
`switch 6` and `switch %val` go to the default dest `%bb1`.

Alive2: https://alive2.llvm.org/ce/z/uSTj6w
Godbolt: https://godbolt.org/z/MGrG84bzr

This patch will benefit many rust applications and some C/C++
applications (e.g., cvc5).
2024-04-15 16:40:16 +08:00
Yingwei Zheng
e0a628715a
[ValueTracking] Convert isKnownNonZero to use SimplifyQuery (#85863)
This patch converts `isKnownNonZero` to use SimplifyQuery. Then we can
use the context information from `DomCondCache`.

Fixes https://github.com/llvm/llvm-project/issues/85823.
Alive2: https://alive2.llvm.org/ce/z/QUvHVj
2024-04-12 23:47:20 +08:00
Stephen Tozer
ffd08c7759
[RemoveDIs][NFC] Rename DPValue -> DbgVariableRecord (#85216)
This is the major rename patch that prior patches have built towards.
The DPValue class is being renamed to DbgVariableRecord, which reflects
the updated terminology for the "final" implementation of the RemoveDI
feature. This is a pure string substitution + clang-format patch. The
only manual component of this patch was determining where to perform
these string substitutions: `DPValue` and `DPV` are almost exclusively
used for DbgRecords, *except* for:

- llvm/lib/target, where 'DP' is used to mean double-precision, and so
appears as part of .td files and in variable names. NB: There is a
single existing use of `DPValue` here that refers to debug info, which
I've manually updated.
- llvm/tools/gold, where 'LDPV' is used as a prefix for symbol
visibility enums.

Outside of these places, I've applied several basic string
substitutions, with the intent that they only affect DbgRecord-related
identifiers; I've checked them as I went through to verify this, with
reasonable confidence that there are no unintended changes that slipped
through the cracks. The substitutions applied are all case-sensitive,
and are applied in the order shown:

```
  DPValue -> DbgVariableRecord
  DPVal -> DbgVarRec
  DPV -> DVR
```

Following the previous rename patches, it should be the case that there
are no instances of any of these strings that are meant to refer to the
general case of DbgRecords, or anything other than the DPValue class.
The idea behind this patch is therefore that pure string substitution is
correct in all cases as long as these assumptions hold.
2024-03-19 20:07:07 +00:00
Yingwei Zheng
252d01952c
[InstCombine] Drop UB-implying attrs/metadata after speculating an instruction (#85542)
When speculating an instruction in `InstCombinerImpl::FoldOpIntoSelect`,
the call may result in undefined behavior. This patch drops all
UB-implying attrs/metadata to fix this.

Fixes #85536.
2024-03-17 14:15:27 +08:00
Yingwei Zheng
cf5cd98e74
[InstCombine] Support and/or in getFreelyInvertedImpl using DeMorgan's Law (#85193)
This patch adds the support for and/or in `getFreelyInvertedImpl` using
DeMorgan's Law:
```
(~(A | B)) -> (~A & ~B)
(~(A & B)) -> (~A | ~B)
```
Alive2: https://alive2.llvm.org/ce/z/Uig8-j
2024-03-15 19:10:02 +08:00
Noah Goldstein
70d0ebb279 [InstCombine] Fix behavior for (fmul (sitfp x), 0)
Bug was introduced in #82555

We where missing check that the constant was non-zero for signed + mul
transform.

Closes #85298
2024-03-14 17:41:25 -05:00
Stephen Tozer
2e865353ed
[RemoveDIs][NFC] Move DPValue::filter -> filterDbgVars (#85208)
This patch changes DPValue::filter to be a non-member method
filterDbgVars. There are two reasons for this: firstly, the name of
DPValue is about to change to DbgVariableRecord, which will result in
every `for` loop that uses DPValue::filter to require a line break. This
is a small thing, but it makes the rename patch more difficult to
review, and is just generally more awkward for what is a fairly common
loop. Secondly, the intent is to later break up the DPValue class into
subclasses, at which point it would be better to have a non-member
function that allows template arguments for the cases we want to filter
with greater specificity.
2024-03-14 12:19:15 +00:00
Yingwei Zheng
fef62be09c
[InstCombine] Canonicalize extractvalue + select (#84686)
This patch canonicalizes `extractvalue (select Cond, TV, FV)` into
`select Cond, (extractvalue TV), (extractvalue FV)`. The latter form may
enable more optimizations.
2024-03-14 14:06:40 +08:00
Nikita Popov
628a79dad3 [InstCombine] Don't generate crash dialog for fixpoint verification failure (NFC)
Fixpoint verification failures outside our tests are usually not
indicative of a bug -- don't be pushy about having people report them.
2024-03-13 16:11:11 +01:00
Stephen Tozer
15f3f446c5
[RemoveDIs][NFC] Rename common interface functions for DPValues->DbgRecords (#84793)
As part of the effort to rename the DbgRecord classes, this patch
renames the widely-used functions that operate on DbgRecords but refer
to DbgValues or DPValues in their names to refer to DbgRecords instead;
all such functions are defined in one of `BasicBlock.h`,
`Instruction.h`, and `DebugProgramInstruction.h`.

This patch explicitly does not change the names of any comments or
variables, except for where they use the exact name of one of the
renamed functions. The reason for this is reviewability; this patch can
be trivially examined to determine that the only changes are direct
string substitutions and any results from clang-format responding to the
changed line lengths. Future patches will cover renaming variables and
comments, and then renaming the classes themselves.
2024-03-12 14:53:13 +00:00
Noah Goldstein
8d976c7f20 [InstCombine] Make (binop ({s|u}itofp),({s|u}itofp)) transform more flexible to mismatched signs
Instead of taking the sign of the cast operation as the required since
for the transform, only force a sign if an operation is maybe
negative.

This gives us more flexability when checking if the floats are safely
converable to integers.

Closes #84389
2024-03-09 11:06:02 -06:00
Noah Goldstein
946ea4e3ca [InstCombine] Add folds for (fp_binop ({s|u}itofp x), ({s|u}itofp y))
The full fold is one of the following:
1) `(fp_binop ({s|u}itofp x), ({s|u}itofp y))`
    -> `({s|u}itofp (int_binop x, y))`
2) `(fp_binop ({s|u}itofp x), FpC)`
    -> `({s|u}itofp (int_binop x, (fpto{s|u}i FpC)))`

And support the following binops:
    `fmul` -> `mul`
    `fadd` -> `add`
    `fsub` -> `sub`

Proofs: https://alive2.llvm.org/ce/z/zuacA8

The proofs timeout, so they must be reproduced locally.

Closes #82555
2024-03-06 13:28:04 -06:00
Noah Goldstein
0f5849eeee [InstCombine] Move folding (add (sitofp x), (sitofp y)) impl to InstructionCombiner; NFC 2024-03-06 13:28:04 -06:00
hanbeom
6cdf596c52
[InstCombine] If inst in unreachable refers to an inst change it to poison (#78444)
Instructions in unreachable basic blocks are removed, but terminators
are not. In this case, even instructions that are only referenced by
a terminator, such as a return instruction, cannot be processed
properly.

This patch changes the operand of a return instruction in an
unreachable basic block to poison if it refers to the instruction,
allowing the instruction to be properly processed.

Fixes #65107.
2024-03-06 09:42:33 +01:00
Yingwei Zheng
abe4677d9f
[InstCombine] Fix infinite loop due to incorrect DoesConsume (#82973)
When a call to `getFreelyInvertedImpl` with a select/phi node fails,
`DoesConsume` should not be changed.

Fixes https://github.com/llvm/llvm-project/issues/82877.
2024-03-05 03:51:06 +08:00
Yingwei Zheng
e88c255313
[InstCombine] Add support for cast instructions in getFreelyInvertedImpl (#82451)
This patch adds support for cast instructions in `getFreelyInvertedImpl`
to enable more optimizations.
Alive2: https://alive2.llvm.org/ce/z/F6maEE
2024-02-23 03:40:39 +08:00
Orlando Cazalet-Hyams
ababa96475
[RemoveDIs][NFC] Introduce DbgRecord base class [1/3] (#78252)
Patch 1 of 3 to add llvm.dbg.label support to the RemoveDIs project. The
patch stack adds a new base class

    -> 1. Add DbgRecord base class for DPValue and the not-yet-added
          DPLabel class.
       2. Add the DPLabel class.
       3. Enable dbg.label conversion and add support to passes.

Patches 1 and 2 are NFC.

In the near future we also will rename DPValue to DbgVariableRecord and
DPLabel to DbgLabelRecord, at which point we'll overhaul the function
names too. The name DPLabel keeps things consistent for now.
2024-02-20 16:00:55 +00:00
Nikita Popov
26d4afc3de
[InstCombine] Fold gep of exact unsigned division (#82334)
Extend the transform added in
https://github.com/llvm/llvm-project/pull/76458 to also handle unsigned
division. X exact/ Y * Y == X holds independently of whether the
division is signed or unsigned.

Proofs: https://alive2.llvm.org/ce/z/wFd5Ec
2024-02-20 12:48:13 +01:00
Matt Arsenault
9dd2c59312
InstCombine: Enable SimplifyDemandedUseFPClass and remove flag (#81108)
This completes the unrevert of ef388334ee5a3584255b9ef5b3fefdb244fa3fd7.
2024-02-14 00:28:46 +05:30
Matt Arsenault
decbd29f9e
Reapply "InstCombine: Introduce SimplifyDemandedUseFPClass"" (#74056)
This reverts commit ef388334ee5a3584255b9ef5b3fefdb244fa3fd7.

The referenced issue violates the spec for finite-only math only by
using a return value for a constant infinity. If the interpretation
is results and arguments cannot violate nofpclass, then any
std::numeric_limits<T>::infinity() result is invalid under
-ffinite-math-only. Without this interpretation the utility of
nofpclass is slashed.
2024-02-08 14:12:39 +05:30
Yingwei Zheng
7ab0a87143
[InstCombine] Try to freely invert phi nodes (#80804)
This patch tries to invert phi nodes if all incoming values are either
constants or nots.
2024-02-07 21:39:12 +08:00
Yingwei Zheng
f37d81f8a3
[PatternMatch] Add a matching helper m_ElementWiseBitCast. NFC. (#80764)
This patch introduces a matching helper `m_ElementWiseBitCast`, which is
used for matching element-wise int <-> fp casts.
The motivation of this patch is to avoid duplicating checks in
https://github.com/llvm/llvm-project/pull/80740 and
https://github.com/llvm/llvm-project/pull/80414.
2024-02-07 21:02:13 +08:00
Yingwei Zheng
cdd9221489
[InstCombine] Simplify the overflow result of umulov X, X (#80796)
This patch does the following folds if only the overflow result is used:
```
extractvalue (umul.with.overflow iN X, X), 1 -> icmp ugt X, 2^(N/2)-1
```
Alive2: https://alive2.llvm.org/ce/z/a8yPC6
2024-02-07 03:10:41 +08:00