958 Commits

Author SHA1 Message Date
Kazu Hirata
804467de94 Use isNegative (NFC) 2023-04-15 14:26:24 -07:00
Nikita Popov
a162ddf7f2 [InstCombine] Remove various checks for opaque pointers (NFC)
All pointers are opaque now, so these are no longer necessary.
2023-04-06 09:45:51 +02:00
Nikita Popov
238a59c3f1 [InstCombine] Remove varargs cast transform (NFC)
This is no longer relevant with opaque pointers.

Also drop the CastInst::isLosslessCast() method, which was only
used here.
2023-04-05 16:36:21 +02:00
Jie Fu
297242a2bb [InstCombine] Fix -Wimplicit-fallthrough in InstCombinerImpl::visitCallInst (NFC)
/data/llvm-project/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp:3078:3: error: unannotated fall-through between switch labels [-Werror,-Wimplicit-fallthrough]
  default:
  ^
/data/llvm-project/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp:3078:3: note: insert 'break;' to avoid fall-through
  default:
  ^
  break;
1 error generated.
2023-03-31 22:52:46 +08:00
Nikita Popov
6261adfa51 [InstCombine] Fold more intrinsics over selects
Move this handling to a centralized place and extend it to handle
saturating add/sub intrinsics.

I originally wanted to make this fully generic rather than
whitelist based, because this is legal and likely profitable for all
speculatable intrinsics. The caveat is that for vector selects,
the intrinsic can't perform cross-lane operations like a shuffle
or reduction, which we don't really expose as a generic property
right now. So for now I'm just extending the list.
2023-03-31 16:32:21 +02:00
Jun Zhang
cea938390e
[InstCombine] Try to recognize bswap pattern when calling funnel shifts
Alive2: https://alive2.llvm.org/ce/z/dxxD7B
Fixes: https://github.com/llvm/llvm-project/issues/60690

Signed-off-by: Jun Zhang <jun@junz.org>

Differential Revision: https://reviews.llvm.org/D146637
2023-03-24 10:51:36 +08:00
Arthur Eubanks
3f23c7f5be [InstSimplify] Actually use NewOps for calls in simplifyInstructionWithOperands
Resolves a TODO.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D146599
2023-03-22 09:22:00 -07:00
Matt Arsenault
ec2234a21a InstCombine: Fold is.fpclass for single infinity to fcmp
llvm.is.fpclass(x, fcPosInf) -> fcmp oeq x, +inf
llvm.is.fpclass(x, fcNegInf) -> fcmp oeq x, -inf
llvm.is.fpclass(x, ~fcPosInf) -> fcmp one x, +inf
llvm.is.fpclass(x, ~fcNegInf) -> fcmp one x, -inf

llvm.is.fpclass(x, fcPosInf|fcNan) -> fcmp ueq x, +inf
llvm.is.fpclass(x, fcNegInf|fcNan) -> fcmp ueq, -inf
llvm.is.fpclass(x, ~fcPosInf & ~fcNan) -> fcmp one, x, +inf
llvm.is.fpclass(x, ~fcNegInf & ~fcNan) -> fcmp one, x, -inf

This regresses some of the logic of fcmp tests. These should be restored
in a future patch to better handle combining logic of fcmp and class.
2023-03-17 11:52:16 -04:00
Matt Arsenault
80512789b6 InstCombine: Fold is.fpclass nan|zero to fcmp ueq 0 2023-03-17 11:42:38 -04:00
Matt Arsenault
2726fd2e8c Revert "InstCombine: Fold is.fpclass nan|zero to fcmp ueq 0"
This reverts commit b5ff7640cd1391ebe67e47a043fed6faf8bdcbb7.

Merge resolution gone wrong merged two unrelated patches
2023-03-17 11:42:38 -04:00
Matt Arsenault
b5ff7640cd InstCombine: Fold is.fpclass nan|zero to fcmp ueq 0 2023-03-17 11:29:50 -04:00
Matt Arsenault
4109e3f1db Reapply "InstCombine: Fold is.fpclass(x, fcZero) to fcmp oeq 0"
This reverts commit 458ad6900e6635e4640229d40e6f713147e52de7.
2023-03-15 07:06:50 -04:00
Nikita Popov
fdda602c04 Revert "[InstCombine] Return instruction from replaceUse()"
This reverts commit 27c4e233104ba765cd986b3f8b0dcd3a6c3a9f89.

I think I made a mistake with the use in RemoveConditionFromAssume(),
because the instruction being changed is not the current one, but
the next assume. Revert the change for now.
2023-03-14 17:46:33 +01:00
Nikita Popov
27c4e23310 [InstCombine] Return instruction from replaceUse()
Same as with other replacement methods, it's generally necessary
to report a change on the instruction itself, e.g. by returning
it from the visit method (or possibly explicitly adding it to the
worklist).

Return Instruction * from replaceUse() to encourage the usual
"return replaceXYZ" pattern.
2023-03-14 16:53:03 +01:00
Nikita Popov
af101f9ae0 [IR] Allow !range on vector of integer instructions
Inspired by https://reviews.llvm.org/D144467#4188310, this allows
!range on vector of integer instructions, with the usual
element-wise interpretation, which is already used by various
analysis APIs that support vectors.

Differential Revision: https://reviews.llvm.org/D145920
2023-03-14 09:41:56 +01:00
Philip Reames
dae682ce92 [IRBuilder] Add utilities for materializing scalable values [nfc]
These idioms already appear a number of places in code, and upcoming changes to the various sanitizers continue to need more instances of the same patterns.

Differential Revision: https://reviews.llvm.org/D145945
2023-03-13 11:54:19 -07:00
Matt Arsenault
18fa83ed36 ADT: Move some FPClassTest utility functions out of instcombine 2023-03-03 18:20:47 -04:00
David Goldblatt
4d59ffb0d1 [InstCombine] Simplify separate_storage assumptions
Before this change, we call getUnderlyingObject on each separate_storage
operand on every alias() call (potentially requiring lots of pointer
chasing). Instead, we rewrite the assumptions in instcombine to do this
pointer-chasing once.

We still leave the getUnderlyingObject calls in alias(), just expecting
them to be no-ops much of the time. This is relatively fast (just a
couple dyn_casts with no pointer chasing) and avoids making alias
analysis results depend on whether or not instcombine has been run.

Differential Revision: https://reviews.llvm.org/D144933
2023-03-03 13:26:29 +01:00
Nikita Popov
3c2b1853d1 [InstCombine] Avoid ConstantExpr::getSelect() use (NFCI)
Instead let IRBuilder take care of constant folding.

In preparation for removing select constantexprs.
2023-02-27 16:32:11 +01:00
Noah Goldstein
f35e3fa53b Add transforms for (max/min (xor X, Pow2), X) -> (and/or X, Pow2/~Pow2)
X ^ Pow2 is guranteed to flip one bit. We can use this to speedup
max/min by just selecting X with/without (or/andnot) the flipped bit
respectively.

Alive2 Links:
smax-neg: https://alive2.llvm.org/ce/z/j3QYFs
smin-neg: https://alive2.llvm.org/ce/z/bFYnQW
smax-pos: https://alive2.llvm.org/ce/z/4xYSxR
smin-pos: https://alive2.llvm.org/ce/z/H3RPKj
umax    : https://alive2.llvm.org/ce/z/P4oRcX
umin    : https://alive2.llvm.org/ce/z/vWZG6p

Differential Revision: https://reviews.llvm.org/D144606
2023-02-24 15:22:09 -06:00
Nick Desaulniers
45a291b5f6 [Dominators] check indirect branches of callbr
This will be necessary to support outputs from asm goto along indirect
edges.

Test via:
  $ pushd llvm/build; ninja IRTests; popd
  $ ./llvm/build/unittests/IR/IRTests \
    --gtest_filter=DominatorTree.CallBrDomination

Also, return nullptr in Instruction::getInsertionPointAfterDef for
CallBrInst as was recommened in
https://reviews.llvm.org/D135997#3991427.  The following phab review was
folded into this commit: https://reviews.llvm.org/D140166

Link: Link: https://discourse.llvm.org/t/rfc-syncing-asm-goto-with-outputs-with-gcc/65453/8

Reviewed By: void, efriedma, ChuanqiXu, MaskRay

Differential Revision: https://reviews.llvm.org/D135997
2023-02-16 17:58:33 -08:00
Matt Arsenault
458ad6900e Revert "InstCombine: Fold is.fpclass(x, fcZero) to fcmp oeq 0"
This reverts commit df78976d023a6b7fcf64bc695261b7b402fcede0.

I pushed the wrong patch
2023-02-15 05:27:26 -04:00
Matt Arsenault
df78976d02 InstCombine: Fold is.fpclass(x, fcZero) to fcmp oeq 0
This requires the denormal mode to definitively be IEEE handling.
2023-02-15 05:21:00 -04:00
Matt Arsenault
43b5008e85 InstCombine: Fold is.fpclass (fabs x), mask -> is.fpclass x, (fabs mask) 2023-02-05 09:11:39 -04:00
Matt Arsenault
a520851d12 InstCombine: Fold is.fpclass (fneg x) into the test mask 2023-02-05 09:08:41 -04:00
Matt Arsenault
5ccbd07dd6 InstCombine: Fold out is_fpclass inf checks from test mask for known finite sources
Make the partner transform already done for nnan sources.
2023-02-05 09:07:53 -04:00
Matt Arsenault
e9f3034feb InstCombine: Perform basic isnan combines on llvm.is.fpclass
is.fpclass(x, qnan|snan) -> fcmp uno x, 0.0
is.fpclass(nnan x, qnan|snan|other) -> is.fpclass(x, other)

Start porting the existing combines from llvm.amdgcn.class to the
generic intrinsic. Start with the ones which aren't dependent on the
FP mode.
2023-02-05 08:36:09 -04:00
Piotr Fusik
898b5c9f5e [NFC] Fix "form/from" typos
Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D142007
2023-01-22 20:05:51 +01:00
Noah Goldstein
c9755d80a6 Transform ctpop(Pow2) -> icmp ne Pow2, 0
This makes folding to 0/1 later on easier and regardless `icmp ne` is
'probably' faster on most targets (especially for vectors).

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D142253
2023-01-21 23:03:22 -08:00
Sanjay Patel
0eedc9e567 [InstCombine] bitrev (zext i1 X) --> select X, SMinC, 0
https://alive2.llvm.org/ce/z/ZXCtgi

This breaks the infinite combine loop for issue #59897,
but we may still need more changes to avoid those loops.
2023-01-09 12:27:37 -05:00
serge-sans-paille
38818b60c5
Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part
Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955
2023-01-05 14:11:08 +01:00
Fangrui Song
fb8eb84e5f [Transforms,InstCombine] std::optional::value => operator*/operator->
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
2022-12-16 22:57:56 +00:00
Kazu Hirata
6eb0b0a045 Don't include Optional.h
These files no longer use llvm::Optional.
2022-12-14 21:16:22 -08:00
Fangrui Song
d4b6fcb32e [Analysis] llvm::Optional => std::optional 2022-12-14 07:32:24 +00:00
Kazu Hirata
e5ef6aced2 [Analysis] Use std::optional in MemoryBuiltins.cpp (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-11 01:32:26 -08:00
Kazu Hirata
f7dffc28b3 Don't include None.h (NFC)
I've converted all known uses of None to std::nullopt, so we no longer
need to include None.h.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-10 11:24:26 -08:00
Matt Arsenault
e2f56debb3 InstCombine: Fold fabs (copysign x, y) -> fabs x 2022-12-06 08:29:14 -05:00
Paul Walker
6e26ddbc7e [NFC][PatternMatch] Add helper for m_Intrinsic<Intrinsic::experimental_vector_reverse>. 2022-12-05 16:36:08 +00:00
Krzysztof Parzyszek
f3b6dbfda8 Instructions: convert Optional to std::optional 2022-12-04 14:25:11 -06:00
Kazu Hirata
343de6856e [Transforms] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 21:11:37 -08:00
Krzysztof Parzyszek
86fe4dfdb6 TargetTransformInfo: convert Optional to std::optional
Recommit: added missing "#include <cstdint>".
2022-12-02 11:42:15 -08:00
Krzysztof Parzyszek
4e12d1836a Revert "TargetTransformInfo: convert Optional to std::optional"
This reverts commit b83711248cb12639e7ef7303cfbb4452b4067e85.

Some buildbots are failing.
2022-12-02 11:34:04 -08:00
Krzysztof Parzyszek
b83711248c TargetTransformInfo: convert Optional to std::optional 2022-12-02 11:27:12 -08:00
OCHyams
fcd5098a03 [Assignment Tracking][14/*] Account for assignment tracking in instcombine
The Assignment Tracking debug-info feature is outlined in this RFC:

https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir

Most of the updates here are just to ensure DIAssignID attachments are
maintained and propagated correctly.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133307
2022-11-18 09:25:33 +00:00
Patrick Walton
01859da84b [AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().
The pointsToConstantMemory() method returns true only if the memory pointed to
by the memory location is globally invariant. However, the LLVM memory model
also has the semantic notion of *locally-invariant*: memory that is known to be
invariant for the life of the SSA value representing that pointer. The most
common example of this is a pointer argument that is marked readonly noalias,
which the Rust compiler frequently emits.

It'd be desirable for LLVM to treat locally-invariant memory the same way as
globally-invariant memory when it's safe to do so. This patch implements that,
by introducing the concept of a *ModRefInfo mask*. A ModRefInfo mask is a bound
on the Mod/Ref behavior of an instruction that writes to a memory location,
based on the knowledge that the memory is globally-constant memory (in which
case the mask is NoModRef) or locally-constant memory (in which case the mask
is Ref). ModRefInfo values for an instruction can be combined with the
ModRefInfo mask by simply using the & operator. Where appropriate, this patch
has modified uses of pointsToConstantMemory() to instead examine the mask.

The most notable optimization change I noticed with this patch is that now
redundant loads from readonly noalias pointers can be eliminated across calls,
even when the pointer is captured. Internally, before this patch,
AliasAnalysis was assigning Ref to reads from constant memory; now AA can
assign NoModRef, which is a tighter bound.

Differential Revision: https://reviews.llvm.org/D136659
2022-10-31 13:03:41 -07:00
Ahmed Bougacha
bddd9b6b91 [InstCombine] Combine ptrauth sign/resign + auth/resign intrinsics.
(sign|resign) + (auth|resign) can be folded by omitting the middle
sign+auth component if the key and discriminator match.

Differential Revision: https://reviews.llvm.org/D132383
2022-10-24 08:03:14 -07:00
Mike Hommey
86e57e66da [InstCombine] Bail out of casting calls when a conversion from/to byval is involved.
Fixes #58307

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D135738
2022-10-23 09:49:48 +02:00
Sanjay Patel
def6cbd2bd [InstCombine] add assert/test for zext to i1
This is a test to verify that we do not crash with the
problem noted in issue #57986. The root problem should
be fixed with a prior change to InstSimplify.
2022-09-26 16:01:25 -04:00
Zain Jaffal
68cc35d52c
[InstCombine] Matrix multiplication negation optimisation
If one of the operands in a matrix multiplication is negated we can optimise the equation by moving the negation to the smallest element of the operands or the result.

Reviewed By: spatel, fhahn

Differential Revision: https://reviews.llvm.org/D133300
2022-09-20 19:50:39 +01:00
Matt Arsenault
fd37ab6cf6 InstCombine: Pass AssumptionCache through isDereferenceablePointer 2022-09-19 19:10:51 -04:00