12572 Commits

Author SHA1 Message Date
Aleksandr Popov
236e6787de [IRCE] Add NSW to OverflowingBinaryOperator but not BinaryOperator
Fix incorrect setting NSW flag to non-overflowing indvar base (D154954)

Reviewed By: danilaml

Differential Revision: https://reviews.llvm.org/D156577
2023-07-30 11:14:26 +02:00
Yingwei Zheng
92a11eb32c
[ConstraintElim] Add facts implied by MinMaxIntrinsic
Fixes https://github.com/llvm/llvm-project/issues/63896 and https://github.com/rust-lang/rust/issues/113757.
This patch adds facts implied by llvm.smin/smax/umin/umax intrinsics.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D155412
2023-07-24 15:03:34 +08:00
khei4
ea72b5137e Reapply: [MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas""
This reverts commit c9d419c1df72b0160e374f8d0b9f30508b3b98a7.
Differential Revision: https://reviews.llvm.org/D153453
2023-07-24 10:16:49 +09:00
Nuno Lopes
d8e2821170 [LSR] Use poison instead of undef as placeholder [NFC]
This value is patcher afterwards, and only used temporarily during dbg info construction
2023-07-23 15:57:21 +01:00
Nuno Lopes
31d8bdbcad [Scalarizer] Fold -1 mask in shufflevector to poison instead of undef
Per latest LangRef
2023-07-23 15:02:23 +01:00
Nuno Lopes
1844d64818 [RewriteStatepointsForGC] Use poison instead of undef as placeholder [NFC]
This is used in shufflevectors where the placeholder arg is unused.
It's also used when deleting invariant_start
2023-07-22 22:25:56 +01:00
Nuno Lopes
29d0b60430 [StructurizeCFG] Use poison instead of undef as placeholder [NFC]
These are used to create branch instructions. The condition is patched later
2023-07-22 13:23:39 +01:00
khei4
c9d419c1df Revert "Reapply: [MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas"
revert because crash on chrome windows https://reviews.llvm.org/D153453#4524256

This reverts commit 569769b64858fd38f41267db41b461d3163aa754.
2023-07-22 13:08:12 +09:00
Yingwei Zheng
cce5324994
[ConstraintElim] Store the triple Pred + LHS + RHS in ReproducerEntry instead of CmpInst + Not
This patch represents a condition with `Pred + LHS + RHS` in ReproducerEntry instead of `CmpInst + Not`.
It avoids creating temporary ICmpInsts in D155412.

Reviewed By: nikic, fhahn

Differential Revision: https://reviews.llvm.org/D155782
2023-07-20 23:21:36 +08:00
khei4
569769b648 Reapply: [MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas
This reverts commit 8f3864ba4323a253bcf29825d23cd325b52c4106.
Differential Revision: https://reviews.llvm.org/D153453
2023-07-19 18:26:04 +09:00
khei4
8f3864ba43 Revert "Revert "Revert "[MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas"""
This reverts commit b02d349cbfaa81c9bbc928c4de46b12d976c1882.
2023-07-18 18:42:36 +09:00
ManuelJBrito
29b5666fdb [NewGVN] Abort PHIOfOps if singleton PHI is found
Currently we just bypass singleton phis, however we know that
in order to create the phi of ops all phis must be in the same block.
Therefore if one phi is a singleton then the rest are as well.

Differential Revision: https://reviews.llvm.org/D155478
2023-07-18 10:22:09 +01:00
Nikita Popov
9cf5254878 [llvm] Remove some uses of isOpaqueOrPointeeTypeEquals() (NFC) 2023-07-18 11:18:31 +02:00
Nuno Lopes
23c2175945 [LowerMatrixIntrinsics] Use poison instead of undef as placeholder [NFC]
These values don't propagate to the output; they are always replaced with a subsequent shuffle
or insertelement.
Tested equivalence with Alive2, e.g., https://alive2.llvm.org/ce/z/fj4s78.
2023-07-18 09:54:41 +01:00
Nick Desaulniers
dda3b70602 [ConstantHoisting] stop rematerializing InsertionPt
Reading this code, I noticed that we call findMatInsertPt a lot, for the
same inputs. Calculate it once and save the result.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D155237
2023-07-17 15:01:29 -07:00
Nick Desaulniers
bec04b4b00 [ConstantHoisting] use struct rather than tuple for adjustments
We pack this info in a tuple just to spread it back out for a function
call. Spreads in C++ are awkward.  If I want to add an additional
element to the tuple, I need to add more calls to std::get<> later. Just
use a struct.

Reviewed By: void

Differential Revision: https://reviews.llvm.org/D155236
2023-07-17 14:45:07 -07:00
Aleksandr Popov
bca5501869 [IRCE] Add NSW flag to main loop's indvar base
We have guarantees that induction variable will not overflow in the main
loop after the loop constrained. Therefore we can add no wrap flags on
its base in order not to miss info that loop is countable.

Add NSW flag now, since adding NUW flag requires a bit more complicated
analysis.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D154954
2023-07-17 01:03:52 +02:00
Nuno Lopes
68f1391a62 [ScalarizeMaskedMemIntrin] Use poison instead of undef as placeholder [NFC]
This is used for masked out lanes, that are replaced with the passthrough value
2023-07-17 10:11:14 +01:00
Nuno Lopes
89552f3a38 [InferAddressSpaces] Use poison instead of undef as placeholder [NFC]
This placeholder is only used during the execution of the algorithm, and it's patched with a
concrete value at the end
2023-07-16 22:33:09 +01:00
Nuno Lopes
f32a9c3bc6 [LoopIdiomRecognize] Use poison instead of undef as placeholder [NFC]
It was using undef as placeholder for getting the TTI cost for cttz
While at it, update the comments about cttz's last argument (is_zero_poison)
2023-07-16 22:33:09 +01:00
ManuelJBrito
ace9b6bbf5 [NewGVN] Canonicalize expressions for commutative intrinsics
Ensure that commutative intrinsics that only differ by a permutation
of their operands get the same value number by sorting the operand value
numbers.

Fixes https://github.com/llvm/llvm-project/issues/46753

Differential Revision: https://reviews.llvm.org/D155309
2023-07-16 17:24:17 +01:00
khei4
b02d349cbf Revert "Revert "[MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas""
This reverts commit 36a6eb7d12a9f827bf3d5d4e5fdc68b8a62807b2.

[MemCpyOpt] check that load/store and dest/src alloca are all in the same bb

Differential Revision: https://reviews.llvm.org/D153453
Co-authored-by: serge-sans-paille <sguelton@mozilla.com>
2023-07-15 16:27:38 +09:00
Jay Foad
9ff71814cb [EarlyCSE] Do not CSE convergent calls with memory effects
D149348 did this for readnone calls, which are handled by SimpleValue.
This patch does the same for all other CSEable calls, which are handled
by CallValue.

Differential Revision: https://reviews.llvm.org/D153151
2023-07-14 11:43:41 +01:00
Nikita Popov
06807957c5 [llvm] Remove uses of hasSameElemenTypeAs() (NFC)
Always returns true with opaque pointers.
2023-07-14 10:32:43 +02:00
Nikita Popov
61e0822efa [llvm][clang] Remove uses of isOpaquePointerTy() (NFC)
This now always returns true (for pointer types).
2023-07-14 10:27:58 +02:00
Nikita Popov
ddb46abd3c [LSR] Don't consider users of constant outside loop
In CollectLoopInvariantFixupsAndFormulae(), LSR looks at users
outside the loop. E.g. if we have an addrec based on %base, and
%base is also used outside the loop, then we have to keep it in a
register anyway, which may make it more profitable to use
%base + %idx style addressing.

This reasoning doesn't hold up when the base is a constant, because
the constant can be rematerialized. The lsr-memcpy.ll test regressed
when enabling opaque pointers, because inttoptr (i64 6442450944 to ptr)
now also has a use outside the loop (previously it didn't due to a
pointer type difference), and that extra "use" results in worse use
of addressing modes in the loop. However, the use outside the loop
actually gets rematerialized, so the alleged register saving does
not occur.

The same reasoning also applies to other types of constants, such
as global variable references.

Differential Revision: https://reviews.llvm.org/D155073
2023-07-13 12:22:38 +02:00
khei4
36a6eb7d12 Revert "[MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas"
This reverts commit 96ae0851c26237378fa1280b0a9ad713e1b72bdb.
2023-07-13 18:04:49 +09:00
khei4
96ae0851c2 [MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas
Differential Revision: https://reviews.llvm.org/D153453
2023-07-13 14:52:30 +09:00
Nick Desaulniers
cb79b5b571 [ConstantHoisting] simplify NumUses accounting NFC
The `Uses` variable is unnecessary. Just use `UsesNum` instead.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D155020
2023-07-12 09:53:46 -07:00
Nick Desaulniers
1936bb81aa [ConstantHoisting] use BasicBlock::getFirstInsertionPt NFC
Rather than open coding the same implementation.

Reviewed By: MaskRay, void

Differential Revision: https://reviews.llvm.org/D154996
2023-07-12 09:40:54 -07:00
Aleksandr Popov
cdcefd2f9a [IRCE] Implement runtime overflow check for computed range's end
Here is activated check elimination which was parsed previously in
https://reviews.llvm.org/D154069

* Added runtime check that computed range's boundary doesn't overflow in
terms of range type.

* From the statement INT_MIN <= END <= INT_MAX is inferred check:
isNonNegative(INT_MAX - END) * isNonNegative(END - INT_MIN).

* If overflow happens, check will return 0 and the safe interval will be
empty.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D154188
2023-07-12 11:19:25 +02:00
Aleksandr Popov
8b19cbfd77 Reland "[IRCE] Parse range checks in the form of 'LHS - RHS vs Limit'"
This reverts commit 4c6f95be29c6ce0f89663a5103c58ee63d76cda3

and relands e16c5c092205f68825466c25a1dd30783c4820f3

https://reviews.llvm.org/D154069
2023-07-11 18:48:14 +02:00
Nick Desaulniers
631576561e [ConstantHoisting] remove a LLVM_DEBUG statement
There is no need to print the entire function after a transform via
LLVM_DEBUG statements.  These can be emulated via:
$ llc -print-after=consthoist -filter-print-funcs=<function name>

Otherwise, this makes the output of
$ llc -debug-only=consthoist
too verbose.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D154904
2023-07-11 09:30:15 -07:00
Nikita Popov
103f1ac406 [CVP] Make select fold use-site specific (PR63756)
We may be able to replace the select with one of its select arms
at some but not all uses.

For uses in phi nodes this was already handled in the getValueOnEdge()
fold (which we can't drop entirely because it also handles an additional
case).

Fixes https://github.com/llvm/llvm-project/issues/63756.
2023-07-10 17:17:09 +02:00
Aleksandr Popov
4c6f95be29 Revert "[IRCE] Parse range checks in the form of 'LHS - RHS vs Limit'"
This reverts commit e16c5c092205f68825466c25a1dd30783c4820f3.

Revert due to Buildbot failure https://lab.llvm.org/buildbot/#/builders/193
2023-07-10 13:59:22 +02:00
Aleksandr Popov
e16c5c0922 [IRCE] Parse range checks in the form of 'LHS - RHS vs Limit'
Introduced the following range checks forms parsing:
* IV - Offset vs Limit
* Offset - IV vs Limit

Range's end boundary is computed as (Offset +/- Limit ).

If it's not possible to prove at compile time that computed upper bound
will not overflow, then scale boundary computation to a wider type to
perform overflow check at runtime.

Runtime overflow will be implemented in the next patch. In the meantime
safe range for such kind of checks isn't computed.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D154069
2023-07-10 13:00:38 +02:00
Youngsuk Kim
f69b9b7cce [llvm] Remove uses of Type::getPointerTo() (NFC)
Partial progress towards removing in-tree uses of `getPointerTo()`,
by employing the following options:

* Drop the call entirely if the sole purpose of it is to support
  a no-op bitcast (remove the no-op bitcast as well).

* Replace with `PointerType::get()`/`PointerType::getUnqual()`.

Also, remove no-op function `EmitBitCastOfLValueToProperType()`.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D154392
2023-07-08 13:05:58 -04:00
Elliot Goodrich
39d8e6e22c Add missing StringExtras.h includes
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.

This is fixing all files missed in b0abd4893fa1.

Differential Revision: https://reviews.llvm.org/D154543
2023-07-08 10:19:07 +01:00
Matt Arsenault
fa90f6b9d0 TTI: Pass function to hasBranchDivergence in a few passes
https://reviews.llvm.org/D152033
2023-07-07 09:49:38 -04:00
Arthur Eubanks
70503aeea7 [DSE] Don't eagerly optimize MemorySSA uses
Compile time improvements:
https://llvm-compile-time-tracker.com/compare.php?from=a4a2b62495a63516a4f782acff1b19361906546b&to=a408521f71702a5c5fb65077adc23413d8631cfc&stat=instructions:u

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D152744
2023-07-06 14:09:47 -07:00
Aleksandr Popov
64e289cf51 [IRCE] Support inverted range check's predicate
IRCE expects true edge of range check's branch comes to loop.
If it meets reverse case - invert the branch.

Reviewed By: skatkov

Differential Revision: https://reviews.llvm.org/D148244
2023-07-03 23:20:17 +02:00
Aleksandr Popov
e7f48d735d [NFC][IRCE] Extract 'IV vs Limit' parsing to a separate method
Next step of the preparatory refactoring for the upcoming support of new
new range check form to parse.

This change isolates logic of 'IV vs Limit' range check parsing to
simplify adding parsers for new range checks forms.

Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154160
2023-07-03 10:19:21 +02:00
Aleksandr Popov
69f99f5308 [NFC][IRCE] Check that Index is AddRec in the parseRangeCheckICmp
Next step of the preparatory refactoring for the upcoming support of
new range check form to parse.

Previous one: https://reviews.llvm.org/D154156

With this change we avoid meaningless parsing after realizing that Index
is not AddRec

Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154158
2023-07-03 10:02:42 +02:00
Florian Hahn
7f5b15ad15
[LSR] Move normalization check to normalizeForPostIncUse.
Move the logic added in 3a57152d85e1 to normalizeForPostIncUse to catch
additional un-invertable cases. This fixes another mis-compile pointed
out by @peixin in D153004.
2023-07-04 11:56:51 +01:00
Aleksandr Popov
f56e847362 [IRCE][NFC] Set Index and End together inside parseRangeCheckICmp
Preparatory refactoring for the upcoming support of new range check form
to parse.

With this change we always set Index and End values together in the same
place.

parseRangeCheckICmp specification updated.

Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154156
2023-07-03 05:21:05 +02:00
Aleksandr Popov
7e38ee6765 [NFC][IRCE] Remove dead variables 2023-07-03 02:53:25 +02:00
David Green
db32d11a38 [Reassociate] Keep flags for more unchanged operations
Reassociation destroys nsw/nuw flags from BinOps that are changed. But if the
expression at the end of a tree that was altered, but didn't change itself,
the flags do not need to be removed. For example, if %a, %b and %c are
reassociated in
  %x = add nsw i32 %a, %c
  %y = add nsw i32 %x, %b
  %z = add nsw i32 %y, %d
The value of %y and so add %y %d remains the same, and %z needn't drop the nsw
flags.
https://alive2.llvm.org/ce/z/_juAiV

Differential Revision: https://reviews.llvm.org/D154289
2023-07-03 10:05:40 +01:00
Antonio Frighetto
a2ba4e8075 [ConstraintElimination] Handle solving-only ICMP_NE predicates
Simplification of non-equality predicates for solving constraint
systems is now supported by checking the validity of related
inequalities and equalities.

Differential Revision: https://reviews.llvm.org/D152684
2023-06-29 21:22:48 +02:00
Florian Hahn
1689c357ae
[ConstraintElim] Allow and check preconditions in doesHold.
Delegate checking of the constraint & its preconditions to the existing
::isValid. This reduces duplication and allows additional optimizations
together with D152730.
2023-06-29 09:17:37 +01:00
Florian Hahn
0ad6879a62
[ConstraintElim] Try to use first cmp to prove second cmp for ANDs.
This patch extends the existing logic to handle cases where we have
branch conditions of the form (AND icmp, icmp) where the first icmp
implies the second. This can improve results in some cases, e.g. if
SimplifyCFG folded conditions from multiple branches to an AND.

The implementation handles this by adding a new type of check
(AndImpliedCheck), which are queued before conditional facts for the same
block.

When encountering AndImpliedChecks during solving, the first condition
is optimistically added to the constraint system, then we check if the
second icmp can be simplified, and finally the newly added entries are
removed.

The reason for doing things this way is to avoid clashes with signed
<-> unsigned condition transfer, which require us to re-order facts to
increase effectiveness.

Reviewed By: nikic, antoniofrighetto

Differential Revision: https://reviews.llvm.org/D151799
2023-06-28 13:19:41 +01:00