211 Commits

Author SHA1 Message Date
Bjorn Pettersson
a20f7efbc5 Remove several no longer needed includes. NFCI
Mostly removing includes of InitializePasses.h and Pass.h in
passes that no longer has support for the legacy PM.
2023-04-17 13:54:19 +02:00
Arthur Eubanks
7c3c981442 [Passes] Remove some legacy passes
DFAJumpThreading
JumpThreading
LibCallsShrink
LoopVectorize
SLPVectorizer
DeadStoreElimination
AggressiveDCE
CorrelatedValuePropagation
IndVarSimplify

These are part of the optimization pipeline, of which the legacy version is deprecated and being removed.
2023-03-10 17:17:00 -08:00
Kazu Hirata
4a05edd410 [llvm] Use APInt::getZero instead of APInt::getNullValue (NFC)
Note that APInt::getNullValue has been soft-deprecated in favor of
APInt::getZero.
2023-02-19 22:42:01 -08:00
Kazu Hirata
dc6238e035 [Scalar] Remove an unused local varable (NFC)
The last uses of CRs were removed on Jan 17, 2023 in commit
61bb549cfd43fe4753fc78075b626ae18106fbeb.
2023-01-29 10:03:15 -08:00
Roman Lebedev
43a59be45d
[CVP] Expand bound udiv's, symmetrically with urem's
Symmetrical with the `urem` case, added in 66efb986322b206834e7c9e1eb777fa053912c39.

Simple case: https://alive2.llvm.org/ce/z/gRumLd / https://alive2.llvm.org/ce/z/rxEeC5
Second variant of precondition: https://alive2.llvm.org/ce/z/cAm9TD
2023-01-20 21:52:16 +03:00
Nikita Popov
61bb549cfd [CVP] Avoid duplicate range calculation (NFC)
Calculate the range once for all the sdiv/srem transforms.
2023-01-17 16:54:51 +01:00
Nikita Popov
004e613ce4 [CVP] Avoid duplicate range calculation (NFC)
Calculate the range once and use it in processURem() and
narrowUDivOrURem().
2023-01-17 16:39:27 +01:00
Nikita Popov
a444fe07dd [CVP] Handle use-site conditions in domain-based folds
As a side-effect, this switchem them to use getConstantRange() rather
than getPredicateAt(). getPredicateAt() is not supposed to be more
powerful than getConstantRange() for non-equality comparisons (as
long as block values are used).
2023-01-17 16:35:18 +01:00
Nikita Popov
5c38c6a3aa [CVP] Handle use-site conditions in more folds 2023-01-17 16:14:55 +01:00
Nikita Popov
d34537a732 [CVP] Handle use-site conditions in urem folds 2023-01-13 15:45:34 +01:00
Nikita Popov
0daa1142ea [CVP] Avoid duplicate range fetch (NFC)
In preparation for switching this to use getConstantRangeAtUse().
2023-01-13 14:56:07 +01:00
Nikita Popov
4f772b0955 [LVI][CVP] Make use of condition known at use
When an instruction is only used in a select or phi operand, we might
be able to make use of additional information from the select/branch
condition. For example in

  %sub = call i16 @llvm.usub.sat.i16(i16 %x, i16 10)
  %cmp = icmp uge i16 %x, 10
  %sel = select i1 %cmp, i16 %sub, i16 42

the usub.sat is only used in a select where %x uge 10 is known to
hold, so we can fold it based on that knowledge.

This addresses the regression reported at
https://reviews.llvm.org/D140798#4039748, but also provides a
solution to a recurring problem we've had, where we fail to make
use of range information after a branch+phi has been converted
into a select. Our current solution to this is to hope that IPSCCP
can perform the fold before that happens, but handling this in LVI
is a somewhat more general solution.

Currently we only make use of this for the willNotOverflow() fold,
but I plan to adjust other folds to use the new API as well.

Differential Revision: https://reviews.llvm.org/D141482
2023-01-12 16:41:31 +01:00
luxufan
5b25a0bcb1 [CVP] Simplify SRem when constantrange abs(lhs) < abs(rhs)
For `srem x, y`, if abs(constant range of x) less than abs(constant
range of y), we can simplify it as:
`srem x, y => x` if y is guaranteed to be positive.
'srem x, y => -x' if y is guaranteed to be negative.

Differential Revision: https://reviews.llvm.org/D140405
2023-01-03 22:51:12 +08:00
Roman Lebedev
08c2f4eb7a
[CVP] When expanding urem, always freeze the nominator
As per the post-commit feedback - that was not the correct precondition
to avoid it here. I think we should generally start changing mentality
about `freeze`, the fact that we have been conditioned to be afraid of it
(or of anything in LLVM in general) is the key problem here.
2022-12-31 05:00:43 +03:00
Roman Lebedev
66efb98632
[CVP] Expand bound urems
This kind of thing happens really frequently in LLVM's very own
shuffle combining methods, and it is even considered bad practice
to use `%` there, instead of using this expansion directly.
Though, many of the cases there have variable divisors,
so this won't help everything.

Simple case: https://alive2.llvm.org/ce/z/PjvYf-
There's alternative expansion via `umin`:
https://alive2.llvm.org/ce/z/hWCVPb

BUT while we can transform the first expansion
into the `umin` one (e.g. for SCEV):
https://alive2.llvm.org/ce/z/iNxKmJ
... we can't go in the opposite direction.

Also, the non-`umin` expansion seems somewhat more codegen-friendly:
https://godbolt.org/z/qzjx5bqWK
https://godbolt.org/z/a7bj1axbx

There's second variant of precondition:
https://alive2.llvm.org/ce/z/zE6cbM
but there the numerator must be non-undef / must be frozen.
2022-12-30 19:40:46 +03:00
Roman Lebedev
3cb827f9d3
[NFC][CVP] processURem(): add statistic and increase readability 2022-12-30 19:40:46 +03:00
Joshua Cao
cca01df291 [CVP] Eliminate urem when LHS < RHS
Fol `X % Y -> X` when we can determine `X < Y` based on constant
range information.

Fixes https://github.com/llvm/llvm-project/issues/58408.

Differential Revision: https://reviews.llvm.org/D138360
2022-12-13 11:40:44 +01:00
Fangrui Song
a996cc217c Remove unused #include "llvm/ADT/Optional.h" 2022-12-05 06:31:11 +00:00
Kazu Hirata
edace862f6 [Scalar] Use std::optional in CorrelatedValuePropagation.cpp (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-25 23:52:49 -08:00
Nikita Popov
68b24c3b44 [CVP] Simplify comparisons without constant operand
CVP currently only tries to simplify comparisons if there is a
constant operand. However, even if both are non-constant, we may
be able to determine the result of the comparison based on range
information.

IPSCCP is already capable of doing this, but because it runs very
early, it may miss some cases.

Differential Revision: https://reviews.llvm.org/D137253
2022-11-03 15:35:27 +01:00
Yuanbo Li
ebd0249fcf [DebugInfo] Missing debug location after replacement in processSRem function
This patch fixes an issue in which CorrelatedValuePropagation::processSRem
would create new instructions to represent the SRem instruction, but would not
correctly copy any existing debug location metadata to the new instruction.

Differential Revision: https://reviews.llvm.org/D132218
2022-09-01 13:18:17 +01:00
Kazu Hirata
0e37ef0186 [Transforms] Fix comment typos (NFC) 2022-08-07 23:55:24 -07:00
Nuno Lopes
373571dbb4 [NFC] Switch a few uses of undef to poison as placeholders for unreachble code 2022-06-30 23:01:43 +01:00
Simon Moll
b8c2781ff6 [NFC] format InstructionSimplify & lowerCaseFunctionNames
Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName".  This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.

This is the alternative to the less invasive clang-format only patch: D126783

Reviewed By: spatel, rengolin

Differential Revision: https://reviews.llvm.org/D126889
2022-06-09 16:10:08 +02:00
Jay Foad
6bec3e9303 [APInt] Remove all uses of zextOrSelf, sextOrSelf and truncOrSelf
Most clients only used these methods because they wanted to be able to
extend or truncate to the same bit width (which is a no-op). Now that
the standard zext, sext and trunc allow this, there is no reason to use
the OrSelf versions.

The OrSelf versions additionally have the strange behaviour of allowing
extending to a *smaller* width, or truncating to a *larger* width, which
are also treated as no-ops. A small amount of client code relied on this
(ConstantRange::castOp and MicrosoftCXXNameMangler::mangleNumber) and
needed rewriting.

Differential Revision: https://reviews.llvm.org/D125557
2022-05-19 11:23:13 +01:00
Craig Topper
4b36d9bde7 [CVP] Preserve exact name when converting sext->zext and ashr->lshr.
Previously we took the old name and always appended a numberic suffix.
Since we're doing a 1:1 replacement, it's clearer to keep the original
name exactly.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D125281
2022-05-10 09:13:59 -07:00
serge-sans-paille
59630917d6 Cleanup includes: Transform/Scalar
Estimated impact on preprocessor output line:
before: 1062981579
after:  1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817
2022-03-03 07:56:34 +01:00
Dmitry Vassiliev
90a3b31091 [Transforms] Enhance CorrelatedValuePropagation to handle both values of select
The "Correlated Value Propagation" pass was missing a case when handling select instructions. It was only handling the "false" constant value, while in NVPTX the select may have the condition (and thus the branches) inverted, for example:
```
loop:
	%phi = phi i32* [ %sel, %loop ], [ %x, %entry ]
	%f = tail call i32* @f(i32* %phi)
	%cmp1 = icmp ne i32* %f, %y
	%sel = select i1 %cmp1, i32* %f, i32* null
	%cmp2 = icmp eq i32* %sel, null
	br i1 %cmp2, label %return, label %loop
```
But the select condition can be inverted:
```
	%cmp1 = icmp eq i32* %f, %y
	%sel = select i1 %cmp1, i32* null, i32* %f
```
The fix is to enhance "Correlated Value Propagation" to handle both branches of the select instruction.

Reviewed By: nikic, lebedev.ri

Differential Revision: https://reviews.llvm.org/D119643
2022-02-23 00:11:20 +04:00
Nikita Popov
18bf42c0a6 [CVP] Extract helper from phi processing (NFC)
So we can use early returns and avoid those awkward !V checks.
2022-02-14 10:51:34 +01:00
Fabian Wolff
45ecfed6c6 [CVP] Remove ashr of -1 or 0
Fixes PR#52190. There is already a check for converting ashr instructions with non-negative left-hand sides into lshr; this patch adds an optimization to remove ashr altogether if the left-hand side is known to be in the range [-1, 1).

Differential Revision: https://reviews.llvm.org/D113835
2021-11-29 15:32:09 -08:00
Artur Pilipenko
aa60d169ea [CVP] Add a cl::opt for canonicalization of signed relational comparisons
This canonicalization breaks the ability to discard checks in some cases.
Add a command line option to disable it. This option is on by default,
so the change is NFC.

See for details:
https://reviews.llvm.org/D112895#3149487
2021-11-24 13:52:38 -08:00
Jun Ma
07333810ca Revert "Revert "Revert "Recommit "Revert "[CVP] processSwitch: Remove default case when switch cover all possible values."""""
This reverts commit c93f93b2e3f28997f794265089fb8138dd5b5f13.
2021-11-24 10:26:37 +08:00
Zarko Todorovski
0d3add216f [llvm][NFC] Inclusive language: Reword replace uses of sanity in llvm/lib/Transform comments and asserts
Reworded some comments and asserts to avoid usage of `sanity check/test`

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D114372
2021-11-23 13:22:55 -05:00
Roman Lebedev
b554e41e2d
[CVP] Canonicalize signed relational comparisons of scalar integers to unsigned comparison predicates
Now that the reasoning was added to ConstantRange in D90924,
this replicates IndVars variant of this transform (D111836)
in a pass that uses value range reasoning for the transform.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D112895
2021-11-01 12:16:05 +03:00
Jun Ma
c93f93b2e3 Revert "Revert "Recommit "Revert "[CVP] processSwitch: Remove default case when switch cover all possible values.""""
This reverts commit 3a998c06a8e93989319238e12b56a731198cc1c2.
2021-11-01 15:31:59 +08:00
Jun Ma
3a998c06a8 Revert "Recommit "Revert "[CVP] processSwitch: Remove default case when switch cover all possible values."""
This reverts commit 8ba2adcf9e54b34ba8efa73ac0d81a1192e4f614.
2021-09-27 20:39:05 +08:00
Kazu Hirata
8e86c0e4f4 [Scalar] Use make_early_inc_range (NFC) 2021-09-12 08:17:18 -07:00
Chris Lattner
735f46715d [APInt] Normalize naming on keep constructors / predicate methods.
This renames the primary methods for creating a zero value to `getZero`
instead of `getNullValue` and renames predicates like `isAllOnesValue`
to simply `isAllOnes`.  This achieves two things:

1) This starts standardizing predicates across the LLVM codebase,
   following (in this case) ConstantInt.  The word "Value" doesn't
   convey anything of merit, and is missing in some of the other things.

2) Calling an integer "null" doesn't make any sense.  The original sin
   here is mine and I've regretted it for years.  This moves us to calling
   it "zero" instead, which is correct!

APInt is widely used and I don't think anyone is keen to take massive source
breakage on anything so core, at least not all in one go.  As such, this
doesn't actually delete any entrypoints, it "soft deprecates" them with a
comment.

Included in this patch are changes to a bunch of the codebase, but there are
more.  We should normalize SelectionDAG and other APIs as well, which would
make the API change more mechanical.

Differential Revision: https://reviews.llvm.org/D109483
2021-09-09 09:50:24 -07:00
Jun Ma
8ba2adcf9e Recommit "Revert "[CVP] processSwitch: Remove default case when switch cover all possible values.""
Differential Revision: https://reviews.llvm.org/D106056
2021-09-09 16:53:33 +08:00
Sanjay Patel
ec54e275f5 Revert "[CVP] processSwitch: Remove default case when switch cover all possible values."
This reverts commit 9934a5b2ed5aa6e6bbb2e55c3cd98839722c226e.
This patch may cause miscompiles because it missed a constraint
as shown in the examples from:
https://llvm.org/PR51531
2021-08-19 08:43:51 -04:00
Jun Ma
9934a5b2ed [CVP] processSwitch: Remove default case when switch cover all possible values.
Differential Revision: https://reviews.llvm.org/D106056
2021-08-18 10:23:13 +08:00
Nikita Popov
9c91614959 [CVP] Guard against poison in common phi value transform (PR50399)
The common phi value transform replaces constants with values that
have the same value as the constant on a given edge. However, LVI
generally only provides information that is correct up to poison,
so this can end up replacing a well-defined value with poison.
D69442 addressed an instance of this problem by clearing poison
flags on the generating instruction, which was sufficient at the
time. rGa917fb89dc28 made LVI's edge value analysis slightly more
powerful, and clearing poison flags is no longer sufficient.

This patch changes the transform to instead explicitly guard against
a poison value instead. This should be satisfied for most cases due
to a prior branch on poison.

Fixes https://bugs.llvm.org/show_bug.cgi?id=50399.

Differential Revision: https://reviews.llvm.org/D102966
2021-05-25 20:47:17 +02:00
Arthur Eubanks
6b9524a05b [NewPM] Don't mark AA analyses as preserved
Currently all AA analyses marked as preserved are stateless, not taking
into account their dependent analyses. So there's no need to mark them
as preserved, they won't be invalidated unless their analyses are.

SCEVAAResults was the one exception to this, it was treated like a
typical analysis result. Make it like the others and don't invalidate
unless SCEV is invalidated.

Reviewed By: asbirlea

Differential Revision: https://reviews.llvm.org/D102032
2021-05-18 13:49:03 -07:00
Evgeniy Brevnov
35e95c6817 [CVP] processCallSite returns wrong status
Recently processMinMaxIntrinsic has been added and we started to observe a number of analysis get invalidated after CVP. The problem is CVP conservatively returns 'true'  even if there were no modifications to IR. I found one more place besides processMinMaxIntrinsic  which has the same problem. I think processMinMaxIntrinsic and similar should better have boolean return status to prevent similar issue reappear in future.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D100538
2021-04-19 12:13:22 +07:00
Roman Lebedev
9829f5e6b1
[CVP] @llvm.[us]{min,max}() intrinsics handling
If we can tell that either one of the arguments is taken,
bypass the intrinsic.

Notably, we are indeed fine with non-strict predicate:
* UL: https://alive2.llvm.org/ce/z/69qVW9 https://alive2.llvm.org/ce/z/kNFTKf
      https://alive2.llvm.org/ce/z/AvaPw2 https://alive2.llvm.org/ce/z/oxo53i
* UG: https://alive2.llvm.org/ce/z/wxHeGH https://alive2.llvm.org/ce/z/Lf76qx
* SL: https://alive2.llvm.org/ce/z/hkeTGS https://alive2.llvm.org/ce/z/eR_b-W
* SG: https://alive2.llvm.org/ce/z/wEqRm7 https://alive2.llvm.org/ce/z/FpAsVr

Much like with all other comparison handling in CVP,
while we could sort-of handle two Value's,
at least for plain ICmpInst it does not appear to be worthwhile.

This only fires 78 times on test-suite + dt + rs,
but we don't canonicalize to these yet. (only SCEV produces them)
2021-04-11 00:33:47 +03:00
Roman Lebedev
a407738def
[NFC][CVP] Add statistic for function pointer argument non-null-ness deduction 2021-04-10 21:23:20 +03:00
Roman Lebedev
fe7b3ad8d5
[CVP] LVI: Use in-block values when checking value signedness domain
This has a huge positive impact on all the folds that use these helpers,
as it can be seen on vanilla test-suite + rawspeed + darktable:
correlated-value-propagation.NumSRems             +75.68% (+ 28)
correlated-value-propagation.NumAShrs             +63.87% (+198)
correlated-value-propagation.NumSDivs             +49.42% (+127)
correlated-value-propagation.NumSExt              + 8.85% (+593)
correlated-value-propagation.NumUDivURemsNarrowed	+ 8.65% (+34)

... while having pretty minimal compile-time impact:
https://llvm-compile-time-tracker.com/compare.php?from=e8c7f43e2c2c6f3581ec1c6489ec21ad9f98958a&to=4cd197711e58ee1b2faeee0c35eea54540185569&stat=instructions
2021-04-10 21:10:59 +03:00
Roman Lebedev
257eda0794
[NFC][LVI] getPredicateAt(): drop default value for UseBlockValue
The default is likely wrong.
Out of all the callees, only a single one needs to pass-in false (JumpThread),
everything else either already passes true, or should pass true.

Until the default is flipped, at least make it harder to unintentionally
add new callees with UseBlockValue=false.
2021-04-10 20:46:01 +03:00
Roman Lebedev
c329a47d9e
[CVP] @llvm.abs() handling
Iff we know the sigdness domain of the argument,
we can either skip @llvm.abs, or do negation directly.

Notably, INT_MIN can belong to either domain:
* X u<= INT_MIN --> X  is always fine
  https://alive2.llvm.org/ce/z/QB8j-C https://alive2.llvm.org/ce/z/7sFKpS
* X s<= 0 --> -X  is always fine
  https://alive2.llvm.org/ce/z/QbGSyq https://alive2.llvm.org/ce/z/APsN84

If all else fails, try to inferr NSW flag:
https://alive2.llvm.org/ce/z/qCJfYm
2021-04-10 16:47:31 +03:00
Nikita Popov
2b494f85f1 [CVP] Remove -cvp-dont-add-nowrap-flags option
This option was originally added to work around a bug in LFTR.
The bug has long since been fixed.
2021-03-07 18:19:31 +01:00