Removes DependenceInfo::getRuntimeAssumptions(), DependenceInfo::Assumptions,
and the print of "Runtime Assumptions:". The runtime assumptions are still
properly attached to each Dependence result and printed as part of the
per-dependence output.
This patch moves the validation logic of delinearization results from DA
to Delinearization. Also call it in `printDelinearization` to test its
behavior. The motivation is as follows:
- Almost the same code exists in `tryDelinearizeFixedSize` and
`tryDelinearizeParametricSize`. Consolidating it in Delinearization
avoids code duplication.
- Currently this validation logic is not well tested. Moving it to
Delinearization allows us to write regression tests easily.
This patch changes the test outputs and debug messages, but otherwise
NFCI.
This patch replaces the delinearization function used in DA, switching
from one that depends on type information in GEPs to one that does not.
There are three types of changes in regression tests: improvements,
degradations, and degradations but the related features will be
removed. Since there were very few cases that are classified into the
second category, I believe the impact of this change should be
practically insignificant.
This patch fixes the unexpected result in monotonicity check for
`@step_is_variant` in `monotonicity-no-wrap-flags.ll`. Currently, the
SCEV is considered non-monotonic if it contains an expression which is
neither loop-invariant nor an affine addrec. In `@step_is_variant`, the
`offset_i` satisfies this condition, but `offset_i + j` was classified
as monotonic.
The root cause is that a non-outermost loop was passed to monotonicity
checker instead of the outermost one. This patch ensures that the
correct outermost loop is passed.
Rely on the product of `UpperBound` and `AbsCoeff` only if SCEV
can prove that there is no overflow. Also the same about the result
of the subtraction of `DstConst` from `SrcConst` to calculate `Delta`.
Given a `SCEVMulExpr` such as `5 * %m`, `gcdMIVtest` in DA assumes the
value as a multiple of 5 in a mathematical sense. However, this is not
necessarily true if `5 * %m` overflows, especially because an odd number
has an inverse modulo `2^64`. Such incorrect assumptions can lead to
invalid analysis results.
This patch stops unconditionally extracting a constant operand from
`SCEVMulExpr`. Instead, it only allows this when the `SCEVMulExpr` has
the `nsw` flag.
The type sizes of backedge taken counts for two loops can be different
and this is to fix the crash in haveSameSD
(https://github.com/llvm/llvm-project/issues/165014).
---------
Co-authored-by: Shimin Cui <scui@xlperflep9.rtp.raleigh.ibm.com>
This patch fixes the computation of the absolute value for SCEV.
Previously, it was calculated as `AbsX = SE->isKnownNonNegative(X) ? X :
-X`, which would incorrectly assume that `!isKnownNonNegative` implies
`isKnownNegative`. This assumption does not hold in general, for
example, when `X` is a `SCEVUnknown` and it can be an arbitrary value.
To compute the absolute value properly, we should use
ScalarEvolution::getAbsExpr instead.
Fix#149977
PR #157084 added an option `da-run-siv-routines-only` to run only SIV
routines in DA. This PR replaces that option with a more fine-grained
one that allows to select other than SIV routines as well. This option
is useful for regression testing of individual DA routines. This patch
also reorganizes regression tests that use `da-run-siv-routines-only`.
The dependence testing functions in DA assume that the analyzed AddRec
does not wrap over the entire iteration space. For AddRecs that may
wrap, DA should conservatively return unknown dependence. However, no
validation is currently performed to ensure that this condition holds,
which can lead to incorrect results in some cases.
This patch introduces the notion of *monotonicity* and a validation
logic to check whether a SCEV is monotonic. The monotonicity check
classifies the SCEV into one of the following categories:
- Unknown: Nothing is known about the monotonicity of the SCEV.
- Invariant: The SCEV is loop-invariant.
- MultivariateSignedMonotonic: The SCEV doesn't wrap in a signed sense
for any iteration of the loops in the loop nest.
The current validation logic basically searches an affine AddRec
recursively and checks whether the `nsw` flag is present. Notably, it is
still unclear whether we should also have a category for unsigned
monotonicity.
The monotonicity check is still under development and disabled by
default for now. Since such a check is necessary to make DA sound, it
should be enabled by default once the functionality is sufficient.
Split off from #154527.
Bail out when the backedge taken count is a CouldNotCompute SCEV in
function isKnownLessThan; we cannot and do not want to query things like
its Type.
Fixes#159979
This is a follow-up PR for post-commit comments in #128782 .
- Fix variable name to camel case.
- Change the output format to make it easier to handle with FileCheck.
- Regenerate assertions of regression tests.
When building an assert-enabled target, silence the following:
```
C:\git\llvm-project\llvm\include\llvm/Analysis/DependenceAnalysis.h(290): warning C4018: '<=': signed/unsigned mismatch
```
When there is a dependency between two memory instructions in separate loops that have the same iteration space and depth, SIV will be able to test them and compute the direction and the distance of the dependency.
This patch adds an overflow check to the `exactSIVtest` function to fix
the issue demonstrated in the test case added in #157085. This patch
only fixes one of the routines. To fully resolve the test case, the
other functions need to be addressed as well.
This patch introduces a new option, `da-run-siv-routines-only`, which
runs only the SIV family routines in the DA. This is useful for testing
(regression tests, not dependence tests) as it helps detect behavioral
changes in the SIV routines. Actually, regarding the test cases added in
#157085, fixing the incorrect result requires changes across multiple
functions (at a minimum, `exactSIVtest`, `gcdMIVtest` and
`symbolicRDIVtest`). It is difficult to address all of them at once.
This patch also generates the CHECK directives using the new option for
`ExactSIV.ll` as it is necessary for subsequent patches. However, I
believe it will also be useful for other `xxSIV.ll` tests. Notably, the
SIV family routines tend to be affected by other routines, as they are
typically invoked at the beginning of the overall analysis.
This patch removes base pointers from subscripts when delinearization
fails. Previously, in such cases, the pointer type SCEVs were used
instead of offset SCEVs derived from them.
For example, here is a portion of the debug output when analyzing
`strong0` in `test/Analysis/DependenceAnalysis/StrongSIV.ll`:
```
testing subscript 0, SIV
src = {(8 + %A),+,4}<nuw><%for.body>
dst = {(8 + %A),+,4}<nuw><%for.body>
Strong SIV test
Coeff = 4, i64
SrcConst = (8 + %A), ptr
DstConst = (8 + %A), ptr
Delta = 0, i64
UpperBound = (-1 + %n), i64
Distance = 0
Remainder = 0
```
As shown above, the `SrcConst` and `DstConst` are pointer values rather
than integer offsets. `%A` should be removed.
This change is necessary for #157086, since
`ScalarEvolution::willNotOverflow` expects integer type SCEVs as
arguments. This change alone alone should not affect the analysis
results.
This patch refactors `exactSIVtest` and `exactRDIVtest` by consolidating
duplicated logic into a single function. Same as #152688, the main goal
is to improve code maintainability, since extra validation logic (as
written in TODO comments) may be necessary.
This patch refactors `gcdMIVtest` by consolidating duplicated logic into
a single function. The main goal of this change is to improve code
maintainability rather than readability, especially since we may need to
revise this logic for correctness (as noted in the added TODO comments).
I hope this patch is NFC, but I've also added several new assertions,
which may cause some previously passing cases to fail.
Delinearization provides two values: the size of the array, and the
subscript of the access. DA checks their validity (`0 <= subscript <
size`), with some special handling. In particular, to ensure `subscript
< size`, calculate the maximum value of `subscript - size` and check if
it is negative. There was an issue in its process: when `subscript -
size` is expressed as an affine format like `init + step * i`, the value
in the last iteration (`start + step * (num_iterations - 1)`) was
assumed to be the maximum value. This assumption is incorrect in the
following cases:
- When `step` is negative
- When the AddRec wraps
This patch introduces extra checks to ensure the sign of `step` and
verify the existence of nsw/nuw flags.
Also, `isKnownNonNegative(S - smax(1, Size))` was used as a regular
check, which is incorrect when `Size` is negative. This patch also
replace it with `isKnownNonNegative(S - Size)`, although it's still
unclear whether using `isKnownNonNegative` is appropriate in the first
place.
Fix#150604
As specified in #53942, DA assumes base pointer invariance in its
process. Some cases were fixed by #116628. However, that PR only
addressed the parts related to AliasAnalysis, so the original issue
persists in later stages, especially when the AliasAnalysis results in
`MustAlias`.
This patch insert an explicit loop-invariant checks for the base pointer
and skips analysis when it is not loop-invariant.
Fix the cases added in #148240.
DependenceAnalysis checks whether the given addresses are divisible by
the element size of corresponding load/store instructions. However, this
check was only executed when the two instructions (Src and Dst) are
different. We must also perform the same check when Src and Dst are the
same instruction.
Fix the test added in #147715.
A Dependence object has two entries: Distance and Direction. The former
represents the distance of the dependence, while the latter
characterizes the distance by whether the value of it is positive,
negative, or zero (especially, zero is represented by EQ in DA). So it
is expected that the Distance equals zero iff the Direction is EQ.
However, this condition was not satisfied in some cases.
This patch adds a logic to set the Distance to zero if the Direction is
EQ. Although it is ideal that the Distance is updated to zero
simultaneously when the Direction is set to EQ, achieving it would
require changing the entire code in DA.
This patch corrects the behavior of the Dependence Analysis for memory
accesses that do not start at the same offset or do not have similar
strides. When offsets or strides cannot be disambiguated at compile
time, DA collects a set of runtime assumptions under which the
dependence test becomes valid. The default remains the same as before
the patch: DA rejects the dependence test as undecidable instead of
collecting runtime assumptions.
---------
Co-authored-by: Michael Kruse <github@meinersbur.de>
Co-authored-by: Ryotaro Kasuga <kasuga.ryotaro@fujitsu.com>
Parameter PossiblyLoopIndependent has lost its intended purpose. This
flag is always set to true in all cases when depends() is called, hence
we want to reconsider the utility of this variable and remove it from
the function signature entirely. This is an NFC patch.
This patch fixes two bugs:
https://github.com/llvm/llvm-project/issues/41488https://github.com/llvm/llvm-project/issues/53942
The dependence analysis assumes that the base address of array accesses
is invariant across loop iterations. In both bugs the base address
evolves following loop iterations: the base address flip-flops between
two different memory objects.
The patch uses the cross iteration mode of alias analysis to disambiguate the
base objects.
Modify the DA pretty printer to match the output of other analysis
passes. This enables update_analyze_test_checks.py to also work on DA
tests. Auto generate all the Dependence Analysis tests.
This patch is the first of the two-patch series (D130188, D130179) that
resolve PR56275 (https://github.com/llvm/llvm-project/issues/56275)
which is a missed opportunity, where a perfrectly valid case for loop
interchange failed interchange legality.
If the distance/direction vector produced by dependence analysis (DA) is
negative, it needs to be normalized (reversed). This patch provides helper
functions `isDirectionNegative()` and `normalize()` in DA that does the
normalization, and clients can query DA to do normalization if needed.
A pass option `<normalized-results>` is added to DependenceAnalysisPrinterPass,
and we leverage it to update DA test cases to make sure of test coverage. The
test cases added in `Banerjee.ll` shows that negative vectors are normalized
with `print<da><normalized-results>`.
Reviewed By: bmahjour, Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D130188
This is a follow-up patch to D122857 where we added delinearization of
fixed-size arrays to loop cache analysis, which resulted in some duplicate
code, i.e., "tryDelinearizeFixedSize()", in LoopCacheCost.cpp and
DependenceAnalysis.cpp. Refactoring is done in this patch.
This patch refactors out the main logic of "tryDelinearizeFixedSize()" as
"tryDelinearizeFixedSizeImpl()" and moves it to Delinearization.cpp, such that
clients can reuse "llvm::tryDelinearizeFixedSizeImpl()" wherever they would
like to delinearize fixed-size arrays. Currently it has two users, i.e.,
DependenceAnalysis.cpp and LoopCacheCost.cpp.
Reviewed By: Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D124745
To represent various loop levels within a nest, DA implements a special
numbering scheme (see comment atop establishNestingLevels). The goal of
this numbering scheme appears to be representing each unique loop
distinctively by using as little memory as possible. This numbering
scheme is simple when the source and destination of the dependence are
in the same loop. In such cases the level is simply the depth of the
loop in which src and dst reside. When the src and dst are not in the
same loop, we could run into the following situation exposed by
https://reviews.llvm.org/D71539. This patch fixes this by detecting
such cases in checkSubscripts and treating them as non-linear/non-affine.
Reviewed By: Meinersbur
Differential Revision: https://reviews.llvm.org/D110973
Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the error has been removed,
cl::ZeroOrMore is unneeded.
Also remove cl::init(false) while touching the lines.
Refactor from iteratively using BitCastInst::getOperand()
to using stripPointerCasts() instead. This is an improvement
since now we are able to analyze more cases, please refer
to test cases added in this patch.
Reviewed By: Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D123559
