This patch adds an overflow check to the `exactSIVtest` function to fix
the issue demonstrated in the test case added in #157085. This patch
only fixes one of the routines. To fully resolve the test case, the
other functions need to be addressed as well.
This patch introduces a new option, `da-run-siv-routines-only`, which
runs only the SIV family routines in the DA. This is useful for testing
(regression tests, not dependence tests) as it helps detect behavioral
changes in the SIV routines. Actually, regarding the test cases added in
#157085, fixing the incorrect result requires changes across multiple
functions (at a minimum, `exactSIVtest`, `gcdMIVtest` and
`symbolicRDIVtest`). It is difficult to address all of them at once.
This patch also generates the CHECK directives using the new option for
`ExactSIV.ll` as it is necessary for subsequent patches. However, I
believe it will also be useful for other `xxSIV.ll` tests. Notably, the
SIV family routines tend to be affected by other routines, as they are
typically invoked at the beginning of the overall analysis.
This patch removes base pointers from subscripts when delinearization
fails. Previously, in such cases, the pointer type SCEVs were used
instead of offset SCEVs derived from them.
For example, here is a portion of the debug output when analyzing
`strong0` in `test/Analysis/DependenceAnalysis/StrongSIV.ll`:
```
testing subscript 0, SIV
src = {(8 + %A),+,4}<nuw><%for.body>
dst = {(8 + %A),+,4}<nuw><%for.body>
Strong SIV test
Coeff = 4, i64
SrcConst = (8 + %A), ptr
DstConst = (8 + %A), ptr
Delta = 0, i64
UpperBound = (-1 + %n), i64
Distance = 0
Remainder = 0
```
As shown above, the `SrcConst` and `DstConst` are pointer values rather
than integer offsets. `%A` should be removed.
This change is necessary for #157086, since
`ScalarEvolution::willNotOverflow` expects integer type SCEVs as
arguments. This change alone alone should not affect the analysis
results.
This patch refactors `exactSIVtest` and `exactRDIVtest` by consolidating
duplicated logic into a single function. Same as #152688, the main goal
is to improve code maintainability, since extra validation logic (as
written in TODO comments) may be necessary.
This patch refactors `gcdMIVtest` by consolidating duplicated logic into
a single function. The main goal of this change is to improve code
maintainability rather than readability, especially since we may need to
revise this logic for correctness (as noted in the added TODO comments).
I hope this patch is NFC, but I've also added several new assertions,
which may cause some previously passing cases to fail.
Delinearization provides two values: the size of the array, and the
subscript of the access. DA checks their validity (`0 <= subscript <
size`), with some special handling. In particular, to ensure `subscript
< size`, calculate the maximum value of `subscript - size` and check if
it is negative. There was an issue in its process: when `subscript -
size` is expressed as an affine format like `init + step * i`, the value
in the last iteration (`start + step * (num_iterations - 1)`) was
assumed to be the maximum value. This assumption is incorrect in the
following cases:
- When `step` is negative
- When the AddRec wraps
This patch introduces extra checks to ensure the sign of `step` and
verify the existence of nsw/nuw flags.
Also, `isKnownNonNegative(S - smax(1, Size))` was used as a regular
check, which is incorrect when `Size` is negative. This patch also
replace it with `isKnownNonNegative(S - Size)`, although it's still
unclear whether using `isKnownNonNegative` is appropriate in the first
place.
Fix#150604
As specified in #53942, DA assumes base pointer invariance in its
process. Some cases were fixed by #116628. However, that PR only
addressed the parts related to AliasAnalysis, so the original issue
persists in later stages, especially when the AliasAnalysis results in
`MustAlias`.
This patch insert an explicit loop-invariant checks for the base pointer
and skips analysis when it is not loop-invariant.
Fix the cases added in #148240.
DependenceAnalysis checks whether the given addresses are divisible by
the element size of corresponding load/store instructions. However, this
check was only executed when the two instructions (Src and Dst) are
different. We must also perform the same check when Src and Dst are the
same instruction.
Fix the test added in #147715.
A Dependence object has two entries: Distance and Direction. The former
represents the distance of the dependence, while the latter
characterizes the distance by whether the value of it is positive,
negative, or zero (especially, zero is represented by EQ in DA). So it
is expected that the Distance equals zero iff the Direction is EQ.
However, this condition was not satisfied in some cases.
This patch adds a logic to set the Distance to zero if the Direction is
EQ. Although it is ideal that the Distance is updated to zero
simultaneously when the Direction is set to EQ, achieving it would
require changing the entire code in DA.
This patch corrects the behavior of the Dependence Analysis for memory
accesses that do not start at the same offset or do not have similar
strides. When offsets or strides cannot be disambiguated at compile
time, DA collects a set of runtime assumptions under which the
dependence test becomes valid. The default remains the same as before
the patch: DA rejects the dependence test as undecidable instead of
collecting runtime assumptions.
---------
Co-authored-by: Michael Kruse <github@meinersbur.de>
Co-authored-by: Ryotaro Kasuga <kasuga.ryotaro@fujitsu.com>
Parameter PossiblyLoopIndependent has lost its intended purpose. This
flag is always set to true in all cases when depends() is called, hence
we want to reconsider the utility of this variable and remove it from
the function signature entirely. This is an NFC patch.
This patch fixes two bugs:
https://github.com/llvm/llvm-project/issues/41488https://github.com/llvm/llvm-project/issues/53942
The dependence analysis assumes that the base address of array accesses
is invariant across loop iterations. In both bugs the base address
evolves following loop iterations: the base address flip-flops between
two different memory objects.
The patch uses the cross iteration mode of alias analysis to disambiguate the
base objects.
Modify the DA pretty printer to match the output of other analysis
passes. This enables update_analyze_test_checks.py to also work on DA
tests. Auto generate all the Dependence Analysis tests.
This patch is the first of the two-patch series (D130188, D130179) that
resolve PR56275 (https://github.com/llvm/llvm-project/issues/56275)
which is a missed opportunity, where a perfrectly valid case for loop
interchange failed interchange legality.
If the distance/direction vector produced by dependence analysis (DA) is
negative, it needs to be normalized (reversed). This patch provides helper
functions `isDirectionNegative()` and `normalize()` in DA that does the
normalization, and clients can query DA to do normalization if needed.
A pass option `<normalized-results>` is added to DependenceAnalysisPrinterPass,
and we leverage it to update DA test cases to make sure of test coverage. The
test cases added in `Banerjee.ll` shows that negative vectors are normalized
with `print<da><normalized-results>`.
Reviewed By: bmahjour, Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D130188
This is a follow-up patch to D122857 where we added delinearization of
fixed-size arrays to loop cache analysis, which resulted in some duplicate
code, i.e., "tryDelinearizeFixedSize()", in LoopCacheCost.cpp and
DependenceAnalysis.cpp. Refactoring is done in this patch.
This patch refactors out the main logic of "tryDelinearizeFixedSize()" as
"tryDelinearizeFixedSizeImpl()" and moves it to Delinearization.cpp, such that
clients can reuse "llvm::tryDelinearizeFixedSizeImpl()" wherever they would
like to delinearize fixed-size arrays. Currently it has two users, i.e.,
DependenceAnalysis.cpp and LoopCacheCost.cpp.
Reviewed By: Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D124745
To represent various loop levels within a nest, DA implements a special
numbering scheme (see comment atop establishNestingLevels). The goal of
this numbering scheme appears to be representing each unique loop
distinctively by using as little memory as possible. This numbering
scheme is simple when the source and destination of the dependence are
in the same loop. In such cases the level is simply the depth of the
loop in which src and dst reside. When the src and dst are not in the
same loop, we could run into the following situation exposed by
https://reviews.llvm.org/D71539. This patch fixes this by detecting
such cases in checkSubscripts and treating them as non-linear/non-affine.
Reviewed By: Meinersbur
Differential Revision: https://reviews.llvm.org/D110973
Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the error has been removed,
cl::ZeroOrMore is unneeded.
Also remove cl::init(false) while touching the lines.
Refactor from iteratively using BitCastInst::getOperand()
to using stripPointerCasts() instead. This is an improvement
since now we are able to analyze more cases, please refer
to test cases added in this patch.
Reviewed By: Meinersbur, #loopoptwg
Differential Revision: https://reviews.llvm.org/D123559
This renames the primary methods for creating a zero value to `getZero`
instead of `getNullValue` and renames predicates like `isAllOnesValue`
to simply `isAllOnes`. This achieves two things:
1) This starts standardizing predicates across the LLVM codebase,
following (in this case) ConstantInt. The word "Value" doesn't
convey anything of merit, and is missing in some of the other things.
2) Calling an integer "null" doesn't make any sense. The original sin
here is mine and I've regretted it for years. This moves us to calling
it "zero" instead, which is correct!
APInt is widely used and I don't think anyone is keen to take massive source
breakage on anything so core, at least not all in one go. As such, this
doesn't actually delete any entrypoints, it "soft deprecates" them with a
comment.
Included in this patch are changes to a bunch of the codebase, but there are
more. We should normalize SelectionDAG and other APIs as well, which would
make the API change more mechanical.
Differential Revision: https://reviews.llvm.org/D109483
None of this logic has anything to do with SCEV's internals, it just uses the existing public APIs. As a result, we can move the code from ScalarEvolution.cpp/hpp to Delinearization.cpp/hpp with only minor changes.
This was discussed in advance on today's loop opt call. It turned out to be easy as hoped.
Function exploreDirections() in DependenceAnalysis implements a recursive
algorithm for refining direction vectors. This algorithm has worst-case
complexity of O(3^(n+1)) where n is the number of common loop levels.
In this patch I'm adding a threshold to control the amount of time we
spend in doing MIV tests (which most of the time end up resulting in over
pessimistic direction vectors anyway).
Reviewed By: Meinersbur
Differential Revision: https://reviews.llvm.org/D107159
D104806 broke some uses of getMinusSCEV() in DependenceAnalysis:
subtraction with different pointer bases returns a SCEVCouldNotCompute.
Make sure we avoid cases involving such subtractions.
Differential Revision: https://reviews.llvm.org/D106099
Patch by Artem Radzikhovskyy!
Allow delinearization of fixed sized arrays if we can prove that the GEP indices do not overflow the array dimensions. The checks applied are similar to the ones that are used for delinearization of parametric size arrays. Make sure that the GEP indices are non-negative and that they are smaller than the range of that dimension.
Changes Summary:
- Updated the LIT tests with more exact values, as we are able to delinearize and apply more exact tests
- profitability.ll - now able to delinearize in all cases, no need to use -da-disable-delinearization-checks flag and run the test twice
- loop-interchange-optimization-remarks.ll - in one of the cases we are able to delinearize without using -da-disable-delinearization-checks
- SimpleSIVNoValidityCheckFixedSize.ll - removed unnecessary "-da-disable-delinearization-checks" flag. Now can get the exact answer without it.
- SimpleSIVNoValidityCheckFixedSize.ll and PreliminaryNoValidityCheckFixedSize.ll - made negative tests more explicit, in order to demonstrate the need for "-da-disable-delinearization-checks" flag
Differential Revision: https://reviews.llvm.org/D101486
Patch by Artem Radzikhovskyy!
Symptom: ExactSIV test produced incorrect analysis of dependencies see LIT tests
Bug: At the end of the algorithm when determining dependence direction original author forgot to divide intermediate results by gcd and round result toward zero
Although this bug can be fixed with significantly fewer changes I opted to write the code in such a way that reflects the original algorithm that Banerjee proposed, for easier reference in the future. This surprisingly results in shorter code, and fewer quotient and max/min calculations.
Changes Summary:
- fixed findGCD to return valid x and y so that they match the function description where: ax - by = gcd(a,b)
- Fixed ExactSIV test, to produce proper results
- Documented the extension of Banerjee's algorithm that the original code author introduced. Banerjee's original algorithm only tested whether Dst depends on Src, the extension also allows us to test whether Src depends on Dst, in one pass.
- ExactRDIV test worked fine. Since it uses findGCD(), it needed to be updated.Since ExactRDIV test has very few changes from the core algorithm of ExactSIV I modified the test to have consistent format as ExactSIV.
- Updated the LIT tests to be testing for correct values.
Differential Revision: https://reviews.llvm.org/D100331
Main reason is preparation to transform AliasResult to class that contains
offset for PartialAlias case.
Reviewed By: asbirlea
Differential Revision: https://reviews.llvm.org/D98027
Currently, we have some confusion in the codebase regarding the
meaning of LocationSize::unknown(): Some parts (including most of
BasicAA) assume that LocationSize::unknown() only allows accesses
after the base pointer. Some parts (various callers of AA) assume
that LocationSize::unknown() allows accesses both before and after
the base pointer (but within the underlying object).
This patch splits up LocationSize::unknown() into
LocationSize::afterPointer() and LocationSize::beforeOrAfterPointer()
to make this completely unambiguous. I tried my best to determine
which one is appropriate for all the existing uses.
The test changes in cs-cs.ll in particular illustrate a previously
clearly incorrect AA result: We were effectively assuming that
argmemonly functions were only allowed to access their arguments
after the passed pointer, but not before it. I'm pretty sure that
this was not intentional, and it's certainly not specified by
LangRef that way.
Differential Revision: https://reviews.llvm.org/D91649
All existing SCEV cast types operate on integers.
D89456 will add SCEVPtrToIntExpr cast expression type.
I believe this is best for consistency.
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D89455
Summary:
Currently the dependence analysis in LLVM is unable to compute accurate
dependence vectors for multi-dimensional fixed size arrays.
This is mainly because the delinearization algorithm in scalar evolution
relies on parametric terms to be present in the access functions. In the
case of fixed size arrays such parametric terms are not present, but we
can use the indexes from GEP instructions to recover the subscripts for
each dimension of the arrays. This patch adds this ability under the
existing option `-da-disable-delinearization-checks`.
Authored By: bmahjour
Reviewer: Meinersbur, sebpop, fhahn, dmgreen, grosser, etiotto, bollu
Reviewed By: Meinersbur
Subscribers: hiraditya, arphaman, Whitney, ppc-slack, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72178
Summary:
[DA] Move common code in checkSrcSubscript and checkDstSubscript to a
new function checkSubscript. This avoids duplicate code and possible
out of sync in the future.
Reviewers: sebpop, jmolloy, reames
Reviewed By: sebpop
Subscribers: bmahjour, hiraditya, llvm-commits, amehsan
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D71087
Patch by zhongduo.
Summary:
The current da printer shows the dependence without indicating
which instructions are being considered as the src vs dst. It
also silently ignores call instructions, despite the fact that
they create confused dependence edges to other memory
instructions. This patch addresses these two issues plus a
couple of minor non-functional improvements.
Authored By: bmahjour
Reviewer: dmgreen, fhahn, philip.pfaffe, chandlerc
Reviewed By: dmgreen, fhahn
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D71088
This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of recompilation.
I found this fact by looking at this table, which is sorted by the
number of times a file was changed over the last 100,000 git commits
multiplied by the number of object files that depend on it in the
current checkout:
recompiles touches affected_files header
342380 95 3604 llvm/include/llvm/ADT/STLExtras.h
314730 234 1345 llvm/include/llvm/InitializePasses.h
307036 118 2602 llvm/include/llvm/ADT/APInt.h
213049 59 3611 llvm/include/llvm/Support/MathExtras.h
170422 47 3626 llvm/include/llvm/Support/Compiler.h
162225 45 3605 llvm/include/llvm/ADT/Optional.h
158319 63 2513 llvm/include/llvm/ADT/Triple.h
140322 39 3598 llvm/include/llvm/ADT/StringRef.h
137647 59 2333 llvm/include/llvm/Support/Error.h
131619 73 1803 llvm/include/llvm/Support/FileSystem.h
Before this change, touching InitializePasses.h would cause 1345 files
to recompile. After this change, touching it only causes 550 compiles in
an incremental rebuild.
Reviewers: bkramer, asbirlea, bollu, jdoerfert
Differential Revision: https://reviews.llvm.org/D70211
