IRCE currently has two profitability checks:
1. min number of iterations (10 by default)
2. branch is highly biased (> 15/16)
However, it may still be profitable to eliminate range checks even if
the branch isn't as biased. Consider, for example, a loop with 100
iterations, where IRCE currently eliminates all 100 range checks. The
same range checks performed over a loop with 200 iterations aren't
eliminated because the branch is 50-50.
This patch proposes to relax the profitability checks of IRCE. Namely,
instead of the two checks currenly in place, consider IRCE profitable if
the branch probability scaled by the expected number of iterations
(i.e., the estimated number of eliminated checks) is over a threshold.
This covers the minimum number of iterations check (there are at least
as many iterations as eliminated range checks), and changes the bias
check from a percent of iterations to at least a constant threshold of
eliminated checks.
If the number of iterations can't be estimated, the check falls back to
the current 15/16 likelihood check.
This refactoring hoists the profitability check earlier in the pipeline,
so that for loops that are not profitable to transform there is no
iteration over the basic blocks or LoopStructure computation.
Motivated by PR #104659 that tweaks how the profitability of individual
branches is evaluated.
We have guarantees that induction variable will not overflow in the main
loop after the loop constrained. Therefore we can add no wrap flags on
its base in order not to miss info that loop is countable.
Add NSW flag now, since adding NUW flag requires a bit more complicated
analysis.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154954
Here is activated check elimination which was parsed previously in
https://reviews.llvm.org/D154069
* Added runtime check that computed range's boundary doesn't overflow in
terms of range type.
* From the statement INT_MIN <= END <= INT_MAX is inferred check:
isNonNegative(INT_MAX - END) * isNonNegative(END - INT_MIN).
* If overflow happens, check will return 0 and the safe interval will be
empty.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154188
Introduced the following range checks forms parsing:
* IV - Offset vs Limit
* Offset - IV vs Limit
Range's end boundary is computed as (Offset +/- Limit ).
If it's not possible to prove at compile time that computed upper bound
will not overflow, then scale boundary computation to a wider type to
perform overflow check at runtime.
Runtime overflow will be implemented in the next patch. In the meantime
safe range for such kind of checks isn't computed.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154069
IRCE expects true edge of range check's branch comes to loop.
If it meets reverse case - invert the branch.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D148244
Next step of the preparatory refactoring for the upcoming support of new
new range check form to parse.
This change isolates logic of 'IV vs Limit' range check parsing to
simplify adding parsers for new range checks forms.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154160
Next step of the preparatory refactoring for the upcoming support of
new range check form to parse.
Previous one: https://reviews.llvm.org/D154156
With this change we avoid meaningless parsing after realizing that Index
is not AddRec
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154158
Preparatory refactoring for the upcoming support of new range check form
to parse.
With this change we always set Index and End values together in the same
place.
parseRangeCheckICmp specification updated.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D154156
Simplify parseRangeCheckICmp:
- If RHS is loop-variant, swap LHS/RHS and swap predicate
- all checks are either IV >(=) const or IV <(=) RHS (maybe not const)
Patch by Aleksandr Popov!
Differential Revision: https://reviews.llvm.org/D148720
The motivation is to make an opportunity to compute and return
expressions after parsing ICmp into a range check (e.g. Length + 1).
Patch by Aleksandr Popov!
Differential Revision: https://reviews.llvm.org/D148205
It seems that existing logic is too strict about latch block exit count.
It is required to be computable, however it is not used in any computations,
and effectively the only thing it is used for is to get the type of computed
exit count.
Sometimes the exit count for latch block is not known, but the loop is still
finite because of other exits, and safe bounds are still computable. In this case,
we miss an opportunity to apply IRCE.
We could instead use a more relaxed version - max symbolic exit count, which,
if exists, is enough to say that the loop is finite, and its type should be good enough.
There is a subtlety with type: we do not support latch count type wider than range
check type. Because of that, we want to have the narrowest type available. So if it
can be computed from latch block immediately, take it. Otherwise, take whatever whole
loop provides and hope that it's type isn't too wide.
Differential Revision: https://reviews.llvm.org/D147910
Reviewed By: danilaml
It is not actually used for any computations. Its only purpose is to
check that the loop is finite and find out the type of computed exit
count. Refactor code so that we only store this type.
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
When IRCE runs on outer loop and sees a check of an AddRec of
inner loop, it crashes with an assert in SCEV that the AddRec
must be loop invariant.
This adds a bail out if the AddRec which is checked in icmp
is for another loop.
Fixes https://github.com/llvm/llvm-project/issues/58912.
Differential Revision: https://reviews.llvm.org/D137822
Since SCEV learned to look through single value phis with
20d798bd47ec5191de1b2a8a031da06a04e612e1, whenever we add
a new input to a Phi, we should make sure that the old cached
value is dropped. Otherwise, it may lead to various miscompiles,
such as breach of dominance as shown in the bug
https://github.com/llvm/llvm-project/issues/57335
isSafeToExpand() for addrecs depends on whether the SCEVExpander
will be used in CanonicalMode. At least one caller currently gets
this wrong, resulting in PR50506.
Fix this by a) making the CanonicalMode argument on the freestanding
functions required and b) adding member functions on SCEVExpander
that automatically take the SCEVExpander mode into account. We can
use the latter variant nearly everywhere, and thus make sure that
there is no chance of CanonicalMode mismatch.
Fixes https://github.com/llvm/llvm-project/issues/50506.
Differential Revision: https://reviews.llvm.org/D129630
