If a query uses an exclusion set but we haven't used it to determine the
result, we can cache the query without exclusion set too. When we lookup
a cached result we can check for the non-exclusion set version first.
We used to check the query instructions for effects but that does not
work well with complex accesses we will probably support in the future.
Now we simply let the user decide what accesses to look for.
Before we might have ended up queriying the AAExecutionDomain of a
different function, which resulted in wrong optimistic results.
Partially fixes https://github.com/llvm/llvm-project/issues/60425
We improved our simplification and this exposed a bug in the store
elimination. A load that had dead uses and assume uses was thought to be
used by assumes only. Consequently we also deleted the "dead use users".
This was a problem because a dead use just means we will not use the
load there. The user might still be needed.
Exposed by OvO, reported by @ye-luo.
If an instruction is executed in an aligned region we can ignore
threading effects and use CFG reasoning (dominance and reachability).
This is true because all threads are together in an aligned region and
there cannot be one waiting for a signal at a place not connected via
the control flow.
More dedicated tests will follow.
More details can be found here:
"Co-Designing an OpenMP GPU Runtime and Optimizations for Near-Zero
Overhead Execution", IPDPS 2022,
https://www.osti.gov/servlets/purl/1890094
If we modified the IR during manifest, e.g., SPMDzation, we might end up
with un-cached reachability queries. This is not good as the result is
going to be optimistic. We now cache the updateImpl result and use it
during manifest.
Bug was exposed in a follow up extension.
While full support requires more work (see TODOs), this allows us to
handle vector writes with a single constant value properly. For now,
we can handle the same constant values stored to all elements if
everything is of a fixed size.
Even if we have multiple access ranges, the access can be exact. It is
not a MUST access but that is taken care of elsewhere. The tests were
wrong as they contained uninitialized memory. When the memory is
initialized it works as expected.
Even if a barrier does not enforce aligned execution, it will
effectively be like an aligned barrier if it is executed by all threads
in an aligned way. We lack control flow divergence analysis here so we
can only do (basic block) local reasoning for now.
With this patch we track aligned barriers in AAExecutionDomain and also
delete unnecessary barriers there. This allows us to eliminate barriers
across blocks, across functions, and in the presence of complex accesses
that do not force a barrier. Further, we can use the collected
information to enable store-load forwarding in a threaded environment
(follow up patch).
Differential Revision: https://reviews.llvm.org/D140463
A `ConstantExpr` ICmp is pretty much the same thing as an ICmpInst when
we want to simplify it. We just need to be less restrictive wrt. the
type and use the static helper functions directly.
Fixes: https://github.com/llvm/llvm-project/issues/59767
We use the actual access (=remote) instruction when reasoning about
accesses, except for one leftover use case of the local instruction.
This caused us to potentially ignore the dominating write if the read
and write were in a different function than the (stack) allocation.
Reported by @ye-luo
This resolves a recent regression introduced by a bug fix and allows us
to use dominating write information (formerly HasBeenWrittenTo
information) to skip potential interfering accesses.
Generally, there are two changes here:
1) If we have dominating writes they form a chain and we can look at the
least one to minimize the distance between the write and the (read)
access in question.
2) If such a least dominating write exists, we can ignore writes in
other functions as long as they cannot be reached from code between
this write and the (read) access in question.
We have all the tools available to make such queries and the positive
tests show the result. Note that the negative test from the bug fix is
still in tree and not affected.
As a side-effect, we can remove the (arbitrary) treshold now on the
number of interfering accesses since we do not iterate over dominating
ones anymore.
Recent commit introducing AA for getting underluying objects of a
pointer created an uninitialized boolean, which causes tests to fail
when built unter asan/ubsan. This initialized that variable.
This patch introduces a new AA `AAUnderlyingObjects`. It is basically like a wrapper
AA of the function `AA::getAssumedUnderlyingObjects`, but it can recursively do
query if the underlying object is an indirect access, such as a phi node or a select
instruction.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D141164
HandlePassthroughUser may sometimes create a new entry for the OffsetInfo of a
user in the OffsetInfoMap. This can invalidate outstanding references into the
map, including the one which needs to be copied into the new entry. This
produces invalid offset info that can trigger assertions.
Fixed this by not using references at this point. The bug was originally
introduced in commit ID 0dc0a441323d41b4860668f38d290579e0de130c.
Reviewed By: ronlieb
Differential Revision: https://reviews.llvm.org/D140837
AAPotentialConstantValues now works for PHI and Load by simply examinig
AAPotentialValues for the instruction itself.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D140371
Previously reverted in 8b446ea2ba39e406bcf940ea35d6efb4bb9afe95
Reapplying because this commit is NOT DEPENDENT on the reverted commit
fc21f2d7bae2e0be630470cc7ca9323ed5859892, which broke the ASAN buildbot.
See https://reviews.llvm.org/rGfc21f2d7bae2e0be630470cc7ca9323ed5859892 for
more information.
The arguments to a PHI may represent a recurrence by eventually using the output
of the PHI itself. This is now handled by checking for cycles in the control
flow. If a PHI is not in a recurrence, it is now able to report multiple offsets
instead of conservatively reporting unknown.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D138991
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This fixes check-clang-tools.
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This fixes clang.
Previously reverted in 12696d302d146ffe616eecab3feceba9d29be2db
The arguments to a PHI may represent a recurrence by eventually using the output
of the PHI itself. This is now handled by checking for cycles in the control
flow. If a PHI is not in a recurrence, it is now able to report multiple offsets
instead of conservatively reporting unknown.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D138991
The arguments to a PHI may represent a recurrence by eventually using the output
of the PHI itself. This is now handled by checking for cycles in the control
flow. If a PHI is not in a recurrence, it is now able to report multiple offsets
instead of conservatively reporting unknown.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D138991
Even if all loads and stores are in `nosync` functions we cannot
guarantee there is no synchronization going on between them. As such, we
cannot use CFG reasoning. We could check the entire module, or, what
happens now to minimize test churn, is to check if all accesses are in
the same function that is `nosync`. A follow up will undo some of the
regressions where possible.
Similarly, reachability cannot be used to exclude an access if the
access is not known to be executed by the same thread as the given
instruction.
The OpenMP-opt test was added for the latter problem.
We had two AAs for reachability but it was very cumbersome to extend
them. We also had some fallback to use LLVM-core mechanisms and cache
the result. The new design shares the query code and interface nicely
between AAIntraFnReachability and AAInterFnReachability.
As part of the rewrite we also added the ExclusionSet to the queries.
Even if a value is for sure written we need to visit the call sites as
they might end up inside the function that reads and writes the value.
In a follow up we can introduce correct reasoning to avoid the backwards
traversal in this case and instead check if any call site between the
write and the read might reach a potential write we want to exclude.
An expression of the form `gep(base, select(pred, const1, const2))` can result
in a set of offsets instead of just one. PointerInfo can now track these sets
instead of conservatively modeling them as Unknown. In general, AAPointerInfo
now uses AAPotentialConstantValues to examine the operands of the GEP.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D138646
An expression of the form `gep(base, select(pred, const1, const2))` can result
in a set of offsets instead of just one. PointerInfo can now track these sets
instead of conservatively modeling them as Unknown. In general, AAPointerInfo
now uses AAPotentialConstantValues to examine the operands of the GEP.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D138646
Similar to dominance reasoning, we cannot use CFG reachability if the
instructions might be executed by different threads. A follow up will
improve our sensitivity for situations when it is OK to use graph
reasoning.
