This reverts commit 6fbf022908c104a380fd1854fb96eafc64509366, restoring
commit bf6ff4fd4b735afffc65f92a4a79f6610e7174c3 with a fix for a bot
failure due to a previously unstable iteration order.
Differential Revision: https://reviews.llvm.org/D141077
This patch adds a new preheader block the VPlan to place SCEV expansions
expansions like the trip count. This preheader block is disconnected
at the moment, as the bypass blocks of the skeleton are not yet modeled
in VPlan.
The preheader block is executed before skeleton creation, so the SCEV
expansion results can be used during skeleton creation. At the moment,
the trip count expression and induction steps are expanded in the new
preheader. The remainder of SCEV expansions will be moved gradually in
the future.
D147965 will update skeleton creation to use the steps expanded in the
pre-header to fix#58811.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D147964
This patch implements the mem2reg interfaces for MemRef types. This only supports scalar memrefs of a small list of types. It would be beneficial to create more interfaces for default values before expanding support to more types. Additionally, I am working on an upcoming revision to bring SROA to MLIR that should help with non-scalar memrefs.
Reviewed By: gysit, Mogball
Differential Revision: https://reviews.llvm.org/D149441
This patch adds the necessary hacks to support global constructors and
destructors. This is an incredibly hacky process caused by the primary
fact that Nvidia does not provide any binary tools and very little
linker support. We first had to emit references to these functions and
their priority in D149451. Then we dig them out of the module once it's
loaded to manually create the list that the linker should have made for
us. This patch also contains a few Nvidia specific hacks, but it passes
the test, albeit with a stack size warning from `ptxas` for the
callback. But this should be fine given the resource usage of a common
test.
This also adds a dependency on LLVM to the NVPTX loader, which hopefully doesn't
cause problems with our CUDA buildbot.
Depends on D149451
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D149527
This patch mostly adapts the existing AMDGPUCtorDtorLoweringPass for use
by the Nvidia backend. This pass transforms the ctor / dtor list into a
kernel call that can be used to invoke those functinos. Furthermore, we
emit globals such that the names and addresses of these constructor
functions can be found by the driver. Unfortunately, since NVPTX has no
way to emit variables at a named section, nor a functioning linker to
provide the begin / end symbols, we need to mangle these names and have
an external application find them.
This work is related to the work in D149398 and D149340.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D149451
The previous patches added the necessary support for global constructors
used to register tests. This patch enables the AMDGPU target to build
and run the unit tests on the GPU. Currently this only tests the `ctype`
tests, but adding more should be straightforward from here on.
Reviewed By: sivachandra
Differential Revision: https://reviews.llvm.org/D149517
The following pattern is common in the llvm codebase, as well as in downstream projects:
```
llvm::to_vector(llvm::map_range(container, lambda))
```
This patch introduces a shortcut for this called `map_to_vector`.
This template depends on both `llvm/ADT/SmallVector.h` and `llvm/ADT/STLExtras.h`, and since these are both relatively large and do not depend on each other, the `map_to_vector` helper is placed in a new header under `llvm/ADT/SmallVectorExtras.h`. Only a handful of use cases have been updated to use the new helper.
Differential Revision: https://reviews.llvm.org/D145390
The shared code for lowering the sum and product operations in
flang/lib/Optimizer/HLFIR/Transforms/LowerHLFIRIntrinsics.cpp have
been moved into a new class HlfirReductionIntrinsicConverion.
Depends on: D148719
Differential Revision: https://reviews.llvm.org/D149644
Carries out the initial lowering of the product intrinsic into HLFIR
following a similar method to sum, the --use-hlfir-intrinsic-ops flag
in test/Lower/HLFIR/expr-box is set to false so that the tests will
pass until hlfir.product is lowered into fir.call
Depends on: D147624
Differential Revision: https://reviews.llvm.org/D148719
Adds a HLFIR operation for the PRODUCT intrinsic according to
the design set out in flang/doc/HighLevelFIR.md
Since the PRODUCT intrinsic is essentially identical to SUM
in terms of its arguments and result characteristics in the
Fortran Standard, the operation definition and subsequent
tests also take the same form.
Differential Revision: https://reviews.llvm.org/D147624
This commit eliminates the uninitialized error state from the class
RegionRawOffsetV2 (which is locally used by the Clang Static Analyzer
checker alpha.security.ArrayBoundV2) and replaces its use with
std::optional.
Motivated by https://reviews.llvm.org/D148355#inline-1437928
Moreover, the code of RegionRawOffsetV2::computeOffset() is rearranged
to clarify its behavior. The helper function getValue() was eliminated
by picking a better initial value for the variable Offset; two other
helper functions were replaced by the lambda function Calc() because
this way it doesn't need to take the "context" objects as parameters.
This reorganization revealed some surprising (but not outright buggy)
behavior that's marked by a FIXME and will be revisited in a separate
commit.
Differential Revision: https://reviews.llvm.org/D149259
It looks like the intention here is to truncate a XLenVT -> i1, in
which case we should be emitting snez instead of sneq if I'm understanding
correctly.
Reviewed By: jacquesguan, frasercrmck
Differential Revision: https://reviews.llvm.org/D149732
They don't convey any useful information and make the documentation
unnecessarily hard to read.
Differential Revision: https://reviews.llvm.org/D149641
Another test based on review comments added late in the review.
This one confirms that the multiplication and addition of the outer
index to the inner index and thus form the 2D index.
Reviewed By: tblah
Differential Revision: https://reviews.llvm.org/D149265
Adds HSA timeout hint of 2 seconds to the AMDGPU nextgen-plugin to improve
performance of small kernels.
The HSA runtime may stay in HSA_WAIT_STATE_ACTIVE for up to the timeout
value before switching to HSA_WAIT_STATE_BLOCKED. This can improve
latency from which small kernels can benefit.
The value was determined via experimentation w/ different benchmarks.
The timeout value can be overriden using the environment variable
LIBOMPTARGET_AMDGPU_STREAM_BUSYWAIT with a value in microseconds.
Original author: Greg Rodgers <Gregory.Rodgers@amd.com>
Contributions from: JP Lehr <JanPatrick.Lehr@amd.com>
Differential Revision: https://reviews.llvm.org/D148808
The qualifier alignment fixer appeared to ignore any ranges specified for limiting formatting.
This change ensures that it only formats affected lines to avoid unexpected changes.
Fixes#54888.
Differential Revision: https://reviews.llvm.org/D149643
This patch changes check for nested TaskGroups so that it allows
parallel execution for TaskGroups. Following pattern would not work
parallelly with current check:
std::function<void()> Fn = [&]() {
parallel::TaskGroup tg;
tg.spawn([&]() { });
};
ThreadPool Pool;
Pool.async(Fn);
Pool.async(Fn);
Pool.wait();
One of the TaskGroup would work sequentially as current check
verifies overall number of TaskGroup. Two not nested
TaskGroups can work parallelly but current check prevents this.
Also this patch avoids parallel mode for TaskGroup
in parallel::strategy.ThreadsRequested == 1 case.
This patch is a followup of discussion from D142318
Differential Revision: https://reviews.llvm.org/D148984
Instead of looking for the existence of a `ret` instruction after disassembly
as a way to check that the right ISA is present in the produced binary, check
instead the ELF header of the file. This is a more reliable solution, since the
alternative can easily result in false positives.
Differential Revision: https://reviews.llvm.org/D149578
The step is already expanded in the VPlan. Use this expansion instead.
This is a step towards modeling fixing up IV users in VPlan.
It also fixes a crash casued by SCEV-expanding the Step expression in
fixupIVUsers, where the IR is in an incomplete state
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D147963
Add hlfir.forall_mask, hlfir.where, and hlfir.elsewhere operations that
are operations that holds (optionally for hlfir.elsewhere) the
evaluation of a logical mask that controls the evaluation of nested
operations.
They allow representing Fortran forall control mask, as well as where
and eslewhere statements/constructs.
They use the OrderedAssignmentTreeOpInterface since they can all be used
inside Forall and their masks should be fully evaluated for all the
index-value set induced by parent Forall before any of the nested
operations in their body is evaluated.
I initially tried making them into a single operation with some attributes
to make a difference, but I felt this made the verifier/parser/printer and
usages messier/tricky compared to making three distinct operations that
represent the three Fortran feature in a vanilla way.
Differential Revision: https://reviews.llvm.org/D149754
This patch adds the hlfir.forall operation and the
OrderAssignmentTreeOpInterface that allows representing Fortran forall.
It uses regions to keep Fortran expression evaluation independent from
each other in the IR. Forall assignments inside hlfir.forall are
represented with hlfir.region_assign which also keeps the IR generated
for each expressions independently.
The goal of this representation is to provide a representation that is
straightforward to generate from Fortran parse tree without any analysis, while
providing enough structure information so that an optimization pass can decide
how to schedule, and save if needed, the evaluations of the Forall and Where
expression and statements. It allows the data dependency analysis to be done at
the HLFIR level.
The OrderAssignmentTreeOpInterface allows ensuring that the Forall/Where
tree structure is kept in the IR. It will allow visiting this tree in
the IR without hard coding the operation structures in the pass.
Differential Revision: https://reviews.llvm.org/D149734
We might classify different clang types to the same interp types, so
skip the cast in that case.
No test attached since this is already exercised a few times in the
existing tests.
The `-v` flag means verbose and not version.
With `clang` this flag prints the version and exits successfully.
Under `GCC` this is not a valid command line so the binary exits with
an error.
The `valueDelta` map was used to calculate the symbol value deltas from
the previous iteration. Since the symbol values themselves are also
updated every iteration, the following invariant holds:
```
sa[i].offset == sa[i].d->value + valueDelta[sa[i].d]
```
Note that `sa[i].offset` contains the original value of `sa[i].d` and is
never changed.
This means that the current way of updating symbol values can be
rewritten to not need the `valueDelta` map:
```
sa[i].d->value -= delta - valueDelta.find(sa[i].d)->second;
<=> (replace invariant)
sa[i].d->value -= delta - (sa[i].offset - sa[i].d->value);
<=>
sa[i].d->value = sa[i].d->value - (delta - (sa[i].offset - sa[i].d->value));
<=>
sa[i].d->value = sa[i].d->value - delta + sa[i].offset - sa[i].d->value;
<=>
sa[i].d->value = sa[i].offset - delta;
```
This patch implements this simplification. I believe this improves the
readability of the code as it took me quite some time to understand the
use of `valueDelta`. It might also have a slight performance benefit as
it removes one iteration over all relocations every relax iteration.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D149735
