The OpenACC standard specifies an `atomic` construct in section 2.12 (of
3.3 spec), used to ensure that a specific location is accessed or
updated atomically. Four different clauses are allowed: `read`, `write`,
`update`, or `capture`. If no clause appears, it is as if `update` is
used.
The OpenMP specification defines the same clauses for `omp atomic`. The
types of expression and the clauses in the OpenACC spec match the OpenMP
spec exactly. The main difference is that the OpenMP specification is a
superset - it includes clauses for `hint` and `memory order`. It also
allows conditional expression statements. But otherwise, the expression
definition matches.
Thus, for OpenACC, we refactor and reuse the OpenMP implementation as
follows:
* The atomic operations are duplicated in OpenACC dialect. This is
preferable so that each language's semantics are precisely represented
even if specs have divergence.
* However, since semantics overlap, a common interface between the
atomic operations is being added. The semantics for the interfaces are
not generic enough to be used outside of OpenACC and OpenMP, and thus
new folders were added to hold common pieces of the two dialects.
* The atomic interfaces define common accessors (such as getting `x` or
`v`) which match the OpenMP and OpenACC specs. It also adds common
verifiers intended to be called by each dialect's operation verifier.
* The OpenMP write operation was updated to use `x` and `expr` to be
consistent with its other operations (that use naming based on spec).
The frontend lowering necessary to generate the dialect can also be
reused. This will be done in a follow up change.
This patch is part of a larger initiative aimed at fixing floating-point
`max` and `min` operations in MLIR:
https://discourse.llvm.org/t/rfc-fix-floating-point-max-and-min-operations-in-mlir/72671.
This patch addresses task 1.1 from the plan. It involves modifying the
lowering process for `arith.minf` and `arith.maxf` operations.
Specifically, the change replaces the usage of `llvm.minnum` and
`llvm.maxnum` with `llvm.minimum` and `llvm.maximum`, respectively. This
adjustment is necessary because the `m**num` intrinsics are not suitable
for the mentioned MLIR operations due to semantic discrepancies in
handling NaNs, positive and negative floating-point zeros.
One-Shot Bufferize correctly handles RaW conflicts around repetitive regions (loops). Specical handling is needed for parallel regions. These are a special kind of repetitive regions that can have additional RaW conflicts that would not be present if the regions would be executed sequentially.
Example:
```
%0 = bufferization.alloc_tensor()
scf.forall ... {
%1 = linalg.fill ins(...) outs(%0)
...
scf.forall.in_parallel {
tensor.parallel_insert_slice %1 into ...
}
}
```
A separate (private) buffer must be allocated for each iteration of the `scf.forall` loop.
This change adds a new interface method to `BufferizableOpInterface` to detect parallel regions. By default, regions are assumed to be sequential.
A buffer is privatized if an OpOperand bufferizes to a memory read inside a parallel region that is different from the parallel region where operand's value is defined.
Differential Revision: https://reviews.llvm.org/D159286
scf.forall ops without shared outputs (i.e., fully bufferized ops) are
lowered to scf.parallel. scf.forall ops are typically lowered by an
earlier pass depending on the execution target. E.g., there are
optimized lowerings for GPU execution. This new lowering is for
completeness (convert-scf-to-cf can now lower all SCF loop constructs)
and provides a simple CPU lowering strategy for testing purposes.
scf.parallel is currently lowered to scf.for, which executes
sequentially. The scf.parallel lowering could be improved in the future
to run on multiple threads.
It was common to see `value.getValue().empty()` or `value.size()`
instead of the idiomatic `value.empty()`.
Depends on D159455
Differential Revision: https://reviews.llvm.org/D159456
Tensor pack operations are optimistically lowered to pad + insert_slice
when the pack operation only pads the input tensor. The existing
lowering emits insert_slice operations which do not meet the
rank-reducibility requirements of insert_slice.
This change updates the logic in linalg::lowerPack to first check the
rank-reducibility requirement. When the requirement is not met, the
lowering will emit the full sequence of pad + expand + transpose.
Reviewed By: chelini
Differential Revision: https://reviews.llvm.org/D159382
The primary motivation is to we have a simple mechanism to extract
values from attributes in folders and canon patterns without having to
re-fold constants or write nested conditions over attribute types.
Matching over attributes composes especially well with fold adaptors.
Update folds in Arith and SPIRV dialects to match over attributes, where
applicable.
Reviewed By: mehdi_amini, zero9178
Differential Revision: https://reviews.llvm.org/D159437
Support environments where logical types do not necessarily correspond to allowed storage access types.
Also make pattern match failures more descriptive.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D159386
Introduce a simple conversion of a memref.alloc/dealloc pair into an
alloca in the same scope. Expose it as a transform op and a pattern.
Allocas typically lower to stack allocations as opposed to alloc/dealloc
that lower to significantly more expensive malloc/free calls. In
addition, this can be combined with allocation hoisting from loops to
further improve performance.
This patch pairs a promised interface with the object (Op/Attr/Type/Dialect) requesting the promise, ie:
```
declarePromisedInterface<MyAttr, MyInterface>();
```
Allowing to make fine grained promises. It also adds a mechanism to query if `Op/Attr/Type` has an specific promise returning true if the promise is there or if an implementation has been added. Finally it adds a couple of `Attr|TypeConstraints` that can be used in ODS to query if the promise or an implementation is there.
This patch tries to solve 2 issues:
1. Different entities cannot use the same promise.
```
declarePromisedInterface<MyInterface>();
// Resolves a promise.
MyAttr1::attachInterface<MyInterface>(ctx);
// Doesn't resolves a promise, as the previous attachment removed the promise.
MyAttr2::attachInterface<MyInterface>(ctx);
```
2. Is not possible to query if a promise has been declared.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D158464
There are two motivations for this change:
1. It considerably simplifies adding support for the realloc operation to the
new buffer deallocation pass by lowering the realloc such that no
deallocation operation is inserted and the deallocation pass itself can
insert that dealloc
2. The lowering is expressed on a higher level and thus easier to understand,
and the lowerings of the memref operations it is composed of don't have to
be duplicated in the MemRefToLLVM lowering (also see discussion in
https://reviews.llvm.org/D133424)
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D159430
This patch adds a new class Identifier to store identifiers in PresburgerSpace
and their types.
Identifiers were added earlier and were stored as a void pointer, and their type
in the form of mlir::TypeId in PresburgerSpace. To get an identifier, a user of
PresburgerSpace needed to know the type of identifiers. This was a problem for
users of PresburgerSpace like IntegerRelation, which want to work on
identifiers without knowing their type.
The Identifier class allows users like IntegerRelation to work on identifiers
without knowing their type, and also exposes an easier way to work with
Identifiers.
Reviewed By: arjunp
Differential Revision: https://reviews.llvm.org/D146909
Fix a typo (incorrectly calling getNumDomainVars instead of
getNumRangeVars) in intersectRange from 3dd9931.
Reviewed By: Groverkss
Differential Revision: https://reviews.llvm.org/D159413
`TargetOptions` includes a field for passing additional command line options to
the GPU compilation process. This field is typically used during the 'gpu-module-to-binary`
pass:
```
--gpu-module-to-binary=opts="-v -c"
```
The problem is that `tokenizeCmdOptions` receives the quoted string, which produces an
incorrect tokenization for `"-v -c"`. This patch removes quotes, fixing this issue.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D159434
The newly folded patterns get produced by type emulation code, so we
cannot rely on the Arith folders handling these folds.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D159394
Printing cyclic attributes and types currently has no first-class
support within the AsmPrinter and AsmParser. The workaround for this
issue used in all mutable attributes and types upstream has been to
create a `thread_local static SetVector` keeping track of currently
parsed and printed attributes.
This solution is not ideal readability wise due to the use of globals
and keeping track of state. Worst of all, this pattern had to be
reimplemented for every mutable attribute and type.
This patch therefore adds support for this pattern in `AsmPrinter` and
`AsmParser` replacing the use of this pattern. By calling
`tryStartCyclingPrint/Parse`, the mutable attribute or type are
registered in an internal stack. All subsequent calls to the function
with the same attribute or type will lead to returning failure. This way
the nesting can be detected and a short form printed or parsed instead.
Through the resetter returned by the call, the cyclic printing or
parsing region automatically ends on return.
FuncToLLVM uses the data layout string attribute in 3 different ways:
1. LowerToLLVMOptions options(&getContext(), getAnalysis<DataLayoutAnalysis>().getAtOrAbove(m));
2. options.dataLayout = llvm::DataLayout(this->dataLayout);
3. m->setAttr(..., this->dataLayout));
The 3rd way is unrelated to the other 2 and occurs after conversion, making it confusing.
This revision separates this post-hoc module annotation functionality into its own pass.
The convert-func-to-llvm pass loses its `data-layout` option and instead recovers it from
the `llvm.data_layout` attribute attached to the module, when present.
In the future, `LowerToLLVMOptions options(&getContext(), getAnalysis<DataLayoutAnalysis>().getAtOrAbove(m))` and
`options.dataLayout = llvm::DataLayout(dataLayout);` should be unified.
Reviewed By: ftynse, mehdi_amini
Differential Revision: https://reviews.llvm.org/D157604
The same transform op can now be used to apply registered pass pipelines.
This revision also adds a helper function for querying `PassPipelineInfo` objects and moves the corresponding `lookup` function for `PassInfo` objects to the `PassInfo` class.
Differential Revision: https://reviews.llvm.org/D159211
`bufferize_to_allocation` does not supports ops with regions, unless `bufferize_destination_only` is set. In that case, only the operand is replaced with an allocation and wrapped in a `to_tensor` op. The error checking was too strict.
Differential Revision: https://reviews.llvm.org/D159420
This commit enables the expression of negated native constraints in PDLL:
If a constraint is prefixed with "not" it is parsed as a negated constraint and hence the attribute `isNegated` of the emitted `pdl.apply_native_constraint` operation is set to `true`.
In first instance this is only supported for the calling of external native C++ constraints and generation of PDL patterns.
Previously, negating a native constraint would have been handled by creating an additional native call, e.g.
```PDLL
Constraint checkA(input: Attr);
Constarint checkNotA(input: Attr);
```
or by including an explicit additional operand for negation, e.g.
`Constraint checkA(input: Attr, negated: Attr);`
With this a constraint can simply be negated by prefixing it with `not`. e.g.
```PDLL
Constraint simpleConstraint(op: Op);
Pattern example {
let inputOp = op<test.bar>() ->(type: Type);
let root = op<test.foo>(inputOp.0) -> ();
not simpleConstraint(inputOp);
simpleConstraint(root);
erase root;
}
```
Depends on [[ https://reviews.llvm.org/D153871 | D153871 ]]
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D153959
Incorporated two header files directly into other since
other parts were used (and it makes it hard to find the
definitions). Removed TODOs that are less likely to be done.
Reviewed By: yinying-lisa-li
Differential Revision: https://reviews.llvm.org/D159381
Incorporated two header files directly into other since
other parts were used (and it makes it hard to find the
definitions). Removed TODOs that are less likely to be done.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D159330
This patch moves the call for translating an MLIR module to LLVM IR to the
beginning of the translation process. This enables the use of dialect
attributes attached to `builtin.module` operations and the `amendOperation()`
flow to initialize dialect-specific global configuration before translating
the contents of the module.
Currently, this patch does not impact the generated IR on its own. Testing
infrastructure to allow translating the Test dialect to LLVM IR is added, so
that it can be checked that the current behavior is not broken in the future.
Differential Revision: https://reviews.llvm.org/D158278
The revision adds the dbg label intrinsic to the allow list of
operations that are legal to inline.
Reviewed By: zero9178
Differential Revision: https://reviews.llvm.org/D159359
This reverts a feature introduced in commit
2a5d497494c24425e99655b85e2277dd3f15a400. The goal of that commit was to
allow `StringAttr`s to by used transparently wherever Python `str`s are
expected. But, as the tests in https://reviews.llvm.org/D159182 reveal,
pybind11 doesn't do this conversion based on `__str__` automatically,
unlike for the other types introduced in the commit above. At the same
time, changing `__str__` breaks the symmetry with other attributes of
`print(attr)` printing the assembly of the attribute, so the change
probably has more disadvantages than advantages.
Reviewed By: springerm, rkayaith
Differential Revision: https://reviews.llvm.org/D159255
This was introduced before the Optional directive and uses Variadic, but
it's really optional.
Reviewed By: nicolasvasilache, benmxwl-arm, dcaballe
Differential Revision: https://reviews.llvm.org/D159259
