This patch introduces an MLIR attribute to the OpenMP dialect
representing the clauses that a 'requires' directive can define.
The `OffloadModuleInterface` is also updated to provide methods to get
and set a new dialect attribute `omp.requires`, to allow storing and using this
information during the lowering stages to LLVM IR.
Differential Revision: https://reviews.llvm.org/D147214
This commit changes intrinsics that have immarg parameter attributes to
model these parameters as attributes, instead of operands. Using
operands only works if the operation is an `llvm.mlir.constant`,
otherwise the exported LLVMIR is invalid.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D151692
SmallSetVector has an inefficiency where it does set insertions
regardless of the number of elements present within it. This contrasts
with other "Small-" containers where they use linear scan up to a
certain size "N", after which they switch to another strategy.
This patch implements this functionality in SetVector, adding a template
parameter "N" which specifies the number of elements upto which the
SetVector follows the "small" strategy. Due to the use of "if
constexpr", there is no "small" code emitted when N is 0 which makes
this a zero overhead change for users using the default behaviour.
This change also allows having SmallSetVector use DenseSet instead of
SmallDenseSet by default, which helps a little with performance.
The reason for implementing this functionality in SetVector instead of
SmallSetVector is that it allows reusing all the code that is already
there and it is just augmented with the "isSmall" checks.
This change gives a good speedup (0.4%):
https://llvm-compile-time-tracker.com/compare.php?from=086601eac266ec253bf313c746390ff3e5656132&to=acd0a72a4d3ee840f7b455d1b35d82b11ffdb3c0&stat=instructions%3Au
Differential Revision: https://reviews.llvm.org/D152497
Promised interfaces allow for a dialect to "promise" the implementation of an interface, i.e.
declare that it supports an interface, but have the interface defined in an extension in a library
separate from the dialect itself. A promised interface is powerful in that it alerts the user when
the interface is attempted to be used (e.g. via cast/dyn_cast/etc.) and the implementation has
not yet been provided. This makes the system much more robust against misconfiguration,
and ensures that we do not lose the benefit we currently have of defining the interface in
the dialect library.
Differential Revision: https://reviews.llvm.org/D120368
ApplyEachOpTrait applies to payload ops associated with its operand
handle one-by-one in order. If a handle is consumed, this usually
indicates that the associated payload ops are erased or rewritten. Add a
check that we don't consume an ancestor payload operation before
consuming its descendant, as the latter is likely to be a dangling
pointer. Transform operations for which this is a legitimate behavior
(i.e., they consume the handle but don't actually erase or rewrite the
payload operation) should implement the interface directly and allow for
repeated handles.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D152510
This is useful for transformations such as bufferization, which is looking for tensor.extract_slice/insert_slice pairs.
Also fix the documentation of the corresponding tranform op.
Differential Revision: https://reviews.llvm.org/D152455
A TransformRewriter (with attached TrackingListener) will be added to an interface method in a subsequent revision.
Differential Revision: https://reviews.llvm.org/D152426
This type was removed in D151520, but downstream dependencies use it. By
introducing a temporary allias, we can give these dependencies a chance to
update their uses of the type and give them a chance to migrate without
breakages.
Reviewed By: csigg
Differential Revision: https://reviews.llvm.org/D152509
When looking for replacement ops (`findReplacementOp`) distinguish between "no replacement could be found" and "this op should be dropped from the mapping". The latter case will be utilized in a subsequent revision when a payload op is mapped to a consumed handle.
Differential Revision: https://reviews.llvm.org/D152375
This patch enables specifying scalable vector sizes when using the
Transform dialect to drive vectorisation, e.g.:
```
transform.structured.masked_vectorize %0 vector_sizes [8, 16, [4]]
```
This is implemented by extending the MaskedVectorizeOp with a dedicated
attribute for "scalability" and by overloading `parseDynamicIndexList`
so that MaskedVectorizeOp can continue using the auto-generated parser
and printer.
At the moment, only the trailing vec size can be scalable. The following
is not yet supported:
```
transform.structured.masked_vectorize %0 vector_sizes [8, [16], [4]]
```
As the vectoriser does not support scalable vectorisation just yet, a
warning is issues when scalable vector sizes are used. You can also use
the debug output, `--debug-only=linalg-vectorization`, to check whether
scalable vectorisation has been switched on.
This change is a part of a larger effort to enable scalable
vectorisation in Linalg. See this RFC for more context:
* https://discourse.llvm.org/t/rfc-scalable-vectorisation-in-linalg/
Similar patch for tiling: https://reviews.llvm.org/D150944
Differential Revision: https://reviews.llvm.org/D151892
Add an options object to allow control of the slice computation (for
both forward and backward slice). This makes the ABI stable, and also
allows avoiding an assert that makes the slice analysis unusable for
operations with multiple blocks.
Reviewed By: hanchung, nicolasvasilache
Differential Revision: https://reviews.llvm.org/D151520
IRDL operations were inconsistent in their naming. They now
all end with the `Op` suffix.
Reviewed By: rriddle, jpienaar
Differential Revision: https://reviews.llvm.org/D152354
`irdl.c_pred` is an attribute constraint defined by a C++ predicate.
Contrary to the other constraints, this operation cannot be used in
dialects that are registered at runtime. Its principal use is to
share dialect definitions that are defined in C++ or ODS.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D152301
The `quant` dialect documentation currently mostly empty (https://mlir.llvm.org/docs/Dialects/QuantDialect/).
This patch moves the comments to TableGen statements.
By looking at the generated `QuantDialect.md`, I've confirmed that each change in this patch will end up in the documentation.
The only thing that I wasn't able to document was the `UniformQuantizedType`.
I haven't found a way to document a `DialectType` since most appear to be private in other dialects; suggestions are welcome.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D151649
Even though this feature was deprecated in release 11.2,
any library before this version still supports the feature,
which is why we are making it available under a macro.
Reviewed By: K-Wu
Differential Revision: https://reviews.llvm.org/D152290
Document better that unary/binary may only feed to the output
or the input of a custom reduction (not even a regular reduction
since it may have "no value"!). Also fixes a bug when present
branch is empty and feeds into custom reduction.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D152224
* Remove `transform::PatternRegistry`.
* Add a new op for each currently registered pattern set.
* Change names of vector dialect pattern selector ops, so that they are consistent with the remaining code base.
* Remove redundant `transform.vector.extract_address_computations` op.
Differential Revision: https://reviews.llvm.org/D152249
Add a TransformInterpreterPassBase capability to generate the (shared)
module containing the transform script during the pass initialization.
This is helpful to programmatically generate the script as opposed to
parsing it from the textual module.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D152185
All vector transform ops are now `PatternDescriptorOpInterface` ops that merely select the patterns. The patterns are applied by the `apply_patterns` op. This is to ensure that ops are properly tracked. (TrackingListener is used in the implementation of `apply_patterns`.) Furthermore, handles are no longer invalidated when applying patterns in the vector tests.
Differential Revision: https://reviews.llvm.org/D152174
Patterns should be selected by adding ops that implement `PatternDescriptorOpInterface` to the region of `apply_pattern` ops. Such ops can have operands, allowing for pattern parameterization. The existing way of selecting patterns from the PatternRegistry is deprecated.
Differential Revision: https://reviews.llvm.org/D152167
Consider mixed precision data type, i.e., F16 input lhs, F16 input rhs, F32 accumulation, and F32 output. This is typically written as F32 <= F16*F16 + F32.
During vectorization from linalg to vector for mixed precision data type (F32 <= F16*F16 + F32), linalg.matmul introduces arith.extf on input lhs and rhs operands.
"linalg.matmul"(%lhs, %rhs, %acc) ({
^bb0(%arg1: f16, %arg2: f16, %arg3: f32):
%lhs_f32 = "arith.extf"(%arg1) : (f16) -> f32
%rhs_f32 = "arith.extf"(%arg2) : (f16) -> f32
%mul = "arith.mulf"(%lhs_f32, %rhs_f32) : (f32, f32) -> f32
%acc = "arith.addf"(%arg3, %mul) : (f32, f32) -> f32
"linalg.yield"(%acc) : (f32) -> ()
})
There are backend that natively supports mixed-precision data type and does not need the arith.extf. For example, NVIDIA A100 GPU has mma.sync.aligned.*.f32.f16.f16.f32 that can support mixed-precision data type. However, the presence of arith.extf in the IR, introduces the unnecessary casting targeting F32 Tensor Cores instead of F16 Tensor Cores for NVIDIA backend. This patch adds a folding pattern to fold arith.extf into vector.contract
Differential Revision: https://reviews.llvm.org/D151918
This patch updates the docs for fuse_into_containing_op. It
updates the returned values to include the new_containing_op
and adds a brief description of the new_containing_op.
updates the docs with new changes in the
op regarding the return of the new_containing_op as well
as a brief description o
Differential Revision: https://reviews.llvm.org/D152044
Certain functions were declared in `VectorOps.h` instead of `VectorTransforms.h` or `VectorRewritePatterns.h`.
Differential Revision: https://reviews.llvm.org/D152146
Also support replacing payload ops with ConstantLike ops in the TrackingListener, even if the replacement op does not have the same name. (Not supported for ops with multiple results, as this would require splitting the handle.)
Differential Revision: https://reviews.llvm.org/D152127
Before serializing, optimizations on llvm were only called on path to
hsaco, and not cubin. Define opt-level for `gpu-to-cubin` pass as well,
and move call to optimize llvm to a common place.
Reviewed By: bondhugula
Differential Revision: https://reviews.llvm.org/D151554
Previously the SignedVarInt was incorrectly defined. Follow up work
needed for improving Array printing/parsing, but correcting the
definitions for now.
Add a new transform op that applies patterns to a targeted payload op. Patterns can be registered by transform dialect extensions in a pattern registry.
Differential Revision: https://reviews.llvm.org/D151983
