When establishing the correspondence between transform values and
payload operations or parameters, check that the latter are non-null and
report errors. This was previously allowed for exotic cases of partially
successfull transformations with "apply each" trait, but was dangerous.
The "apply each" implementation was reworked to remove the need for this
functionality, so this can now be hardned to avoid null pointer
dereferences.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D141142
Adapt the implementation of TransformEachOpTrait to the existence of
parameter values recently introduced into the transform dialect. In
particular, allow `applyToOne` hooks to return a list containing a mix
of `Operation *` that will be associated with handles and `Attribute`
that will be associated with parameter values by the trait
implementation of the transform interface's `apply` method.
Disentangle the "transposition" of the list of per-payload op partial
results to decrease its overall complexity and detemplatize the code
that doesn't really need templates. This removes the poorly documented
special handling for single-result ops with TransformEachOpTrait that
could have assigned null pointer values to handles.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D140979
This makes it more consistent with the recently added
TransformParamTypeInterface.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D140977
Introduce a new kind of values into the transform dialect -- parameter
values. These values have a type implementing the new
`TransformParamTypeInterface` and are associated with lists of
attributes rather than lists of payload operations. This mechanism
allows one to wrap numeric calculations, typically heuristics, into
transform operations separate from those at actually applying the
transformation. For example, tile size computation can be now separated
from tiling itself, and not hardcoded in the transform dialect. This
further improves the separation of concerns between transform choice and
implementation.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D140976
This is part of an effort to migrate from llvm::Optional to
std::optional. 22426110c5ef changed the way mlir-tblgen generates .inc
files, emitting std::optional when an Optional attribute is specified in
a .td file. It also changed several .td files hard-coding llvm::Optional
to use std::optional. However, the patch excluded a few .td files in
SPIRV and Bufferization hard-coding llvm::Optional. This patch fixes
that defect, and after this patch, references to llvm::Optional in .cpp
and .h files can be replaced mechanically.
See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>
Differential Revision: https://reviews.llvm.org/D140329
The pattern generalizes a tensor::UnPackOp into a sequence of tensor +
Linalg ops, when the outer dims are all 1s. It uses the trick of
rank-reduced tensor.extract_slice to get the tile; transpose the tile;
extract sub tile for incomplete cases if needed; use tensor.insert_slice
to insert it to the destination tensor.
Reviewed By: tyb0807, chelini
Differential Revision: https://reviews.llvm.org/D140254
std::optional::value() has undesired exception checking semantics and is
unavailable in older Xcode (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS). The
call sites block std::optional migration.
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch changes the way mlir-tblgen generates .inc
files, and modifies tests and documentation appropriately. It is a "no
compromises" patch, and doesn't leave the user with an unpleasant mix of
llvm::Optional and std::optional.
A non-trivial change has been made to ControlFlowInterfaces to split one
constructor into two, relating to a build failure on Windows.
See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>
Differential Revision: https://reviews.llvm.org/D138934
The pattern generalizes a tensor::PackOp into a sequence of tensor +
Linalg ops, when the outer dims are all 1s. It uses the trick of
rank-reduced tensor.extract_slice to get the tile; transpose the tile;
use tensor.insert_slice to insert it to the destination of inner tile.
Reviewed By: pifon2a, tyb0807
Differential Revision: https://reviews.llvm.org/D140060
The assert message was previously ignored. The lowered IR now calls `puts` it in case of a failed assertion.
Differential Revision: https://reviews.llvm.org/D138647
This patch adds the `ExpandStridedMetadataPass` to the LowerToLLVM named
pipeline.
This is required now that "complex" memref operations, like subviews,
need to be expanded before being lowered.
Differential Revision: https://reviews.llvm.org/D139841
This patch changes the `test-lower-to-llvm` pass into a named pipeline.
The functionality is unchanged but thanks to this change, we don't have
to pull the dependencies of all the passes that this pass calls.
In other words, `TestLowerToLLVMPass::getDependDialects` was supposed to
transitively declare all the dialects that were used in the union of all
the invoked passes.
NFC
Differential Revision: https://reviews.llvm.org/D139840
If an input bbArg is not used, its corresponding input operand is removed. If there are duplicate input operands or input operands that are also used as output operands, the duplicate input operands are removed. Output operands are never modified.
Differential Revision: https://reviews.llvm.org/D139709
Originally, inferReturnTensorTypes didn't support shaped type components
containing an attribute just because there wasn't any motivating use-case.
Removing that limitation and using it to set the encoding attribute for
RankedTensorType.
Updated the existing test to set result attribute based on the first operand,
if available.
Signed-off-by: Smit Hinsu <smittvhinsu@gmail.com>
Differential Revision: https://reviews.llvm.org/D139271
At the moment, they are a part of EmptyOp::getCanonicalizationPatterns. When
extract_slice(tensor.empty) is rewritten as a new tensor.empty, it could
happen that we end up with two tensor.empty ops, since the original
tensor.empty can have two users. After bufferization such cases result in two
allocations.
Differential Revision: https://reviews.llvm.org/D139308
This covers `SDot`, `SUDot`, and `UDot`. The `*AccSat` version will be
added in a follow-up revision.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D139242
It introduces a pattern that swaps `linalg.generic + tensor.pack` to
`tensor.pack + linalg.generic`. It requires all the iteration types
being parallel; the indexing map of output operand is identiy. They can
all be relaxed in the future.
The user can decide whether the propagation should be applied or not by
passing a control function.
Reviewed By: mravishankar
Differential Revision: https://reviews.llvm.org/D138882
Currently, assemblyFormat `custom<A>($a) custom<B>($b)` has different spacing
if used for Ops vs Attrs/Types. Ops insert a space if needed before the custom directive,
while attributes and types do not.
This leads to the following two patterns in attributes / types:
```
# 1. Whitespace literal
let assemblyFormat = "... ` ` custom<A>($a)"
# 2. Custom printer code includes spacing
void printB(...) {
printer << ' ' << b;
}
```
Moving this spacing into the generated code allows for some cleanup in mlir and
improves the consistency of custom directives.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D138235
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Define rewrite patterns in-line so that we don't have to jump around so
much when editing this file.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D139239
TensorCopyInsertion should not have been exposed as a pass. This was a flaw in the original design. It is a preparation step for bufferization and certain transforms (that would otherwise be legal) are illegal between TensorCopyInsertion and actual rewrite to MemRef ops. Therefore, even if broken down as two separate steps internally, they should be exposed as a single pass.
This change affects the sparse compiler, which uses `TensorCopyInsertionPass`. A new `SparsificationAndBufferizationPass` is added to replace all passes in the sparse tensor pipeline from `TensorCopyInsertionPass` until the actual bufferization (rewrite to memref/non-tensor). It is generally unsafe to run arbitrary passes in-between, in particular passes that hoist tensor ops out of loops or change SSA use-def chains along tensor ops.
Differential Revision: https://reviews.llvm.org/D138915
This reverts commit 4e6dab98e0cb60c635656a818062887d97c3ef5f.
Re-apply: D138988 after fixing error on windows. Remove test for boolean
attributes as it does not make sense to apply these constraints on
boolean array.
This reverts commit dd0de4dca92cd6affafb47f788b64e99187168f1.
Build on mlir-windows fails:
Step 6 (build-check-mlir-build-only) failure: build (failure)
C:\buildbot\mlir-x64-windows-ninja\build\tools\mlir\test\lib\Dialect\Test\TestOps.cpp.inc(928): error C2220: the following warning is treated as an error
C:\buildbot\mlir-x64-windows-ninja\build\tools\mlir\test\lib\Dialect\Test\TestOps.cpp.inc(928): warning C4804: '>': unsafe use of type 'bool' in operation
C:\buildbot\mlir-x64-windows-ninja\build\tools\mlir\test\lib\Dialect\Test\TestOps.cpp.inc(7419): warning C4804: '>': unsafe use of type 'bool' in operation
- `DenseArrayStrictlyPositive` all elements are required to be > 0.
Returns true if the range is empty.
- `DenseArrayNonNegative` all elements are required to be >= 0. Returns
true if the range is empty.
Both constraints will simplify verifier logic as we move from using `I64ArrayAttr` to `DenseI64ArrayAttr`.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D138988
This fixes the case where scf::LoopNest::loops is empty.
Change LoopVector and ValueVector to SmallVector.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D136926
This commit extends the `ResourceLimitsAttr` to support specifying
a minimal and maximal subgroup size, and extends `EntryPointABIAttr`
to support specifying the requested subgroup size. This is possible
now in Vulkan with the VK_EXT_subgroup_size_control extension.
For OpenCL it's possible to use the `SubgroupSize` execution mode
directly.
Reviewed By: ThomasRaoux
Differential Revision: https://reviews.llvm.org/D138962
This helper handles non trivial cases of broadcast + optional transpose creation
that should not leak to the outside world.
Differential Revision: https://reviews.llvm.org/D139003
This is generated by running
```
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.td
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.mlir
```
Reviewed By: rriddle, dcaballe
Differential Revision: https://reviews.llvm.org/D138866
This allows writing simple e2e tests where we can check for the proper materialization
of specific LLVM IR (e.g. `llvm.intr.fmuladd`).
Differential Revision: https://reviews.llvm.org/D138776
This revision refactors and cleans up a bunch of infra related to vector, shapes and indexing into more reusable APIs.
Differential Revision: https://reviews.llvm.org/D138501
Elementwise op fusion conserves the result of the producer in the
fused op, relying on later clean up patterns to drop unused results of
the fused op. Instead, if the producer result has no other use apart
from the consumer op, avoid making the producer result available in
the fused node. This saves some unnecessary IR manipulations.
Differential Revision: https://reviews.llvm.org/D138096
This patch introduces a templated FieldParser to handle optional signed and unsigned integer types - NFC. Additionally, I've added an extra test to ensure that both signed and unsigned integers are properly tested in the templated integer types for FieldParser as well.
This patch adds `parseBase64Bytes` to the parser. It attempts to avoid double-allocating the buffer by re-using the token's spelling directly and eliding the quotes if they exist. It also avoids extra allocations by using std::vector<char> in the API - something we should change when the llvm::decodeBase64 API changes.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D138090
Ops that were modifed in-place (`finalizeRootUpdate` was called) should be reprocessed by the GreedyPatternRewriter. This is currently not happening with `GreedyRewriteConfig::maxIterations = 1`.
Note: If your project goes into an infinite loop because of this change, you likely have one or multiple faulty patterns that modify the same operations in-place (`updateRootInplace`) indefinitely.
Differential Revision: https://reviews.llvm.org/D138038
The patterns to remove dead arguments and results of `linalg.generic`
operations are not necessarily canonicalizations. Instead a new entry
point `populateEraseUnusedOperandsAndResults` is added to allow using
these patterns when needed. The transformations that rely on this
pattern for cleanup now include these patterns explicitly.
Differential Revision: https://reviews.llvm.org/D138085
Currently CSE does not support CSE of ops with regions. This patch
extends the CSE support to ops with a single region.
Differential Revision: https://reviews.llvm.org/D134306
Depends on D137857
