This revision introduces a pattern to lower `gpu.subgroup_reduce` op into to the `nvvm.redux_sync` op. The op must be run by the entire subgroup, otherwise it is undefined behaviour.
It also adds a flag and populate function, because the op is not avaiable for every gpu (sm80+), so it can be used when it is desired.
Depends on D142088
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D142103
This revision fixes the import of LLVM IR to handle debug metadata with
cyclic dependencies. It deletes the elements list of the composite type
if a cyclic dependency is detected. The revision is meant as a band aid
to avoid infinite recursion during the import of cyclic debug metadata.
Long term solutions are currently discussed here:
https://discourse.llvm.org/t/handling-cyclic-dependencies-in-debug-info/67526/4
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D142086
Ptx model has `redux.sync` that performs reduction operation on the data from each predicated active thread in the thread group. It only is available sm80+.
This revision adds redux as on op to nvvm dialect.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D142088
There are now three options:
* `AnyOp` (previously `false`)
* `ExistingAndNewOps` (previously `true`)
* `ExistingOps`: this one is new.
The last option corresponds to what the `applyOpPatternsAndFold(Operation*, ...)` overload is doing. It is now also supported on the `applyOpPatternsAndFold(ArrayRef<Operation *>, ...)` overload.
Differential Revision: https://reviews.llvm.org/D141904
This transform is complementary to the `structured.pack` op which
allows packing a whole op but does not allow transposes on the individual
operands.
`structured.pack_transpose` allows transposing single operands connected to
pack or unpack ops after the fact.
This makes the system overall more composable than e.g. a giant transform
op with all permutation specified at once.
Differential Revision: https://reviews.llvm.org/D142053
Reductions can't be folded into plain arith ops until we can mask
those arith ops.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D141645
This reverts commit 455305624884cf9237143e2ba0635fcc5ba5206a.
Linker error, unbreak build while I work out how to fix it.
Differential Revision: https://reviews.llvm.org/D142142
Implement InferIntRangeInterface for all operations in the Index dialect. The
inference implementation, unlike the one for Arith, accounts for the
fact that Index can be either 64 or 32 bits long by evaluating both
cases. Bounds are stored as if index were i64, but when inferring new
bounds, we compute both f(...) and f(trunc(...)). We then compare
trunc(f(...)) to f(trunc(...)). If they are equal in the relevant
range components, we use the 64-bit range computation, otherwise we
give the range ext(f(trunc(...))) union f(...).
Note that this can cause surprising behavior as seen in the tests,
where, for example, the order of min and max operations impacts the
behavior of the inference. The inference could perhaps be made more
precise in the future (ex. by tracking 32 and 64-bit results
separately and having them influence each other somehow) butt, since
my project targets an index=i32 platform and doesn't see index-valued
values > uint32_max, I'm not too concerned about it.
Depends on https://reviews.llvm.org/D141299
Depends on https://reviews.llvm.org/D141296
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D140899
simd nontemporal construct is represented as a list of variables
which have low locality accross simd iterations
Added verifier of nontemporal clause. MLIR tests were updated to test
correctness of MLIR definition of nontemporal clause.
Differential Revision: https://reviews.llvm.org/D140553
Reviewed By: kiranchandramohan
This commit introduces LLVM's `MemoryEffects` attribute and replaces the
deprecated usage of `llvm.readnone` in the LLVM dialect.
The absence of the attribute on a `LLVMFuncOp` implies that it might
access all kinds of memory. This semantic corresponds to `llvm::Function`'s
behaviour.
Depends on D142002
Differential Revision: https://reviews.llvm.org/D142013
Simplify the handling of silenceable failures in the transform dialect.
Previously, the logic of `TransformEachOpTrait` required that
`applyToEach` returned a list of null pointers when a silenceable
failure was emitted. This was not done consistently and also crept into
ops without this trait although they did not require it. Handle this
case earlier in the interpreter and homogeneously associated preivously
unset transform dialect values (both handles and parameters) with empty
lists of the matching kind. Ignore the results of `applyToEach` for the
targets for which it produced a silenceable failure. As a result, one
never needs to set results to lists containing nulls. Furthermore, the
objects associated with transform dialect values must never be null.
Depends On D140980
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D141305
Use the recently introduced transform dialect parameter mechanism to
perform controllable multi-size tiling with sizes computed at the
transformation time rather than at runtime.
This requires to generalize tile and split structured transform
operations to work with any transform dialect handle types, which is
desirable in itself to avoid unchecked overuse of PDL OperationType.
Reviewed By: shabalin
Differential Revision: https://reviews.llvm.org/D140980
This commit ensures that all functions produced by `FuncToLLVM` drop the
llvm.linkage attribute. Furthermore, it adds a small test that checks if
the readnone attribute is preserved.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D142002
This lets users of FunctionOpInterface finally have the
name/visibility accessors from SymbolOpInterface. This also
lets us remove the clunky "getName" method from FunctionOpInterface.
Differential Revision: https://reviews.llvm.org/D140199
This allows for interfaces to define a set of "base classes",
which are interfaces whose methods/extra class decls/etc.
should be inherited by the derived interface. This more
easily enables combining interfaces and their dependencies,
without lots of awkard casting. Additional implicit conversion
operators also greatly simplify the conversion process.
One other aspect of this "inheritance" is that we also implicitly
add the base interfaces to the attr/op/type. The user can still
add them manually if desired, but this should help remove some
of the boiler plate when an interface has dependencies.
See https://discourse.llvm.org/t/interface-inheritance-and-dependencies-interface-method-visibility-interface-composition
Differential Revision: https://reviews.llvm.org/D140198
SymbolOpInterface overrides the base classof to provide support
for optionally implementing the interface. This is currently placed
in the extraClassDeclarations, but that is kind of awkard given that
it requires underlying knowledge of how the base classof is implemented.
This commit adds a proper "extraClassOf" field to allow interfaces to
implement this, which abstracts away the default classof logic.
Differential Revision: https://reviews.llvm.org/D140197
It is preferred to merge tosa.transpose operations together rather than convert
one to a tosa.reshape. This is to leverage the tosa.transpose -> tosa.transpose
merging canonicalization.
Reviewed By: AviadCo
Differential Revision: https://reviews.llvm.org/D141434
As discussed in https://reviews.llvm.org/D140886, emitting a warning if the old API is used may be beneficial to encourage migration to the new fold API.
This reuse the existing `Deprecated` infrastructure within TableGen, and simply marks the `def` for `kEmitRawAttributesFolder` causing a use of it in a record (even if set within a base class) to emit a warning.
Error message as printed in the terminal:
```
Included from C:/llvm-project/mlir/python/mlir/dialects/TensorOps.td:13:
Included from C:/llvm-project/mlir/include\mlir/Dialect/Tensor/IR/TensorOps.td:12:
C:/llvm-project/mlir/include\mlir/Dialect/Tensor/IR/TensorBase.td:14:5: warning: Using deprecated def `kEmitRawAttributesFolder`
def Tensor_Dialect : Dialect {
^
note: 'useFoldAPI' of 'kEmitRawAttributesFolder' (default) has been deprecated and is pending removal. Please switch to 'kEmitFoldAdaptorFolder'. See https://discourse.llvm.org/t/psa-new-improved-fold-method-signature-has-landed-please-update-your-downstream-projects/67618
```
Differential Revision: https://reviews.llvm.org/D141604
IR is now always parsed in opaque pointer mode, unless
-opaque-pointers=0 is explicitly given. There is no automatic
detection of typed pointers anymore.
The -opaque-pointers=0 option is added to any remaining IR tests
that haven't been migrated yet.
Differential Revision: https://reviews.llvm.org/D141912
`sym_name` is an optional attribute of `ModuleOp`, so it is unsafe to
fetch the underlying value without checking whether it is non-empty.
Such unsafe dereferencing causes the lower-host-to-llvm-calls_fail.mlir
test to segfault. Although this bug existed for four months, it wasn't
triggered, since previous tests executed a code path that used a default
value instead of one fetched from the module attribute.
This patch makes the code use a default value if the optional attribute
does not have a value.
Reviewed By: stella.stamenova
Differential Revision: https://reviews.llvm.org/D141941
Add pattern to lower transfer_write with permutation map that are not
permutation of minor identity map.
Differential Revision: https://reviews.llvm.org/D141815
The op `emitc.include` does not have results and thus will be elided
during canonicalization, which is not correct behavior. This change
removes the 'Pure' trait and adds a canonicalization test.
Reviewed By: jpienaar, marbre
Differential Revision: https://reviews.llvm.org/D141704
The LLVM inliner was missing the `handleTerminator` method in the
Dialect interface implementation.
Fixes#60093
Differential Revision: https://reviews.llvm.org/D141901
This patch made a minor refactor of LoopCoalescing.cpp's walkLoops
templated method and placed it in Affine's LoopUtils.cpp/h.
This method is also renamed as coalescePerfectlyNestedLoops method. This
minor change enables this method to be invoked
by both the original LoopCoalescing pass as well as the newly introduced
loop.coalesce transform op.
The loop.coalesce transform op has the ability to coalesce affine, and
scf loop nests, leveraging existing LoopCoalescing
mechanism. I have created it inside the SCFTransformOps.td instead of
AffineTransformOps.td as it feels to be similar
in spirit as the loop.unroll op that can handle both scf and affine
loops. Please let me know if you feel that this op
should be moved into AffineTransformOps.td instead.
The testcase added illustrates loop.coalesce transform op working for
scf, affine loops (inner, outer) as well as
coalesced loop can be further unrolled (achieving composibility).
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D141202
This revision introduces a `transform.structured.pack` operation to
transform any Linalg operation to a higher-dimensional Linalg operation on
packed operands.
`tensor.pack` (resp. `tensor.unpack`) operations are inserted for the operands
(resp. results) that need to be packed (resp. unpacked) according to the
`packed_sizes` specification.
At the moment, the packing operation always pads with `getZeroAttr` which will
need to be adjusted depending on the consumers.
Packing is limited to those dimensions that are indexed only by AffineDimExpr.
Packing more advanced indexings requires modular arithmetic that is outside the
scoped of a `linalg.generic` at the moment.
Differential Revision: https://reviews.llvm.org/D141860
Refactor the definition of the enums that are used in the lower_vectors
operation of the transformation dialect.
This avoid duplicating the definition of all the configurations that
this operation can trigger.
NFC
Differential Revision: https://reviews.llvm.org/D141867
We should not check the element type for opaque pointers. We should
still check that the value operands have the same type though.
This causes a verifier error when converting instructions.ll to
opaque pointers.
If the pointer type is opaque, we should not check the element type.
This causes a verifier failure when converting instructions.ll to
opaque pointers.
Similar to how `makeArrayRef` is deprecated in favor of deduction guides, do the
same for `makeMutableArrayRef`.
Once all of the places in-tree are using the deduction guides for
`MutableArrayRef`, we can mark `makeMutableArrayRef` as deprecated.
Differential Revision: https://reviews.llvm.org/D141814
When providing with a spirv module as input where no conversion happens
the code didn't defend against broken invariant.
We'll fail the pass here, but it's not clear if it is the right thing
or if the module should just be ignored.
Fixes#59971
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D141856
In the process of creating the MDNodes for the TBAA tag operations
we used to produce incomplete MDNodes like:
```
@__tbaa::@tbaa_tag_4 => !{!null, !null, i64 0}
@__tbaa::@tbaa_tag_7 => !{!null, !null, i64 0}
```
This caused the two tags to map to the same incomplete MDNode due to uniquing.
To prevent this, we have to use temporary MDNodes instead of !null's.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D141726
