This commit improves the block signature conversion API of the dialect
conversion.
There is the following comment in
`ArgConverter::applySignatureConversion`:
```
// If no arguments are being changed or added, there is nothing to do.
```
However, the implementation actually used to replace a block with a new
block even if the block argument types do not change (i.e., there is
"nothing to do"). This is fixed in this commit. The documentation of the
public `ConversionPatternRewriter` API is updated accordingly.
This commit also removes a check that used to *sometimes* skip a block
signature conversion if the block was already converted. This is not
consistent with the public `ConversionPatternRewriter` API; blocks
should always be converted, regardless of whether they were already
converted or not.
Block signature conversion also used to be silently skipped when the
specified block was detached. Instead of silently skipping, an assertion
is triggered. Attempting to convert a detached block (which is likely an
erased block) is invalid API usage.
This PR replaces the generation of `vector.shuffle` with
`vector.interleave` in the i4 conversions in vector narrow type
emulation. The multi dimensional semantics of `vector.interleave` allow
us to enable these conversion emulations also for multi dimensional
vectors.
The `SerializeToCubin` pass was deprecated in September 2023 in favor of
GPU compilation attributes; see the [GPU
compilation](https://mlir.llvm.org/docs/Dialects/GPU/#gpu-compilation)
section in the `gpu` dialect MLIR docs.
This patch removes `SerializeToCubin` from the repo.
I believe the semantics should be the same, but this saves 1 op and simplifies the code.
For example, the following two instructions:
```
%2 = cmp sgt %0, %1
%3 = select %2, %0, %1
```
Are equivalent to:
```
%2 = maxsi %0 %1
```
Summary:
Currently, OpenMP handles the `omp requires` clause by emitting a global
constructor into the runtime for every translation unit that requires
it. However, this is not a great solution because it prevents us from
having a defined order in which the runtime is accessed and used.
This patch changes the approach to no longer use global constructors,
but to instead group the flag with the other offloading entires that we
already handle. This has the effect of still registering each flag per
requires TU, but now we have a single constructor that handles
everything.
This function removes support for the old `__tgt_register_requires` and
replaces it with a warning message. We just had a recent release, and
the OpenMP policy for the past four releases since we switched to LLVM
is that we do not provide strict backwards compatibility between major
LLVM releases now that the library is versioned. This means that a user
will need to recompile if they have an old binary that relied on
`register_requires` having the old behavior. It is important that we
actively deprecate this, as otherwise it would not solve the problem of
having no defined init and shutdown order for `libomptarget`. The
problem of `libomptarget` not having a define init and shutdown order
cascades into a lot of other issues so I have a strong incentive to be
rid of it.
It is worth noting that the current `__tgt_offload_entry` only has space
for a 32-bit integer here. I am planning to overhaul these at some point
as well.
This PR adds an optional bitwidth parameter to the vector xfer op
flattening transformation so that the flattening doesn't happen if the
trailing dimension of the read/writen vector is larger than this
bitwidth (i.e., we are already able to fill at least one vector register
with that size).
When a `ModifyOperationRewrite` is committed, the operation may already
have been erased, so `OperationName` must be cached in the rewrite
object.
Note: This will no longer be needed with #81757, which adds a "cleanup"
method to `IRRewrite`.
* When converting a block signature, `ArgConverter` creates a new block
with the new signature and moves all operation from the old block to the
new block. The new block is temporarily inserted into a region that is
stored in `regionMapping`. The old block is not yet deleted, so that the
conversion can be rolled back. `regionMapping` is not needed. Instead of
moving the old block to a temporary region, it can just be unlinked.
Block erasures are handles in the same way in the dialect conversion.
* `regionToConverter` is a mapping from regions to type converter. That
field is never accessed within `ArgConverter`. It should be stored in
`ConversionPatternRewriterImpl` instead.
* `convertedBlocks` is not needed. Old blocks are already stored in
`ConvertedBlockInfo`.
The dialect conversion rolls back in-place op modifications upon
failure. Rolling back modifications of attributes is already supported,
but there was no support for properties until now.
This commit simplifies the internal state of the dialect conversion. A
separate field for the previous state of in-place op modifications is no
longer needed.
This PR adds folders for linalg transpose ops with only one dimension or
an identity permutation. The folding removes the `linalg.transpose` and
just propagates the input tensor.
Implement OpAsmOpInterface for most ops to increase IR readability. For
example `mesh.process_linear_index` would produce a value with name
`proc_linear_idx`.
Added support to vectorized tensor.unpack. The unpack Op is split into a
`vector.transfer_read`, `vector.transpose`, `vector.shape_cast` and a
`vector.transfer_write`.
As part of the renaming the Standard dialect to Func dialect, *support*
for the `func.constant` operation was added to the emitter. However, the
emitter cannot emit function types. Hence the emission for a snippet
like
```
%0 = func.constant @myfn : (f32) -> f32
func.func private @myfn(%arg0: f32) -> f32 {
return %arg0 : f32
}
```
failes with `func.mlir:1:6: error: cannot emit type '(f32) -> f32'`.
This removes `func.constant` from the emitter.
Rename listener callback names:
* `notifyOperationRemoved` -> `notifyOperationErased`
* `notifyBlockRemoved` -> `notifyBlockErased`
The current callback names are misnomers. The callbacks are triggered
when an operation/block is erased, not when it is removed (unlinked).
E.g.:
```c++
/// Notify the listener that the specified operation is about to be erased.
/// At this point, the operation has zero uses.
///
/// Note: This notification is not triggered when unlinking an operation.
virtual void notifyOperationErased(Operation *op) {}
```
This change is in preparation of adding listener support to the dialect
conversion. The dialect conversion internally unlinks IR before erasing
it at a later point of time. There is an important difference between
"remove" and "erase". Lister callback names should be accurate to avoid
confusion.
This is a new ODS feature that allows dialects to define a list of
key/value pair representing an attribute type and a name.
This will generate helper classes on the dialect to be able to
manage discardable attributes on operations in a type safe way.
For example the `test` dialect can define:
```
let discardableAttrs = (ins
"mlir::IntegerAttr":$discardable_attr_key,
);
```
And the following will be generated in the TestDialect class:
```
/// Helper to manage the discardable attribute `discardable_attr_key`.
class DiscardableAttrKeyAttrHelper {
::mlir::StringAttr name;
public:
static constexpr ::llvm::StringLiteral getNameStr() {
return "test.discardable_attr_key";
}
constexpr ::mlir::StringAttr getName() {
return name;
}
DiscardableAttrKeyAttrHelper(::mlir::MLIRContext *ctx)
: name(::mlir::StringAttr::get(ctx, getNameStr())) {}
mlir::IntegerAttr getAttr(::mlir::Operation *op) {
return op->getAttrOfType<mlir::IntegerAttr>(name);
}
void setAttr(::mlir::Operation *op, mlir::IntegerAttr val) {
op->setAttr(name, val);
}
bool isAttrPresent(::mlir::Operation *op) {
return op->hasAttrOfType<mlir::IntegerAttr>(name);
}
void removeAttr(::mlir::Operation *op) {
assert(op->hasAttrOfType<mlir::IntegerAttr>(name));
op->removeAttr(name);
}
};
DiscardableAttrKeyAttrHelper getDiscardableAttrKeyAttrHelper() {
return discardableAttrKeyAttrName;
}
```
User code having an instance of the TestDialect can then manipulate this
attribute on operation using:
```
auto helper = testDialect.getDiscardableAttrKeyAttrHelper();
helper.setAttr(op, value);
helper.isAttrPresent(op);
...
```
The revision does not refactor the inferStaticShape for pack and unpack
ops because they can diverge quickly. Because there are more dimensions
can be inferred (i.e., with inner_tile_sizes) if the pack op does not
have padding value.
This is a follow-up of https://github.com/llvm/llvm-project/pull/80848
Previously, the `tensor.pack` verifier detects unconditional runtime
errors only when tile sizes are static. Now, dynamic tiles are
considered and we only require that the input and either corresponding
tile or output size are static to determine if it will unconditionally
produce errors at runtime.
This mirrors the existing pattern for pushing unpack down through
padding, restricting to cases where the padded dimensions aren't tiled
by the pack.
Additionally reformats the propagation test to make it easier to read.
Extends the `omp.parallel` op by adding a `private` clause to model
[first]private variables. This uses the `omp.private` op to map
privatized variables to their corresponding privatizers.
Example `omp.private` op with `private` variable:
```
omp.parallel private(@x.privatizer %arg0 -> %arg1 : !llvm.ptr) {
^bb0(%arg1: !llvm.ptr):
// ... use %arg1 ...
omp.terminator
}
```
Whether the variable is private or firstprivate is determined by the
attributes of the corresponding `omp.private` op.
Extracts logic from `vector::isContiguousSlice` to check whether
the trailing dim of a memref are contiguous into a dedicated hook
in BuiitinTypes.{h|cpp}.
Follow-up for https://github.com/llvm/llvm-project/pull/76848.
This patch introduces the new MPI dialect into MLIR. The Message Passing
Interface (MPI) is a widely-used standard for distributed programs to
exchange data. This PR goes together with a talk later at today's LLVM
Dev Meeting.
This is just a first, small patch to get going and add the necessary
base files, so that we can add more operations in further patches.
Here's the documentation as generated by `ninja mlir-doc`:
# 'mpi' Dialect
This dialect models the Message Passing Interface (MPI), version
4.0. It is meant to serve as an interfacing dialect that is targeted
by higher-level dialects. The MPI dialect itself can be lowered to
multiple MPI implementations and hide differences in ABI. The dialect
models the functions of the MPI specification as close to 1:1 as
possible
while preserving SSA value semantics where it makes sense, and uses
`memref` types instead of bare pointers.
This dialect is under active development, and while stability is an
eventual goal, it is not guaranteed at this juncture. Given the early
state, it is recommended to inquire further prior to using this dialect.
For an in-depth documentation of the MPI library interface, please refer
to official documentation such as the
[OpenMPI online documentation](https://www.open-mpi.org/doc/current/).
[TOC]
## Operation definition
### `mpi.comm_rank` (mpi::CommRankOp)
_Get the current rank, equivalent to `MPI_Comm_rank(MPI_COMM_WORLD,
&rank)`_
Syntax:
```
operation ::= `mpi.comm_rank` attr-dict `:` type(results)
```
Communicators other than `MPI_COMM_WORLD` are not supported for now.
This operation can optionally return an `!mpi.retval` value that can be
used
to check for errors.
#### Results:
| Result | Description |
| :----: | ----------- |
| `retval` | MPI function call return value
| `rank` | 32-bit signless integer
### `mpi.error_class` (mpi::ErrorClassOp)
_Get the error class from an error code, equivalent to the
`MPI_Error_class` function_
Syntax:
```
operation ::= `mpi.error_class` $val attr-dict `:` type($val)
```
`MPI_Error_class` maps return values from MPI calls to a set of
well-known
MPI error classes.
#### Operands:
| Operand | Description |
| :-----: | ----------- |
| `val` | MPI function call return value
#### Results:
| Result | Description |
| :----: | ----------- |
| `errclass` | MPI function call return value
### `mpi.finalize` (mpi::FinalizeOp)
_Finalize the MPI library, equivalent to `MPI_Finalize()`_
Syntax:
```
operation ::= `mpi.finalize` attr-dict (`:` type($retval)^)?
```
This function cleans up the MPI state. Afterwards, no MPI methods may
be invoked (excpet for MPI_Get_version, MPI_Initialized, and
MPI_Finalized).
Notably, MPI_Init cannot be called again in the same program.
This operation can optionally return an `!mpi.retval` value that can be
used
to check for errors.
#### Results:
| Result | Description |
| :----: | ----------- |
| `retval` | MPI function call return value
### `mpi.init` (mpi::InitOp)
_Initialize the MPI library, equivalent to `MPI_Init(NULL, NULL)`_
Syntax:
```
operation ::= `mpi.init` attr-dict (`:` type($retval)^)?
```
This operation must preceed most MPI calls (except for very few
exceptions,
please consult with the MPI specification on these).
Passing &argc, &argv is not supported currently.
This operation can optionally return an `!mpi.retval` value that can be
used
to check for errors.
#### Results:
| Result | Description |
| :----: | ----------- |
| `retval` | MPI function call return value
### `mpi.recv` (mpi::RecvOp)
_Equivalent to `MPI_Recv(ptr, size, dtype, dest, tag, MPI_COMM_WORLD,
MPI_STATUS_IGNORE)`_
Syntax:
```
operation ::= `mpi.recv` `(` $ref `,` $tag `,` $rank `)` attr-dict `:` type($ref) `,` type($tag) `,` type($rank)(`->` type($retval)^)?
```
MPI_Recv performs a blocking receive of `size` elements of type `dtype`
from rank `dest`. The `tag` value and communicator enables the library
to
determine the matching of multiple sends and receives between the same
ranks.
Communicators other than `MPI_COMM_WORLD` are not supprted for now.
The MPI_Status is set to `MPI_STATUS_IGNORE`, as the status object
is not yet ported to MLIR.
This operation can optionally return an `!mpi.retval` value that can be
used
to check for errors.
#### Operands:
| Operand | Description |
| :-----: | ----------- |
| `ref` | memref of any type values
| `tag` | 32-bit signless integer
| `rank` | 32-bit signless integer
#### Results:
| Result | Description |
| :----: | ----------- |
| `retval` | MPI function call return value
### `mpi.retval_check` (mpi::RetvalCheckOp)
_Check an MPI return value against an error class_
Syntax:
```
operation ::= `mpi.retval_check` $val `=` $errclass attr-dict `:` type($res)
```
This operation compares MPI status codes to known error class
constants such as `MPI_SUCCESS`, or `MPI_ERR_COMM`.
#### Attributes:
<table>
<tr><th>Attribute</th><th>MLIR Type</th><th>Description</th></tr>
<tr><td><code>errclass</code></td><td>::mlir::mpi::MPI_ErrorClassEnumAttr</td><td><details><summary>MPI
error class name</summary>{{% markdown %}}Enum cases:
* MPI_SUCCESS (`MPI_SUCCESS`)
* MPI_ERR_ACCESS (`MPI_ERR_ACCESS`)
* MPI_ERR_AMODE (`MPI_ERR_AMODE`)
* MPI_ERR_ARG (`MPI_ERR_ARG`)
* MPI_ERR_ASSERT (`MPI_ERR_ASSERT`)
* MPI_ERR_BAD_FILE (`MPI_ERR_BAD_FILE`)
* MPI_ERR_BASE (`MPI_ERR_BASE`)
* MPI_ERR_BUFFER (`MPI_ERR_BUFFER`)
* MPI_ERR_COMM (`MPI_ERR_COMM`)
* MPI_ERR_CONVERSION (`MPI_ERR_CONVERSION`)
* MPI_ERR_COUNT (`MPI_ERR_COUNT`)
* MPI_ERR_DIMS (`MPI_ERR_DIMS`)
* MPI_ERR_DISP (`MPI_ERR_DISP`)
* MPI_ERR_DUP_DATAREP (`MPI_ERR_DUP_DATAREP`)
* MPI_ERR_ERRHANDLER (`MPI_ERR_ERRHANDLER`)
* MPI_ERR_FILE (`MPI_ERR_FILE`)
* MPI_ERR_FILE_EXISTS (`MPI_ERR_FILE_EXISTS`)
* MPI_ERR_FILE_IN_USE (`MPI_ERR_FILE_IN_USE`)
* MPI_ERR_GROUP (`MPI_ERR_GROUP`)
* MPI_ERR_INFO (`MPI_ERR_INFO`)
* MPI_ERR_INFO_KEY (`MPI_ERR_INFO_KEY`)
* MPI_ERR_INFO_NOKEY (`MPI_ERR_INFO_NOKEY`)
* MPI_ERR_INFO_VALUE (`MPI_ERR_INFO_VALUE`)
* MPI_ERR_IN_STATUS (`MPI_ERR_IN_STATUS`)
* MPI_ERR_INTERN (`MPI_ERR_INTERN`)
* MPI_ERR_IO (`MPI_ERR_IO`)
* MPI_ERR_KEYVAL (`MPI_ERR_KEYVAL`)
* MPI_ERR_LOCKTYPE (`MPI_ERR_LOCKTYPE`)
* MPI_ERR_NAME (`MPI_ERR_NAME`)
* MPI_ERR_NO_MEM (`MPI_ERR_NO_MEM`)
* MPI_ERR_NO_SPACE (`MPI_ERR_NO_SPACE`)
* MPI_ERR_NO_SUCH_FILE (`MPI_ERR_NO_SUCH_FILE`)
* MPI_ERR_NOT_SAME (`MPI_ERR_NOT_SAME`)
* MPI_ERR_OP (`MPI_ERR_OP`)
* MPI_ERR_OTHER (`MPI_ERR_OTHER`)
* MPI_ERR_PENDING (`MPI_ERR_PENDING`)
* MPI_ERR_PORT (`MPI_ERR_PORT`)
* MPI_ERR_PROC_ABORTED (`MPI_ERR_PROC_ABORTED`)
* MPI_ERR_QUOTA (`MPI_ERR_QUOTA`)
* MPI_ERR_RANK (`MPI_ERR_RANK`)
* MPI_ERR_READ_ONLY (`MPI_ERR_READ_ONLY`)
* MPI_ERR_REQUEST (`MPI_ERR_REQUEST`)
* MPI_ERR_RMA_ATTACH (`MPI_ERR_RMA_ATTACH`)
* MPI_ERR_RMA_CONFLICT (`MPI_ERR_RMA_CONFLICT`)
* MPI_ERR_RMA_FLAVOR (`MPI_ERR_RMA_FLAVOR`)
* MPI_ERR_RMA_RANGE (`MPI_ERR_RMA_RANGE`)
* MPI_ERR_RMA_SHARED (`MPI_ERR_RMA_SHARED`)
* MPI_ERR_RMA_SYNC (`MPI_ERR_RMA_SYNC`)
* MPI_ERR_ROOT (`MPI_ERR_ROOT`)
* MPI_ERR_SERVICE (`MPI_ERR_SERVICE`)
* MPI_ERR_SESSION (`MPI_ERR_SESSION`)
* MPI_ERR_SIZE (`MPI_ERR_SIZE`)
* MPI_ERR_SPAWN (`MPI_ERR_SPAWN`)
* MPI_ERR_TAG (`MPI_ERR_TAG`)
* MPI_ERR_TOPOLOGY (`MPI_ERR_TOPOLOGY`)
* MPI_ERR_TRUNCATE (`MPI_ERR_TRUNCATE`)
* MPI_ERR_TYPE (`MPI_ERR_TYPE`)
* MPI_ERR_UNKNOWN (`MPI_ERR_UNKNOWN`)
* MPI_ERR_UNSUPPORTED_DATAREP (`MPI_ERR_UNSUPPORTED_DATAREP`)
* MPI_ERR_UNSUPPORTED_OPERATION (`MPI_ERR_UNSUPPORTED_OPERATION`)
* MPI_ERR_VALUE_TOO_LARGE (`MPI_ERR_VALUE_TOO_LARGE`)
* MPI_ERR_WIN (`MPI_ERR_WIN`)
* MPI_ERR_LASTCODE (`MPI_ERR_LASTCODE`){{% /markdown
%}}</details></td></tr>
</table>
#### Operands:
| Operand | Description |
| :-----: | ----------- |
| `val` | MPI function call return value
#### Results:
| Result | Description |
| :----: | ----------- |
| `res` | 1-bit signless integer
### `mpi.send` (mpi::SendOp)
_Equivalent to `MPI_Send(ptr, size, dtype, dest, tag, MPI_COMM_WORLD)`_
Syntax:
```
operation ::= `mpi.send` `(` $ref `,` $tag `,` $rank `)` attr-dict `:` type($ref) `,` type($tag) `,` type($rank)(`->` type($retval)^)?
```
MPI_Send performs a blocking send of `size` elements of type `dtype` to
rank
`dest`. The `tag` value and communicator enables the library to
determine
the matching of multiple sends and receives between the same ranks.
Communicators other than `MPI_COMM_WORLD` are not supprted for now.
This operation can optionally return an `!mpi.retval` value that can be
used
to check for errors.
#### Operands:
| Operand | Description |
| :-----: | ----------- |
| `ref` | memref of any type values
| `tag` | 32-bit signless integer
| `rank` | 32-bit signless integer
#### Results:
| Result | Description |
| :----: | ----------- |
| `retval` | MPI function call return value
## Attribute definition
### MPI_ErrorClassEnumAttr
MPI error class name
Syntax:
```
#mpi.errclass<
::mlir::mpi::MPI_ErrorClassEnum # value
>
```
Enum cases:
* MPI_SUCCESS (`MPI_SUCCESS`)
* MPI_ERR_ACCESS (`MPI_ERR_ACCESS`)
* MPI_ERR_AMODE (`MPI_ERR_AMODE`)
* ... *all other MPI error codes*
#### Parameters:
| Parameter | C++ type | Description |
| :-------: | :-------: | ----------- |
| value | `::mlir::mpi::MPI_ErrorClassEnum` | an enum of type
MPI_ErrorClassEnum |
## Type definition
### RetvalType
MPI function call return value
Syntax: `!mpi.retval`
This type represents a return value from an MPI function vall.
This value can be MPI_SUCCESS, MPI_ERR_IN_STATUS, or any error code.
This return value can be compared agains the known MPI error classes
represented by `#mpi.errclass` using the `mpi.retval_check` operation.
The two types duplicated mostly the same values.
Here they are decomposed to carry orthogonal and complimentary
information.
Use `utils::IteratorType` instead of `mesh::IteratorType`. It now has
only parallel and reduction values.
Rename `Partial` to `ReductionKind`.
Add `getReductionLoopIteratorKinds` method to `ShardingInterface`.
We have no need to vectorize affine.apply inside the vectorizing loop.
However, we still need to generate it in the original scalar form. We
have to replace all its operands with the generated scalar operands in
the vectorizing loop, e.g., induction variables.
Introduces conversion of `omp.private`'s regions to the LLVM dialect.
This reuses the already existing conversion pattern for
`ReducetionDeclareOp` and repurposes it to be used for multi-region ops
as well.
