Currently ODS generates `populateDefaultAttrs` or
`populateDefaultProperties` based on whether the dialect opted into
usePropertiesForAttributes. But since individual ops might get opted
into using properties (as long as it has one property), it should
actually just check whether the op itself uses properties. Otherwise
`populateDefaultAttrs` will overwrite existing attrs inside properties
when creating an op. Understandably this becomes moot once everything
switches over to using properties, but this fixes it for now.
This PR makes ODS generate `populateDefaultProperties` as long as the op
itself uses properties.
The generated code for dependent dialects is awkwardly formatted, making
the code harder to read. This change reformats the whitespace to align
code in its context and avoid unnecessary empty lines.
Also included are some typo fixes.
Below are examples of the codegen for a dialect before and after the
change.
Before:
```
GPUDialect::GPUDialect(::mlir::MLIRContext *context)
: ::mlir::Dialect(getDialectNamespace(), context, ::mlir::TypeID::get<GPUDialect>()) {
getContext()->loadDialect<arith::ArithDialect>();
initialize();
}
```
After:
```
GPUDialect::GPUDialect(::mlir::MLIRContext *context)
: ::mlir::Dialect(getDialectNamespace(), context, ::mlir::TypeID::get<GPUDialect>()) {
getContext()->loadDialect<arith::ArithDialect>();
initialize();
}
```
Below are examples of the codegen for a pass before and after the
change.
Before:
```
/// Return the dialect that must be loaded in the context before this pass.
void getDependentDialects(::mlir::DialectRegistry ®istry) const override {
registry.insert<func::FuncDialect>();
registry.insert<tensor::TensorDialect>();
registry.insert<tosa::TosaDialect>();
}
```
After:
```
/// Register the dialects that must be loaded in the context before this pass.
void getDependentDialects(::mlir::DialectRegistry ®istry) const override {
registry.insert<func::FuncDialect>();
registry.insert<tensor::TensorDialect>();
registry.insert<tosa::TosaDialect>();
}
```
Rename interface functions as follows:
* `hasTensorSemantics` -> `hasPureTensorSemantics`
* `hasBufferSemantics` -> `hasPureBufferSemantics`
These two functions return "true" if the op has tensor/buffer operands
but not buffer/tensor operands.
Also drop the "ranked" part from the interface, i.e., do not distinguish
between ranked/unranked types.
The new function names describe the functions more accurately. They also
align their semantics with the notion of "tensor semantics" with the
bufferization framework. (An op is supposed to be bufferized if it has
tensor operands, and we don't care if it also has memref operands.)
This change is in preparation of #75273, which adds
`BufferizableOpInterface::hasTensorSemantics`. By renaming the functions
in the `DestinationStyleOpInterface`, we can avoid name clashes between
the two interfaces.
In the process a couple of test transform dialect ops are added just
for testing. These operations are not intended to use as full flushed
out of transformation ops, but are rather operations added for testing.
A separate operation is added to `LinalgTransformOps.td` to convert a
`TilingInterface` operation to loops using the
`generateScalarImplementation` method implemented by the
operation. Eventually this and other operations related to tiling
using the `TilingInterface` need to move to a better place (i.e. out
of `Linalg` dialect)
* Rename mesh.process_index -> mesh.process_multi_index.
* Add mesh.process_linear_index op.
* Add lowering of mesh.process_multi_index into an expression using
mesh.process_linear_index, mesh.cluster_shape and
affine.delinearize_index.
This is useful to lower mesh ops and prepare them for further lowering
where the runtime may have only the linear index of a device/process.
For example in MPI we have a rank (linear index) in a communicator.
According to
https://mlir.llvm.org/docs/DefiningDialects/Operations/#custom-directives,
custom directive supports attr-dict
> attr-dict Directive: NamedAttrList &
But it doesn't support prop-dict which is introduced into MLIR recently.
It's useful to have tblgen support prop-dict like attr-dict. This PR
enable tblgen to support prop-dict
```bash
error: only variables and types may be used as parameters to a custom directive
... custom<Print>(prop-dict)
```
Co-authored-by: Fung Xie <ftse@nvidia.com>
Support WalkResult for AffineExpr walk and support interrupting walks
along the lines of Operation::walk. This allows interrupted walks when a
condition is met. Also, switch from std::function to llvm::function_ref
for the walk function.
Make it so that PDL in pattern rewrites can be optionally disabled.
PDL is still enabled by default and not optional bazel. So this should
be a NOP for most folks, while enabling other to disable.
This only works with tests disabled. With tests enabled this still
compiles but tests fail as there is no lit config to disable tests that
depend on PDL rewrites yet.
Make it so that PDL in pattern rewrites can be optionally disabled.
PDL is still enabled by default and not optional bazel. So this should
be a NOP for most folks, while enabling other to disable.
This is piped through mlir-tblgen invocation and that could be
changed/avoided by splitting up the passes file instead.
This only works with tests disabled. With tests enabled this still
compiles but tests fail as there is no lit config to disable tests that
depend on PDL rewrites yet.
There is a bit of an issue with how `mlir-linalg-ods-yaml-gen` is
classified in the MLIR build. Due to it being a tool, it is excluded
from the install when using `-DLLVM_BUILD_TOOLS=OFF`. However, it is a
necessary component of the build, so it can cause build issues with
users of the installed LLVM, and so I think it should not be excluded.
It is a tablegen-like tool, so my solution is to reclassify it that way
in the build.
The new patterns break down subgroup reduce ops with vector values into
a sequence of subgroup reductions that fit the native shuffle size. The
maximum/native shuffle size is parametrized.
The overall goal is to be able to perform multi-element reductions with
a sequence of `gpu.shuffle` ops.
The `test-lower-to-nvvm` pipeline serves as the common and proper
pipeline for nvvm+host compilation, and it's used across our CUDA
integration tests.
This PR updates the `test-lower-to-nvvm` pipeline to `gpu-lower-to-nvvm`
and moves it within `InitAllPasses.h`. The aim is to call it from
Python, also having a standardize compilation process for nvvm.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
I was not able to fully triage why this just started failing on one of
our bots as it seems that the use was added 4 months ago. I would assume
that it was accidentally coming in transitively in some way as the dep
was definitely missing.
For context, this started failing in [our
byo_llvm](https://github.com/openxla/iree/blob/main/build_tools/llvm/byo_llvm.sh)
build on a stock build of MLIR on top of an existing LLVM. We were
getting:
```
ld.lld: error: undefined symbol: mlir::registerSPIRVDialectTranslation(mlir::DialectRegistry&) >>> referenced by mlir-opt.cpp
>>> tools/mlir-opt/CMakeFiles/mlir-opt.dir/mlir-opt.cpp.o:(main)
```
Does transformations like
all_reduce(x) + all_reduce(y) -> all_reduce(x + y)
max(all_reduce(x), all_reduce(y)) -> all_reduce(max(x, y))
when the all_reduce element-wise op is max.
Added general rewrite pattern HomomorphismSimplification and
EndomorphismSimplification that encapsulate the general algorithm.
Made specialization for all-reduce with respect to
addf, addi, minsi, maxsi, minimumf and maximumf
in the Arithmetic dialect.
This patch expose the type and attribute names in C++ as methods in the
`AbstractType` and `AbstractAttribute` classes, and keep a map of names
to `AbstractType` and `AbstractAttribute` in the `MLIRContext`. Type and
attribute names should be unique.
It adds support in ODS to generate the `getName` methods in
`AbstractType` and `AbstractAttribute`, through the use of two new
variables, `typeName` and `attrName`. It also adds names to C++-defined
type and attributes.
This patch suggests to change two things. Firstly, it adds a source link
above the generated operations docs (above the `emitOpDoc` calls). This
link will point directly to the source TableGen file for the group of
operations. For example, for the current
[`amdgpu`](https://mlir.llvm.org/docs/Dialects/AMDGPU/) page, the link
will add a source link below the "Operation definition" heading pointing
to
[`mlir/include/mlir/Dialect/AMDGPU/IR/AMDGPU.td`](https://github.com/llvm/llvm-project/blob/main/mlir/include/mlir/Dialect/AMDGPU/IR/AMDGPU.td).
The link is wrapped in a "op-definitions-source-link" class which could
allow for custom styling, but it also looks reasonable without custom
styling I think:
Secondly, this patch simplifies the header names such as "Operation
definition" and "Attribute definition" to "Operations" and "Attributes"
respectively. This is in line with manually defined subheadings on pages
such as the one for the
[`vector`](https://mlir.llvm.org/docs/Dialects/Vector/#operations)
dialect.
The `createParallelComputeFunction` crashed when calling
`getFunctionTypeAttrName` during the creation of a new `FuncOp` inside
the pass. The problem is that `getFunctionTypeAttrName` looks up the
attribute name for the function type which in this case is `func.func`.
However, `name.getAttributeNames()` was empty when clients used
`llvm.func` instead of `func.func`.
To fix this, the `func` dialect is now registered as a dependent
dialect. Also, I've added an assertion which could save other people
some time.
Fixes#71281, fixes#64326.
This PR adds "value casting", i.e., a mechanism to wrap `ir.Value` in a
proxy class that overloads dunders such as `__add__`, `__sub__`, and
`__mul__` for fun and great profit.
This is thematically similar to
bfb1ba7526
and
9566ee2806.
The example in the test demonstrates the value of the feature (no pun
intended):
```python
@register_value_caster(F16Type.static_typeid)
@register_value_caster(F32Type.static_typeid)
@register_value_caster(F64Type.static_typeid)
@register_value_caster(IntegerType.static_typeid)
class ArithValue(Value):
__add__ = partialmethod(_binary_op, op="add")
__sub__ = partialmethod(_binary_op, op="sub")
__mul__ = partialmethod(_binary_op, op="mul")
a = arith.constant(value=FloatAttr.get(f16_t, 42.42))
b = a + a
# CHECK: ArithValue(%0 = arith.addf %cst, %cst : f16)
print(b)
a = arith.constant(value=FloatAttr.get(f32_t, 42.42))
b = a - a
# CHECK: ArithValue(%1 = arith.subf %cst_0, %cst_0 : f32)
print(b)
a = arith.constant(value=FloatAttr.get(f64_t, 42.42))
b = a * a
# CHECK: ArithValue(%2 = arith.mulf %cst_1, %cst_1 : f64)
print(b)
```
**EDIT**: this now goes through the bindings and thus supports automatic
casting of `OpResult` (including as an element of `OpResultList`),
`BlockArgument` (including as an element of `BlockArgumentList`), as
well as `Value`.
Previously only unwrapped values were considered for default values in
builders, expand to Attributes given the change to populate defaults. To avoid
overlap between wrapped and unwrapped, the starting index is incremented
depending on if the smallest starting index for default valued args can be
materialized in unwrapped form or not.
Given the above limitation this is pretty small change.
`optional` is unfortunately meant to also represent conditionally set which
means we cannot allow nullptr for all Attributes marked optional at the moment.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D140705
When emitting bytecode, clients can specify a target dialect version to
emit in `BytecodeWriterConfig`. This exposes a target dialect version to
the DialectBytecodeWriter, which can be queried by name and used to
back-deploy attributes, types, and properties.
This relands 6a0f6dd8359b38340442b7e6b14629c1d6c54a81 that was reverted
due to a missing integration test change.
This commit removes the support for lowering Func to LLVM dialect with
typed pointers. Typed pointers have been deprecated for a while now and
it's planned to soon remove them from the LLVM dialect.
Original PR: https://github.com/llvm/llvm-project/pull/70574
Before D155919, when a dialect was leveraging properties to store
attributes with `usePropertiesForAttributes`, the operand segment sizes
attribute was emitted in the property section in sorted order together
with all the other attributes of an op. After D155919, version 5 of the
bytecode was emitting and parsing operand segment sizes after all the
properties of an op, breaking backward compatibility and back
deployment. This patch fixes the emission ordering and allows to parse
bytecode files emitted before (D155919) with version 5 of MLIR bytecode.
The patch also enables to emit correctly version 5 of MLIR bytecode.
This PR replaces the mixin `OpView` extension mechanism with the
standard inheritance mechanism.
Why? Firstly, mixins are not very pythonic (inheritance is usually used
for this), a little convoluted, and too "tight" (can only be used in the
immediately adjacent `_ext.py`). Secondly, it (mixins) are now blocking
are correct implementation of "value builders" (see
[here](https://github.com/llvm/llvm-project/pull/68764)) where the
problem becomes how to choose the correct base class that the value
builder should call.
This PR looks big/complicated but appearances are deceiving; 4 things
were needed to make this work:
1. Drop `skipDefaultBuilders` in
`OpPythonBindingGen::emitDefaultOpBuilders`
2. Former mixin extension classes are converted to inherit from the
generated `OpView` instead of being "mixins"
a. extension classes that simply were calling into an already generated
`super().__init__` continue to do so
b. (almost all) extension classes that were calling `self.build_generic`
because of a lack of default builder being generated can now also just
call `super().__init__`
3. To handle the [lone single
use-case](https://sourcegraph.com/search?q=context%3Aglobal+select_opview_mixin&patternType=standard&sm=1&groupBy=repo)
of `select_opview_mixin`, namely
[linalg](https://github.com/llvm/llvm-project/blob/main/mlir/python/mlir/dialects/_linalg_ops_ext.py#L38),
only a small change was necessary in `opdsl/lang/emitter.py` (thanks to
the emission/generation of default builders/`__init__`s)
4. since the `extend_opview_class` decorator is removed, we need a way
to register extension classes as the desired `OpView` that `op.opview`
conjures into existence; so we do the standard thing and just enable
replacing the existing registered `OpView` i.e.,
`register_operation(_Dialect, replace=True)`.
Note, the upgrade path for the common case is to change an extension to
inherit from the generated builder and decorate it with
`register_operation(_Dialect, replace=True)`. In the slightly more
complicated case where `super().__init(self.build_generic(...))` is
called in the extension's `__init__`, this needs to be updated to call
`__init__` in `OpView`, i.e., the grandparent (see updated docs).
Note, also `<DIALECT>_ext.py` files/modules will no longer be automatically loaded.
Note, the PR has 3 base commits that look funny but this was done for
the purpose of tracking the line history of moving the
`<DIALECT>_ops_ext.py` class into `<DIALECT>.py` and updating (commit
labeled "fix").
This allows some basic variadic operands in rewrites. There were some workarounds employed (like "aliasing" the attribute). Couldn't find a way to do this directly with properties.
This commit adds the initial version of the mlir-query tool, which leverages the pre-existing matchers defined in mlir/include/mlir/IR/Matchers.h
The tool provides the following set of basic queries:
hasOpAttrName(string)
hasOpName(string)
isConstantOp()
isNegInfFloat()
isNegZeroFloat()
isNonZero()
isOne()
isOneFloat()
isPosInfFloat()
isPosZeroFloat()
isZero()
isZeroFloat()
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D155127
This makes these match the behaviour of optional attributes (which are
omitted when they are their default value of none). This allows for
concise assembly formats without a custom printer.
An extra print of " " is also removed, this does change any existing
uses of oilists, but if the parameter before the oilist is optional,
that would previously add an extra space.
This #68694 + some fixes for the MLIR Python tests, unfortunately GitHub
does not allow re-opening PRs 😕
When defining a multi-line string in tblgen, the output in the Markdown
file currently contains too much whitespace and newlines for Hugo's
Markdown parser. For example, for `arith.addui_extended` the tblgen
```tblgen
let summary = [{
extended unsigned integer addition operation returning sum and overflow bit
}];
```
is currently converted to
```markdown
_
extended unsigned integer addition operation returning sum and overflow bit
_
```
which causes the text to not be italicized (as can be seen at
https://mlir.llvm.org/docs/Dialects/ArithOps/#arithaddui_extended-arithadduiextendedop).
After this PR, the output becomes
```
_Extended unsigned integer addition operation returning sum and overflow bit_
```
This makes these match the behaviour of optional attributes (which are
omitted when they are their default value of `none`). This allows for
concise assembly formats without a custom printer.
An extra print of " " is also removed, this does change any existing
uses of oilists, but if the parameter before the oilist is optional,
that would previously add an extra space.
Follow up to 7d4cd47e242c28c450c1e2a1a9f4bd4b7b5a01ab, where I fixed
just the case of a dash. This fixes it for all possible types of
strings, which can include "-,." etc.
This modifies some code written in
27c6d55cae74125b6381a647533090a72930ecda
It also handles the case of a leading number which is not valid for
python names.
Ref:
- https://llvm.org/docs/TableGen/ProgRef.html#literals
This is set to off by default but enabled for the MLIR project, which
uses Hugo Docs. Downstream projects can choose if they want to set this
option or not.
Also fix an incorrectly closed `<table>` tag.
This PR adds the additional generation of what I'm calling "value
builders" (a term I'm not married to) that look like this:
```python
def empty(sizes, element_type, *, loc=None, ip=None):
return get_result_or_results(tensor.EmptyOp(sizes=sizes, element_type=element_type, loc=loc, ip=ip))
```
which instantiates a `tensor.EmptyOp` and then immediately grabs the
result (`OpResult`) and then returns that *instead of a handle to the
op*.
What's the point of adding these when `EmptyOp.result` already exists?
My claim/feeling/intuition is that eDSL users are more comfortable with
a value centric programming model (i.e., passing values as operands) as
opposed to an operator instantiation programming model. Thus this change
enables (or at least goes towards) the bindings supporting such a user
and use case. For example,
```python
i32 = IntegerType.get_signless(32)
...
ten1 = tensor.empty((10, 10), i32)
ten2 = tensor.empty((10, 10), i32)
ten3 = arith.addi(ten1, ten2)
```
Note, in order to present a "pythonic" API and enable "pythonic" eDSLs,
the generated identifiers (op names and operand names) are snake case
instead of camel case and thus `llvm::convertToSnakeFromCamelCase`
needed a small fix. Thus this PR is stacked on top of
https://github.com/llvm/llvm-project/pull/68375.
In addition, as a kind of victory lap, this PR adds a "rangefor" that
looks and acts exactly like python's `range` but emits `scf.for`.
