This commit adds extra assertions to `OperationFolder` and `OpBuilder`
to ensure that the types of the folded SSA values match with the result
types of the op. There used to be checks that discard the folded results
if the types do not match. This commit makes these checks stricter and
turns them into assertions.
Discarding folded results with the wrong type (without failing
explicitly) can hide bugs in op folders. Two such bugs became apparent
in MLIR (and some more in downstream projects) and are fixed with this
change.
Note: The existing type checks were introduced in
https://reviews.llvm.org/D95991.
Migration guide: If you see failing assertions (`folder produced value
of incorrect type`; make sure to run with assertions enabled!), run with
`-debug` or dump the operation right before the failing assertion. This
will point you to the op that has the broken folder. A common mistake is
a mismatch between static/dynamic dimensions (e.g., input has a static
dimension but folded result has a dynamic dimension).
-- In order to compute maximum, we should always initialise the
result with the largest negative value possible for the concerned
element type, instead of the smallest.
-- This commit essentially adds a fix to this issue.
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com>
This cherry-picks the changes in
llvm-project/5bf701a6687a46fd898621f5077959ff202d716b and extends the
pattern to handle vector types.
To reuse `getBoolAttribute` method, it moves the static method above the
include of generated file.
The logic for chained basic arithmetic operations in the `arith` dialect
was using `getInt()` on `IntegerAttr`. This is a problem for very large
integers. Specifically, in
https://github.com/llvm/llvm-project/issues/64774 the following
assertion failed:
```
Assertion failed: (getSignificantBits() <= 64 && "Too many bits for int64_t"), function getSExtValue, file APInt.h, line 1510.
```
According to a comment on `getInt()`, calls to `getInt()` should be
replaced by `getValue()`:
ab6a66dbec/mlir/include/mlir/IR/BuiltinAttributes.td (L707-L708)
This patch fixes https://github.com/llvm/llvm-project/issues/64774 by
doing such a replacement.
---------
Co-authored-by: Markus Böck <markus.boeck02@gmail.com>
This adds the following canonicalization patterns:
- Inverting condition:
- `select(not(pred), a, b) => select(pred, b, a)`
- Merging consecutive selects with the same predicate
- `select(pred, select(pred, a, b), c) => select(pred, a, c)`
- `select(pred, a, select(pred, b, c)) => select(pred, a, c)`
- Merging consecutive selects with a common value:
- `select(predA, select(predB, a, b), b) => select(and(predA, predB), a,
b)`
- `select(predA, select(predB, b, a), b) => select(and(predA,
not(predB)), a, b)`
- `select(predA, a, select(predB, a, b)) => select(or(predA, predB), a,
b)`
- `select(predA, a, select(predB, b, a)) => select(or(predA,
not(predB)), a, b)`
This patch is part of a larger initiative aimed at fixing floating-point
`max` and `min` operations in MLIR:
https://discourse.llvm.org/t/rfc-fix-floating-point-max-and-min-operations-in-mlir/72671.
Here we introduce new operations for floating-point numbers: `minnum`
and `maxnum`.
These operations have different semantics than `minumumf` and `maximumf`
ops.
They follow the eponymous LLVM intrinsics semantics, which differs
in the handling positive and negative zeros and NaNs.
This patch addresses the 1.3 task from the RFC.
This patch is part of a larger initiative aimed at fixing floating-point `max` and `min` operations in MLIR: https://discourse.llvm.org/t/rfc-fix-floating-point-max-and-min-operations-in-mlir/72671.
This commit addresses Task 1.2 of the mentioned RFC. By renaming these operations, we align their names with LLVM intrinsics that have corresponding semantics.
The primary motivation is to we have a simple mechanism to extract
values from attributes in folders and canon patterns without having to
re-fold constants or write nested conditions over attribute types.
Matching over attributes composes especially well with fold adaptors.
Update folds in Arith and SPIRV dialects to match over attributes, where
applicable.
Reviewed By: mehdi_amini, zero9178
Differential Revision: https://reviews.llvm.org/D159437
If either of the operands of `select` is fully poisoned we can simply return the other.
This PR implements this optimization inside the `fold` method.
Note that this patch is the first to add a dependency on the UB dialect within Arith. Given this was inevitable (and part of the motivation) it should be fine I believe.
Differential Revision: https://reviews.llvm.org/D158986
This reverts commit 7c349c369847dc2f1736efb9c90d03521cd44a90.
Per discussion at
https://reviews.llvm.org/rG7c349c369847dc2f1736efb9c90d03521cd44a90
and elsewhere, the lowering to LLVM defined here isn't what it should
be and the fastmath flag usage isn't correct, so `arith.is_nan` and
`arith.is_inf` cannot exist in their current form.
It's unclear if those operations should be introduced in the future,
since they make the dialect more complex and don't add any expressive
power. Further discussion may be moved to an RFC (or I'll drop this
patch).
Differential Revision: https://reviews.llvm.org/D157543
Add an option to the family of `getIdentity` helper functions so that it is
possible to produce fast-math friendly constants.
For instance, for maxf the identity value is `-inf`, however, if the related
operations are lowered with fast-math (`noinf` in particular), then the value
becomes `poison` and chances are the whole codegen is not going to do what we
want.
To avoid this problem, we add an option to `getIdentity` and friends that
specifies whether a finite value needs to be produced or not.
The patch is NFC for all the code but the lowering of `linalg::softmax`
because we know we lower that with fast-math down the line.
I didn't audit the rest of the code to check if it would make sense to set
this boolean in more places.
Note: It feels kind of wrong to have to know what the lowering may do, but
I don't know what the right (at least short-term) solution is. Long term,
we may want a special "neutral element" attribute for the respective ops. I
didn't think too much about the implications for that.
Differential Revision: https://reviews.llvm.org/D156471
Both LLVM and SPIR-V have some form of "is this float a NaN/Inf"
operation (though LLVM's uses the rather opaque "is.fpclass"
intrinsic), which is not exposed in MLIR.
This has lead to awkward workarounds in -arith-expands-ops where a NaN
test was performed by comparing an operation to itself. This commit
resolves that issue.
Reviewed By: dcaballe, kuhar
Differential Revision: https://reviews.llvm.org/D156169
The verifier for some arith ops were not considering that ranked
tensor types can have encodings.
Differential Revision: https://reviews.llvm.org/D156557
This consolidates where this kind of implementations lives and
refactor the code to have more code sharing.
NFC
Differential Revision: https://reviews.llvm.org/D154362
At the moment, only the trailing dimensions in the vector type can be
scalable, i.e. this is supported:
vector<2x[4]xf32>
and this is not allowed:
vector<[2]x4xf32>
This patch extends the vector type so that arbitrary dimensions can be
scalable. To this end, an array of bool values is added to every vector
type to denote whether the corresponding dimensions are scalable or not.
For example, for this vector:
vector<[2]x[3]x4xf32>
the following array would be created:
{true, true, false}.
Additionally, the current syntax:
vector<[2x3]x4xf32>
is replaced with:
vector<[2]x[3]x4xf32>
This is primarily to simplify parsing (this way, the parser can easily
process one dimension at a time rather than e.g. tracking whether
"scalable block" has been entered/left).
NOTE: The `isScalableDim` parameter of `VectorType` (introduced in this
patch) makes `numScalableDims` redundant. For the time being,
`numScalableDims` is preserved to facilitate the transition between the
two parameters. `numScalableDims` will be removed in one of the
subsequent patches.
This change is a part of a larger effort to enable scalable
vectorisation in Linalg. See this RFC for more context:
* https://discourse.llvm.org/t/rfc-scalable-vectorisation-in-linalg/
Differential Revision: https://reviews.llvm.org/D153372
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This patch updates all remaining uses of the deprecated functionality in
mlir/. This was done with clang-tidy as described below and further
modifications to GPUBase.td and OpenMPOpsInterfaces.td.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D151542
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
* https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…"
* Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This follows a previous patch that updated calls
`op.cast<T>()-> cast<T>(op)`. However some cases could not handle an
unprefixed `cast` call due to occurrences of variables named cast, or
occurring inside of class definitions which would resolve to the method.
All C++ files that did not work automatically with `cast<T>()` are
updated here to `llvm::cast` and similar with the intention that they
can be easily updated after the methods are removed through a
find-replace.
See https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
for the clang-tidy check that is used and then update printed
occurrences of the function to include `llvm::` before.
One can then run the following:
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-export-fixes /tmp/cast/casts.yaml mlir/*\
-header-filter=mlir/ -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D150348
Currently conversions to interfaces may happen implicitly (e.g.
`Attribute -> TypedAttr`), failing a runtime assert if the interface
isn't actually implemented. This change marks the `Interface(ValueT)`
constructor as explicit so that a cast is required.
Where it was straightforward to I adjusted code to not require casts,
otherwise I just made them explicit.
Depends on D148491, D148492
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D148493
This adds `arith::ConstantOp::materialize`, which builds a constant from
an attribute and type only if it would result in a valid op. This is
useful for dialect `materializeConstant` hooks, and allows for removing
the previous `Attribute, Type` builder which was only used during
materialization.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D148491
This patch is mainly to deal with folding trunci with ext，as flows：
trunci(zexti(a)) -> trunci(a)
trunci(zexti(a)) -> trunci(a)
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D140604
Handle the splat and dense case.
I saw this pattern show up in a couple recent SPIR-V-specific bug
report.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D147109
These propagate all the way down to SPIR-V and result in some fishy code
with large constants.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D145423
This enables us to use the common fold helpers on elementwise ops that
produce different result type than operand types, e.g., `arith.cmpi` or
`arith.addui_extended`.
Use the updated helper to teach `arith.cmpi` to fold constant vectors.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D143779
This is the dialect in-tree with the most `fold` method implementations by far. This patch simply changes all implementations to make use of the new signature.
Admittedly, the code readability does not get a lot better in this case, simply due to most methods making use of `constFoldBinaryOp`. I did not modify that function or its interface as part of this patch, but might be something to consider in the future.
Differential Revision: https://reviews.llvm.org/D141490
This revision folds arith.shrui, arith.shrsi and arith.shli with zero
rhs to lhs.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D140749
We shouldn't be checking things that are guaranteed by the op's verifier.
Reviewed By: benshi001
Differential Revision: https://reviews.llvm.org/D140610
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch changes the way mlir-tblgen generates .inc
files, and modifies tests and documentation appropriately. It is a "no
compromises" patch, and doesn't leave the user with an unpleasant mix of
llvm::Optional and std::optional.
A non-trivial change has been made to ControlFlowInterfaces to split one
constructor into two, relating to a build failure on Windows.
See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>
Differential Revision: https://reviews.llvm.org/D138934
- Add a fold for `mulsi_extended(x, 1)`
- Add folds to demote wide integer multiplication to `mul*i_extended` when the result is shifted
and truncated: `trunci(shrui(mul(*ext(x), *ext(y)), c)) -> mul*i_extended(x, y)`
Reviewed By: Mogball, jpienaar
Differential Revision: https://reviews.llvm.org/D139778
This reverts commit 834c17f618ce87b14446e42250d924b8d5f01abe.
Revert to properly address post-commit comments by @jpienaar
under https://reviews.llvm.org/D139778.
- Add a fold for `mulsi_extended(x, 1)`
- Add folds to demote wide integer multiplication to `mul*i_extended` when the result is shifted
and truncated: `trunci(shrui(mul(*ext(x), *ext(y)), c)) -> mul*i_extended(x, y)`
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D139778
Demote `arith.addui_extended` to `arith.addi` when the 'overflow'
result has no uses.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D139722
The goal is to make the naming of the future `_extended` ops more
consistent. With unsigned addition, the carry value/flag and overflow
bit are the same, but this is not true when it comes to signed addition.
Also rename the second result from `carry` to `overflow`.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D139569
