228 Commits

Author SHA1 Message Date
Kazu Hirata
1cf1c21b84
[mlir] Strip away lambdas (NFC) (#143280)
We don't need lambdas here.
2025-06-08 01:34:17 -07:00
Kazu Hirata
03697cf8fc
[mlir] Use llvm::find_if (NFC) (#139912) 2025-05-14 12:48:48 -07:00
Hongren Zheng
3db668e0bb
[mlir][SparseTensor][NFC] Migrate to OpAsmAttrInterface for ASM alias generation (#130483)
After the introduction of `OpAsmAttrInterface`, it is favorable to
migrate code using `OpAsmDialectInterface` for ASM alias generation,
which lives in `Dialect.cpp`, to use `OpAsmAttrInterface`, which lives
in `Attrs.td`. In this way, attribute behavior is placed near its
tablegen definition and people won't need to go through other files to
know what other (unexpected) hooks comes into play.
2025-03-11 00:49:21 +08:00
Kazu Hirata
af6e1881e0
[mlir] Avoid repeated map lookups (NFC) (#113122) 2024-10-21 06:52:24 -07:00
Kazu Hirata
2077fb80ff
[mlir] Avoid repeated map lookups (NFC) (#113074) 2024-10-20 10:42:28 -07:00
Matthias Springer
77f8297c6f
[mlir][sparse] Improve sparse tensor type constraints (#112133)
Sparse tensors are always ranked tensors. Encodings cannot be attached
to unranked tensors. Change the type constraint to `RankedTensorOf`, so
that we generate `TypedValue<RankedTensorType>` instead of
`TypedValue<TensorType>`. This removes the need for type casting in some
cases.

Also improve the verifiers (missing `return` statements) and switch a
few other `AnyTensor` to `AnyRankedTensor`.

This commit is in preparation of a dialect conversion commit that
required fixes in the sparse dialect.
2024-10-13 21:12:38 +02:00
Peiming Liu
b48ef8d8d4
[mlir][sparse] unify block arguments order between iterate/coiterate operations. (#105567) 2024-08-23 14:49:00 -07:00
Peiming Liu
f607102a0d
[mlir][sparse] partially support lowering sparse coiteration loops to scf.while/for. (#105565) 2024-08-23 10:47:44 -07:00
Peiming Liu
c44202574f
[mlir][sparse] support sparsification to coiterate operations. (#102546) 2024-08-20 11:13:38 -07:00
Matthias Springer
7359a6b799
[mlir][ODS] Verify type constraints in Types and Attributes (#102326)
When a type/attribute is defined in TableGen, a type constraint can be
used for parameters, but the type constraint verification was missing.

Example:
```
def TestTypeVerification : Test_Type<"TestTypeVerification"> {
  let parameters = (ins AnyTypeOf<[I16, I32]>:$param);
  // ...
}
```

No verification code was generated to ensure that `$param` is I16 or
I32.

When type constraints a present, a new method will generated for types
and attributes: `verifyInvariantsImpl`. (The naming is similar to op
verifiers.) The user-provided verifier is called `verify` (no change).
There is now a new entry point to type/attribute verification:
`verifyInvariants`. This function calls both `verifyInvariantsImpl` and
`verify`. If neither of those two verifications are present, the
`verifyInvariants` function is not generated.

When a type/attribute is not defined in TableGen, but a verifier is
needed, users can implement the `verifyInvariants` function. (This
function was previously called `verify`.)

Note for LLVM integration: If you have an attribute/type that is not
defined in TableGen (i.e., just C++), you have to rename the
verification function from `verify` to `verifyInvariants`. (Most
attributes/types have no verification, in which case there is nothing to
do.)

Depends on #102657.
2024-08-09 22:04:40 +02:00
Peiming Liu
785a24f156
[mlir][sparse] introduce sparse_tensor.coiterate operation. (#101100)
This PR introduces `sparse_tensor.coiterate` operation, which represents
a loop that traverses multiple sparse iteration space.
2024-07-31 15:14:27 -07:00
Peiming Liu
12189f8005
[mlir][sparse] introduce sparse_tensor.extract_value operation. (#101219) 2024-07-30 16:26:18 -07:00
Peiming Liu
a02010b3e9
[mlir][sparse] support sparsifying sparse kernels to sparse-iterator-based loop (#95858) 2024-06-17 16:50:12 -07:00
Peiming Liu
a43d79af78
[mlir][sparse] add canonicalization patterns for IterateOp. (#95569) 2024-06-14 10:31:29 -07:00
Peiming Liu
e276cf0831
[mlir][sparse] introduce sparse_tensor.iterate operation (#88955)
A `sparse_tensor.iterate` iterates over a sparse iteration space
extracted from `sparse_tensor.extract_iteration_space` operation
introduced in https://github.com/llvm/llvm-project/pull/88554.
2024-06-10 10:20:24 -07:00
Peiming Liu
13af97a70e
[mlir][sparse] allow multiple COO segments in sparse encodings. (#91786)
**NOTE**: we still have implementation holes when handling multiple COO
segments in the encoding. But the format should be considered to be
legal.
2024-05-10 11:36:01 -07:00
Yinying Li
83f3b1cb48
[mlir][sparse] Add verification for explicit/implicit value (#90111)
1. Verify that the type of explicit/implicit values should be the same
as the tensor element type.
2. Verify that implicit value could only be zero.
3. Verify that explicit/implicit values should be numeric.
4. Fix the type change issue caused by SparseTensorType(enc).
2024-05-07 20:28:39 -04:00
Yinying Li
e71eacc5b1
[mlir][sparse] Support explicit/implicit value for complex type (#90771) 2024-05-02 12:28:34 -04:00
Yinying Li
a10d67f9fb
[mlir][sparse] Enable explicit and implicit value in sparse encoding (#88975)
1. Explicit value means the non-zero value in a sparse tensor. If
explicitVal is set, then all the non-zero values in the tensor have the
same explicit value. The default value Attribute() indicates that it is
not set.

2. Implicit value means the "zero" value in a sparse tensor. If
implicitVal is set, then the "zero" value in the tensor is equal to the
implicit value. For now, we only support `0` as the implicit value but
it could be extended in the future. The default value Attribute()
indicates that the implicit value is `0` (same type as the tensor
element type).

Example:

```
#CSR = #sparse_tensor.encoding<{
  map = (d0, d1) -> (d0 : dense, d1 : compressed),
  posWidth = 64,
  crdWidth = 64,
  explicitVal = 1 : i64,
  implicitVal = 0 : i64
}>
```

Note: this PR tests that implicitVal could be set to other values as
well. The following PR will add verifier and reject any value that's not
zero for implicitVal.
2024-04-24 16:20:25 -07:00
Christian Sigg
a5757c5b65
Switch member calls to isa/dyn_cast/cast/... to free function calls. (#89356)
This change cleans up call sites. Next step is to mark the member
functions deprecated.

See https://mlir.llvm.org/deprecation and
https://discourse.llvm.org/t/preferred-casting-style-going-forward.
2024-04-19 15:58:27 +02:00
Peiming Liu
481bd5d416
[mlir][sparse] introduce sparse_tensor.extract_iteration_space operation. (#88554)
A `sparse_tensor.extract_space %tensor at %iterator` extracts a *sparse*
iteration space defined `%tensor`, the operation to traverse the
iteration space will be introduced in following PRs.
2024-04-16 11:32:30 -07:00
Peiming Liu
b9556532c7
Revert "[mlir][sparse] introduce sparse_tensor.iterate operation" (#88953)
Reverts llvm/llvm-project#88807 (merged by mistake)
2024-04-16 11:31:33 -07:00
Peiming Liu
8debcf03c5
[mlir][sparse] introduce sparse_tensor.iterate operation (#88807)
A `sparse_tensor.iterate` iterates over a sparse iteration space
extracted from `sparse_tensor.extract_iteration_space` operation
introduced in https://github.com/llvm/llvm-project/pull/88554.

*DO NOT MERGE* before https://github.com/llvm/llvm-project/pull/88554
2024-04-16 11:31:09 -07:00
Peiming Liu
62fa12ad24
[mlir][sparse] support querying sparse buffer types from sparse tenso… (#88308)
…r encodings.
2024-04-11 10:26:03 -07:00
Peiming Liu
a54930e696
[mlir][sparse] allow YieldOp to yield multiple values. (#87261) 2024-04-01 10:30:36 -07:00
Justin Fargnoli
513cdb8222
[mlir] Declare promised interfaces for all dialects (#78368)
This PR adds promised interface declarations for all interfaces declared
in `InitAllDialects.h`.

Promised interfaces allow a dialect to declare that it will have an
implementation of a particular interface, crashing the program if one
isn't provided when the interface is used.
2024-03-15 20:23:20 -07:00
Peiming Liu
94e27c265a
[mlir][sparse] reuse tensor.insert operation to insert elements into … (#84987)
…a sparse tensor.
2024-03-12 16:59:17 -07:00
Yinying Li
ced1fac8a3
[mlir][sparse] Move n:m printing into toMLIRString (#84264) 2024-03-07 16:13:45 -05:00
Peiming Liu
52b69aa32f
[mlir][sparse] support sparsifying batch levels (#83898) 2024-03-04 14:39:06 -08:00
Aart Bik
e10dc60ad7
[mlir][sparse][nfc] fixed typo in "translate" (#83891) 2024-03-04 12:57:48 -08:00
Peiming Liu
6bc7c9df7f
[mlir][sparse] infer returned type for sparse_tensor.to_[buffer] ops (#83343)
The sparse structure buffers might not always be memrefs with rank == 1
with the presence of batch levels.
2024-02-28 16:10:20 -08:00
Peiming Liu
0d1f95760b
[mlir][sparse] support type conversion from batched sparse tensors to… (#83163)
… memrefs.
2024-02-27 12:05:28 -08:00
Peiming Liu
56d58295dd
[mlir][sparse] Introduce batch level format. (#83082) 2024-02-26 16:08:28 -08:00
Peiming Liu
5248a98724
[mlir][sparse] support SoA COO in codegen path. (#82439)
*NOTE*: the `SoA` property only makes a difference on codegen path, and
is ignored in libgen path at the moment (only SoA COO is supported).
2024-02-20 17:06:21 -08:00
Peiming Liu
f740366fa6
[mlir][sparse] support type conversion from SoA COO to memrefs. (#82398) 2024-02-20 11:19:13 -08:00
Peiming Liu
088c7ce429
[mlir][sparse] introduce SoA level property on singleton level. (#81942) 2024-02-15 16:41:10 -08:00
Peiming Liu
aaf916456a
Reapply "[mlir][sparse] remove LevelType enum, construct LevelType from LevelFormat and Properties" (#81923) (#81934) 2024-02-15 14:48:52 -08:00
Mehdi Amini
513448d28e
Revert "[mlir][sparse] remove LevelType enum, construct LevelType from LevelF…" (#81923)
Reverts llvm/llvm-project#81799 ; this broke the mlir gcc7 bot.
2024-02-15 13:26:44 -08:00
Peiming Liu
235ec0f791
[mlir][sparse] remove LevelType enum, construct LevelType from LevelF… (#81799)
…ormat and properties instead.
2024-02-15 12:31:03 -08:00
Yinying Li
2a6b521b36
[mlir][sparse] Add more tests and verification for n:m (#81186)
1. Add python test for n out of m
2. Add more methods for python binding
3. Add verification for n:m and invalid encoding tests
4. Add e2e test for n:m

Previous PRs for n:m #80501 #79935
2024-02-09 14:34:36 -05:00
Yinying Li
e5924d6499
[mlir][sparse] Implement parsing n out of m (#79935)
1. Add parsing methods for block[n, m].
2. Encode n and m with the newly extended 64-bit LevelType enum.
3. Update 2:4 methods names/comments to n:m.
2024-02-08 14:38:42 -05:00
Peiming Liu
07bf1ddb4e
[mlir][sparse] support non-id map for [Dis]assembleOp (#80355) 2024-02-01 15:11:33 -08:00
Yinying Li
31b72b0742
[mlir][sparse]Make isBlockSparsity more robust (#75113)
1. A single dimension can either be blocked (with floordiv and mod pair)
or non-blocked. Mixing them would be invalid.
2. Block size should be non-zero value.
2023-12-12 13:43:03 -05:00
Aart Bik
3d3e46cc4d
[mlir][sparse] make test for block sparsity more robust (#74798)
For BSR and convolutions, we encounter

(d0, d1, d2, d3) -> ((d0 + d2) floordiv 2, (d1 + d3) floordiv 2, (d0 +
d2) mod 2, (d1 + d3) mod 2)

which crashed the current test. Note that an actual test and working
code is still to follow (since we need to fix a few other things first)
2023-12-08 11:50:10 -08:00
Aart Bik
5b72950394
[mlir][sparse] move all COO related methods into SparseTensorType (#73881)
This centralizes all COO methods, and provides a cleaner API. Note that
the "enc" only constructor is a temporary workaround the need for COO
methods inside the "enc" only storage specifier.
2023-11-30 09:40:39 -08:00
Aart Bik
98f8b1afb4
[mlir][sparse] remove COO test from trait and encoding (#73733)
This is a minor step towards moving ALL COO related tests into the
SparseTensorType class rather than
having it all over the place (with risk of becoming inconsistent). Next
revision will move ALL COO related methods into this class.
2023-11-28 17:46:02 -08:00
Aart Bik
45288085b5
[mlir][sparse] move toCOOType into SparseTensorType class (#73708)
Migrates dangling convenience method into proper SparseTensorType class.
Also cleans up some details (picking right dim2lvl/lvl2dim). Removes
more dead code.
2023-11-28 16:04:01 -08:00
Aart Bik
79cb594fdf
[mlir][sparse] remove unused COO method (#73595)
step closer towards moving all type related methods into encoding and/or
sparse tensor type class
2023-11-27 17:26:45 -08:00
Peiming Liu
4e2f1521ec
[mlir][sparse] code cleanup, remove FIXMEs (#73575) 2023-11-27 14:57:08 -08:00
Aart Bik
1944c4f76b
[mlir][sparse] rename DimLevelType to LevelType (#73561)
The "Dim" prefix is a legacy left-over that no longer makes sense, since
we have a very strict "Dimension" vs. "Level" definition for sparse
tensor types and their storage.
2023-11-27 14:27:52 -08:00