45 Commits

Author SHA1 Message Date
Matthias Springer
f18c3e4e73
[mlir][Transforms] Dialect Conversion: Simplify materialization fn result type (#113031)
This commit simplifies the result type of materialization functions.

Previously: `std::optional<Value>`
Now: `Value`

The previous implementation allowed 3 possible return values:
- Non-null value: The materialization function produced a valid
materialization.
- `std::nullopt`: The materialization function failed, but another
materialization can be attempted.
- `Value()`: The materialization failed and so should the dialect
conversion. (Previously: Dialect conversion can roll back.)

This commit removes the last variant. It is not particularly useful
because the dialect conversion will fail anyway if all other
materialization functions produced `std::nullopt`.

Furthermore, in contrast to type conversions, at least one
materialization callback is expected to succeed. In case of a failing
type conversion, the current dialect conversion can roll back and try a
different pattern. This also used to be the case for materializations,
but that functionality was removed with #107109: failed materializations
can no longer trigger a rollback. (They can just make the entire dialect
conversion fail without rollback.) With this in mind, it is even less
useful to have an additional error state for materialization functions.

This commit is in preparation of merging the 1:1 and 1:N type
converters. Target materializations will have to return multiple values
instead of a single one. With this commit, we can keep the API simple:
`SmallVector<Value>` instead of `std::optional<SmallVector<Value>>`.

Note for LLVM integration: All 1:1 materializations should return
`Value` instead of `std::optional<Value>`. Instead of `std::nullopt`
return `Value()`.
2024-10-23 07:29:17 -07:00
Peiming Liu
f607102a0d
[mlir][sparse] partially support lowering sparse coiteration loops to scf.while/for. (#105565) 2024-08-23 10:47:44 -07:00
Peiming Liu
c44202574f
[mlir][sparse] support sparsification to coiterate operations. (#102546) 2024-08-20 11:13:38 -07:00
Peiming Liu
951a363097
[mlir][sparse] implement sparse_tensor.extract_value operation. (#101220) 2024-07-31 14:47:29 -07:00
Peiming Liu
a02010b3e9
[mlir][sparse] support sparsifying sparse kernels to sparse-iterator-based loop (#95858) 2024-06-17 16:50:12 -07:00
Peiming Liu
d6cc35f7f6
Reapply "[mlir][sparse] implement lowering rules for IterateOp." (#95836) 2024-06-17 13:29:53 -07:00
Shivam Gupta
bba5951b6f
[MLIR] Fix an assert that contains a mistake in conditional operator (#95668)
This is described in (N2) https://pvs-studio.com/en/blog/posts/cpp/1126/
so caught by the PVS Studio analyzer.

Warning message -
V502 Perhaps the '?:' operator works in a different way than it was
expected. The '?:' operator has a lower priority than the '+' operator.
LoopEmitter.cpp 983
V502 Perhaps the '?:' operator works in a different way than it was
expected. The '?:' operator has a lower priority than the '+' operator.
LoopEmitter.cpp 1039

The assert should be
assert(bArgs.size() == reduc.size() + (needsUniv ? 1 : 0));
since + has higher precedence and ? has lower.

This further can be reduce to
assert(aArgs.size() == reduc.size() + needsUniv);
because needUniv is a bool value which is implicitly converted to 0 or
2024-06-18 01:09:30 +05:30
Peiming Liu
996905d815
Revert "[mlir][sparse] implement lowering rules for IterateOp." (#95826)
Reverts llvm/llvm-project#95286
2024-06-17 11:35:23 -07:00
Peiming Liu
3a2e442893
[mlir][sparse] implement lowering rules for IterateOp. (#95286) 2024-06-17 10:45:50 -07:00
Peiming Liu
c42bbda425
[mlir][sparse] implement lowering rules for ExtractIterSpaceOp. (#89143)
**DO NOT MERGE** until https://github.com/llvm/llvm-project/pull/89003
2024-06-12 10:49:12 -07:00
Kazu Hirata
197c3a3efc
Use llvm::less_first (NFC) (#94136) 2024-06-02 07:45:50 -07:00
Yinying Li
e71eacc5b1
[mlir][sparse] Support explicit/implicit value for complex type (#90771) 2024-05-02 12:28:34 -04:00
Peiming Liu
78885395c8
[mlir][sparse] support tensor.pad on CSR tensors (#90687) 2024-05-01 15:37:38 -07:00
Peiming Liu
dbe376651a
[mlir][sparse] handle padding on sparse levels. (#90527) 2024-04-30 09:53:44 -07:00
Aart Bik
65ee8f10b2
[mlir][sparse] fold explicit value during sparsification (#90530)
This ensures the explicit value is generated (and not a load into the
values array). Note that actually not storing values array at all is
still TBD, this is just the very first step.
2024-04-29 18:06:07 -07:00
Peiming Liu
7e2eeb5753
[mlir][sparse] use ValueRange instead of std::pair for iterator position. (#90243)
`ValueRange` is more easy to be extended (e.g., for padded iterator).
2024-04-29 10:47:07 -07:00
Troy Butler
468fecfc39
Fix mismatches between function parameter definitions and declarations (#89512)
Addresses issue #88716.

Some function parameter names in the affected header files did not match
the parameter names in the definitions, or were listed in a different
order.

---------

Signed-off-by: Troy-Butler <squintik@outlook.com>
2024-04-26 13:00:31 +02:00
Christian Sigg
a5757c5b65
Switch member calls to isa/dyn_cast/cast/... to free function calls. (#89356)
This change cleans up call sites. Next step is to mark the member
functions deprecated.

See https://mlir.llvm.org/deprecation and
https://discourse.llvm.org/t/preferred-casting-style-going-forward.
2024-04-19 15:58:27 +02:00
Peiming Liu
a454d92c5a
[mlir][sparse] rename files and unifies APIs (#88162) 2024-04-09 10:59:15 -07:00
Peiming Liu
52b69aa32f
[mlir][sparse] support sparsifying batch levels (#83898) 2024-03-04 14:39:06 -08:00
Peiming Liu
1a0986f0f7
[mlir][sparse] code cleanup (using inferred type to construct to_[buf… (#83361)
…fer] op).
2024-02-28 16:55:28 -08:00
Peiming Liu
56d58295dd
[mlir][sparse] Introduce batch level format. (#83082) 2024-02-26 16:08:28 -08:00
Peiming Liu
5248a98724
[mlir][sparse] support SoA COO in codegen path. (#82439)
*NOTE*: the `SoA` property only makes a difference on codegen path, and
is ignored in libgen path at the moment (only SoA COO is supported).
2024-02-20 17:06:21 -08:00
Peiming Liu
aaf916456a
Reapply "[mlir][sparse] remove LevelType enum, construct LevelType from LevelFormat and Properties" (#81923) (#81934) 2024-02-15 14:48:52 -08:00
Mehdi Amini
513448d28e
Revert "[mlir][sparse] remove LevelType enum, construct LevelType from LevelF…" (#81923)
Reverts llvm/llvm-project#81799 ; this broke the mlir gcc7 bot.
2024-02-15 13:26:44 -08:00
Peiming Liu
235ec0f791
[mlir][sparse] remove LevelType enum, construct LevelType from LevelF… (#81799)
…ormat and properties instead.
2024-02-15 12:31:03 -08:00
Yinying Li
e5924d6499
[mlir][sparse] Implement parsing n out of m (#79935)
1. Add parsing methods for block[n, m].
2. Encode n and m with the newly extended 64-bit LevelType enum.
3. Update 2:4 methods names/comments to n:m.
2024-02-08 14:38:42 -05:00
Peiming Liu
35fae044c5
[mlir][sparse] using non-static field to avoid data races. (#81165) 2024-02-08 10:12:24 -08:00
Yinying Li
cd481fa827
[mlir][sparse] Change LevelType enum to 64 bit (#80501)
1. C++ enum is set through enum class LevelType : uint_64.
2. C enum is set through typedef uint_64 level_type. It is due to the
limitations in Windows build: setting enum width to ui64 is not
supported in C.
2024-02-05 17:00:52 -05:00
Peiming Liu
1ac6846263
[mlir][sparse] support sparse dilated convolution. (#80470) 2024-02-02 11:54:50 -08:00
Peiming Liu
4a653b4df5
[mlir][sparse] Support pretty print to debug sparse iteration. (#80207) 2024-02-01 15:28:36 -08:00
Peiming Liu
1d3300d502
[mlir][sparse] use shared value storage between wrapped iterator and the wrapper. (#80046) 2024-01-30 12:01:19 -08:00
Peiming Liu
de5e4d7c69
[mlir][sparse] fix error when convolution stride is applied on a dens… (#79521)
…e level.
2024-01-25 17:11:24 -08:00
Peiming Liu
982c815aad
[mlir][sparse] fix mismatch between enter/exitWhileLoop (#79493) 2024-01-25 12:21:47 -08:00
Peiming Liu
260e45cff0
[mlir][sparse] fix stack UAF (#79353) 2024-01-24 12:12:55 -08:00
Peiming Liu
298412b578
[mlir][sparse] setup SparseIterator to help generating code to traverse a sparse tensor level. (#78345) 2024-01-24 11:33:06 -08:00
Matthias Springer
5fcf907b34
[mlir][IR] Rename "update root" to "modify op" in rewriter API (#78260)
This commit renames 4 pattern rewriter API functions:
* `updateRootInPlace` -> `modifyOpInPlace`
* `startRootUpdate` -> `startOpModification`
* `finalizeRootUpdate` -> `finalizeOpModification`
* `cancelRootUpdate` -> `cancelOpModification`

The term "root" is a misnomer. The root is the op that a rewrite pattern
matches against
(https://mlir.llvm.org/docs/PatternRewriter/#root-operation-name-optional).
A rewriter must be notified of all in-place op modifications, not just
in-place modifications of the root
(https://mlir.llvm.org/docs/PatternRewriter/#pattern-rewriter). The old
function names were confusing and have contributed to various broken
rewrite patterns.

Note: The new function names use the term "modify" instead of "update"
for consistency with the `RewriterBase::Listener` terminology
(`notifyOperationModified`).
2024-01-17 11:08:59 +01:00
Peiming Liu
cf4dd91165
[mlir][sparse] initialize slice-driven loop-related fields in one place (#76099) 2023-12-20 14:20:57 -08:00
Matthias Springer
10056c821a
[mlir][SCF] scf.parallel: Make reductions part of the terminator (#75314)
This commit makes reductions part of the terminator. Instead of
`scf.yield`, `scf.reduce` now terminates the body of `scf.parallel` ops.
`scf.reduce` may contain an arbitrary number of reductions, with one
region per reduction.

Example:
```mlir
%init = arith.constant 0.0 : f32
%r:2 = scf.parallel (%iv) = (%lb) to (%ub) step (%step) init (%init, %init)
    -> f32, f32 {
  %elem_to_reduce1 = load %buffer1[%iv] : memref<100xf32>
  %elem_to_reduce2 = load %buffer2[%iv] : memref<100xf32>
  scf.reduce(%elem_to_reduce1, %elem_to_reduce2 : f32, f32) {
    ^bb0(%lhs : f32, %rhs: f32):
      %res = arith.addf %lhs, %rhs : f32
      scf.reduce.return %res : f32
  }, {
    ^bb0(%lhs : f32, %rhs: f32):
      %res = arith.mulf %lhs, %rhs : f32
      scf.reduce.return %res : f32
  }
}
```

`scf.reduce` operations can no longer be interleaved with other ops in
the body of `scf.parallel`. This simplifies the op and makes it possible
to assign the `RecursiveMemoryEffects` trait to `scf.reduce`. (This was
not possible before because the op was not a terminator, causing the op
to be DCE'd.)
2023-12-20 11:06:27 +09:00
Peiming Liu
6c06bde7c4
[mlir][sparse] support loop range query using SparseTensorLevel. (#75670) 2023-12-15 16:33:31 -08:00
Peiming Liu
21edad7d07
[mlir][sparse] set up the skeleton for SparseTensorLevel abstraction. (#75645)
Note that at the current moment, the newly-introduced
`SparseTensorLevel` classes are far from complete, we plan to migrate
code generation related to accessing sparse tensor levels to these
classes in the near future to simplify `LoopEmitter`.
2023-12-15 13:34:34 -08:00
Peiming Liu
4a72a4ef12
[NFC][mlir][sparse] remove redundant parameter. (#75551) 2023-12-15 09:29:22 -08:00
Aart Bik
15c06bc4af
[mlir][sparse] comment cleanup in iteration graph sorter (#75508) 2023-12-14 10:56:28 -08:00
Aart Bik
e52c941921
[mlir][sparse] minor cleanup of transform/utils (#75396)
Consistent include macro naming
Modified and added comments
2023-12-13 15:18:35 -08:00
Aart Bik
365777ecbe
[mlir][sparse] refactor utilities into transform/utils dir (#75250)
Separates actual transformation files from supporting utility files in
the transforms directory. Includes a bazel overlay fix for the build (as
well as a bit of cleanup of that file to be less verbose and more
flexible).
2023-12-12 15:34:31 -08:00