35 Commits

Author SHA1 Message Date
Longsheng Mou
f047b735e9
[mlir][NFC] Use getDefiningOp<OpTy>() instead of dyn_cast<OpTy>(getDefiningOp()) (#150428)
This PR uses `val.getDefiningOp<OpTy>()` to replace `dyn_cast<OpTy>(val.getDefiningOp())` , `dyn_cast_or_null<OpTy>(val.getDefiningOp())` and `dyn_cast_if_present<OpTy>(val.getDefiningOp())`.
2025-07-25 10:35:51 +08:00
Maksim Levental
972ac59c9a
[mlir][NFC] update mlir/Dialect create APIs (21/n) (#149928)
See https://github.com/llvm/llvm-project/pull/147168 for more info.
2025-07-22 08:15:27 -04:00
Kazu Hirata
5262865aac
[mlir] Construct SmallVector with ArrayRef (NFC) (#101896) 2024-08-04 11:43:05 -07:00
Aart Bik
365777ecbe
[mlir][sparse] refactor utilities into transform/utils dir (#75250)
Separates actual transformation files from supporting utility files in
the transforms directory. Includes a bazel overlay fix for the build (as
well as a bit of cleanup of that file to be less verbose and more
flexible).
2023-12-12 15:34:31 -08:00
Matthias Springer
851f85fffb
[mlir][SparseTensor] Fix insertion point in createQuickSort (#74549)
`createQuickSort` used to generate invalid IR:
```
"func.func"() <{function_type = (index, index, memref<?xindex>, memref<?xf32>, memref<?xi32>) -> (), sym_name = "_sparse_qsort_0_1_index_coo_1_f32_i32", sym_visibility = "private"}> ({
^bb0(%arg0: index, %arg1: index, %arg2: memref<?xindex>, %arg3: memref<?xf32>, %arg4: memref<?xi32>):
  %0:2 = "scf.while"(%arg0, %arg1) ({
  ^bb0(%arg5: index, %arg6: index):
    // ...
    "scf.condition"(%3, %arg5, %arg6) : (i1, index, index) -> ()
  }, {
  ^bb0(%arg5: index, %arg6: index):
    // ...
    %7:2 = "scf.if"(%6) ({
      %8 = "arith.cmpi"(%2, %3) <{predicate = 7 : i64}> : (index, index) -> i1
      // ...
      "scf.yield"(%9#0, %9#1) : (index, index) -> ()
      %10 = "arith.constant"() <{value = 0 : index}> : () -> index
    }, {
      "scf.yield"(%arg5, %arg5) : (index, index) -> ()
    }) : (i1) -> (index, index)
    "scf.yield"(%7#0, %7#1) : (index, index) -> ()
  }) : (index, index) -> (index, index)
  "func.return"() : () -> ()
}) : () -> ()
within split at mlir/test/Dialect/SparseTensor/buffer_rewriting.mlir:76 offset :11:1: error: 'scf.yield' op must be the last operation in the parent block
```

This commit fixes tests such as
`mlir/test/Dialect/SparseTensor/buffer_rewriting.mlir` when verifying
the IR after each pattern application (#74270).
2023-12-07 08:47:05 +09:00
long.chen
1609f1c2a5
[mlir][affine][nfc] cleanup deprecated T.cast style functions (#71269)
detail see the docment: https://mlir.llvm.org/deprecation/

Not all changes are made manually, most of them are made through a clang
tool I wrote https://github.com/lipracer/cpp-refactor.
2023-11-14 13:01:19 +08:00
Peiming Liu
0083f8338c
[mlir][sparse] renaming sparse_tensor.sort_coo to sparse_tensor.sort (#68161)
Rationale: the operation does not always sort COO tensors (also used for
sparse_tensor.compress for example).
2023-10-03 16:28:25 -07:00
Peiming Liu
bfa3bc4378
[mlir][sparse] unifies sparse_tensor.sort_coo/sort into one operation. (#66722)
The use cases of the two operations are largely overlapped, let's
simplify it and only use one of them.
2023-09-19 17:02:32 -07:00
Peiming Liu
4176ce61f1
[mlir][sparse] fix logical error when generating sort_coo. (#66690)
To fix issue: https://github.com/llvm/llvm-project/issues/66664
2023-09-18 15:26:01 -07:00
bixia1
abb05014f9 [mlir][sparse] Modify the pivot selection method for quick sort.
Previously, we choose the median of three values. We now choose the median of
five values when the number of values being sorted exceed a threshold
(currently 100). This is similar to std::sort.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D145534
2023-03-15 13:53:00 -07:00
bixia1
2ef416273f [mlir][sparse] Improve sort operation by generating inlined code to compare values.
Previously, we generate function calls to compare values for sorting. It turns
out that the compiler doesn't inline those function calls. We now directly
generate inlined code. Also, modify the code for comparing values to use less
number of branches.

This improves all sort implementation in general. For arabic-2005.mtx CSR, the
improvement is around 25%.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D145442
2023-03-14 15:14:49 -07:00
bixia1
f6424d11cb [mlir][sparse] Improve quick sort by using a loop to sort the bigger partition.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D145440
2023-03-10 20:43:08 -08:00
Kohei Yamaguchi
9a29d87538 [mlir][sparse] Add checking parent op of SortOp
Fix crash with segmentation fault caused by setting a parent operator
that is not func::FuncOp with sparse_tensor SortOp.

fixes https://github.com/llvm/llvm-project/issues/59988

Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D143874
2023-02-27 16:37:02 +01:00
bixia1
a150766880 [mlir][sparse] Implement hybrid quick sort for sparse_tensor.sort.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143227
2023-02-08 14:06:31 -08:00
bixia1
3b1c86cd0f [mlir][sparse] Implement heap sort for sparse_tensor.sort.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142913
2023-02-02 15:36:38 -08:00
bixia1
8550aebd57 [mlir][sparse] Extend sorting function generator to support operand beyond (lo, hi, xs, ys).
This is to prepare for implementing a hybrid quick sort, which switches to heap
sort when the recursive depth exceeds certain limits.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142731
2023-01-31 10:22:28 -08:00
bixia1
0c7f1c1520 [mlir][sparse] Extend sparse_tensor.sort with a enum attribute to specify a sorting implementation.
Currently, all the non-stable sorting algorithms are implemented via the
straightforward quick sort. This will be fixed in the following PR.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142678
2023-01-29 18:34:08 -08:00
bixia1
7cec4d169d [mlir][sparse] Change the quick sort pivot selection.
Previously, we choose the value at (lo + hi)/2 as a pivot for partitioning the
data in [lo, hi). We now choose the median for the three values at lo, (lo +
hi)/2, and (hi-1) as a pivot to match the std::qsort implementation.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142679
2023-01-29 17:56:57 -08:00
wren romano
9916ab03f1 [mlir][sparse] (re)introducing getRankedTensorType/getMemrefType
The bulk of D142074 seems to have gotten overwritten due to some sort of merge conflict (afaict there's no record of it having been reverted intentionally).  So this commit redoes those changes.  In addition to the original changes, this commit also:
* moves the definition of `getRankedTensorType` (from `Transforms/CodegenUtils.h` to `IR/SparseTensor.h`), so that it can be used by `IR/SparseTensorDialect.cpp`.
* adds `getMemRefType` as another abbreviation.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142503
2023-01-25 11:29:54 -08:00
Peiming Liu
988733c600 [mlir][sparse] use sparse_tensor::StorageSpecifier to store dim/memSizes
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140130
2022-12-23 00:47:36 +00:00
bixia1
ea4be70cea [mlir][sparse] Fix problems in creating complex zero for initialization.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139591
2022-12-08 07:49:27 -08:00
Aliia Khasanova
399638f98c Merge kDynamicSize and kDynamicSentinel into one constant.
resolve conflicts

Differential Revision: https://reviews.llvm.org/D138282
2022-11-21 13:01:26 +00:00
Aart Bik
c4f1af2bdf [mlir][sparse] avoid single small vector, set exact number 3
Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D138071
2022-11-15 21:04:43 -08:00
Aart Bik
0e1708ff64 [mlir][sparse] cleanup small vector constant hints
Following advise from

https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallvector-h

This revision removes the size hints from SmallVector (unless we are
certain of the resulting number of elements). Also, this replaces
SmallVector references with SmallVectorImpl references.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138063
2022-11-15 15:09:06 -08:00
bixia1
4f729d5a70 [mlir][sparse] Add rewriting rules for sparse_tensor.sort_coo.
Refactor the rewriting of sparse_tensor.sort to support the implementation of
sparse_tensor.sort_coo.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137522
2022-11-14 08:48:53 -08:00
bixia1
5618d2bea9 [mlir][sparse] Add option enable-buffer-initialization to initialize the memory buffers for sparse tensors to support debugging.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137592
2022-11-08 09:54:33 -08:00
bixia1
9b800bf79d [mlir][sparse] Improve the non-stable sort implementation.
Replace the quick sort partition method with one that is more similar to the
method used by C++ std quick sort. This improves the runtime for sorting
sk_2005.mtx by more than 10x.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137290
2022-11-07 07:38:42 -08:00
bixia1
d45be88736 [mlir][sparse] Implement the rewrite for sparse_tensor.push_back a value n times.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136654
2022-10-31 08:19:12 -07:00
Mehdi Amini
9b170d126f Apply clang-tidy fixes for readability-identifier-naming in SparseBufferRewriting.cpp (NFC) 2022-10-13 02:58:34 +00:00
Mehdi Amini
f628d6c3e8 Apply clang-tidy fixes for modernize-use-using in SparseBufferRewriting.cpp (NFC) 2022-10-12 16:22:43 +00:00
bixia1
9409bbb2e0 [mlir][sparse] Implement insertion sort for the stable sort operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135182
2022-10-06 09:48:39 -07:00
bixia1
1c835b5a8e [mlir][sparse] Allow the push_back operator to skip capacity check and reallocation.
Add UnitAttr `inbounds` for this purpose.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134913
2022-09-29 16:38:06 -07:00
bixia1
654bbbde55 [mlir][sparse] Move the implementation of sparse_tensor.push_back to the buffer rewriter.
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D134777
2022-09-29 15:06:00 -07:00
Aart Bik
bfbf3bcb37 [mlir][sparse] fix build breakage due to Arith name change
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D134899
2022-09-29 12:12:43 -07:00
bixia1
062e515b70 [mlir][sparse] Add rewrite rule for the sort operator.
Add sparse-buffer-rewrite pass to rewrite sparse primitives on buffers to MLIR
implementation.

Add sparse rewrite rule for the sort operator.

Add FileCheck test and integration test.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134627
2022-09-29 11:38:19 -07:00