52 Commits

Author SHA1 Message Date
bixia1
f81f0cb75a [mlir][sparse] Split SparseTensorRewrite into PreSparsificationRewrite and PostSparsificationRewrite.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D138153
2022-11-17 07:13:55 -08:00
Peiming Liu
91e7b9e525 [mlir][sparse] annotate loops that are generated by loop emitter.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138155
2022-11-17 00:09:33 +00:00
Peiming Liu
61e5c14fa8 [mlir][sparse] fix memory leakage in concatenate rewriter.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138074
2022-11-16 00:10:33 +00:00
Aart Bik
0e1708ff64 [mlir][sparse] cleanup small vector constant hints
Following advise from

https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallvector-h

This revision removes the size hints from SmallVector (unless we are
certain of the resulting number of elements). Also, this replaces
SmallVector references with SmallVectorImpl references.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138063
2022-11-15 15:09:06 -08:00
bixia1
555e7835f4 [mlir][sparse] Fix rewriting for convert op and concatenate op.
Fix a problem in convert op rewriting where it used the original index for
ToIndicesOp.

Extend the concatenate op rewriting to handle dense destination and dynamic
shape destination.

Make the concatenate op integration test run on the codegen path.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138057
2022-11-15 14:45:21 -08:00
Peiming Liu
c3821e1684 [mlir][sparse] fix bugs in concatenate rewriter.
Reviewed By: aartbik, bixia

Differential Revision: https://reviews.llvm.org/D138053
2022-11-15 19:49:59 +00:00
Peiming Liu
8ffdcc594e [mlir][sparse] fix memory leak sparse2sparse reshape
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137994
2022-11-15 00:19:51 +00:00
bixia1
fa46de16db [mlir][sparse][NFC] Add comments to tests that are run for with and without runtime libraries.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137869
2022-11-14 08:21:33 -08:00
Peiming Liu
725e0849b7 [mlir][sparse] fix incorrect coordinates ordering computed by the foreach operation.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137877
2022-11-12 04:08:50 +00:00
bixia1
57416d872a [mlir][sparse] Fix a bug in rewriting dense2dense convert op.
Permutation wasn't handled correctly. Add a test for the rewriting.

Extend an integration test to run with enable_runtime_library=false to
also test the rewriting.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137845
2022-11-11 09:11:54 -08:00
bixia1
ccd1a5b9cb [mlir][sparse] Fix a bug in rewriting for the convert op.
The code to retrieve the number of entries isn't correct.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137795
2022-11-10 14:30:51 -08:00
Aart Bik
e6cbb91483 [mlir][sparse] skip zeros during dense2sparse
This enables the full matmul integration test with runtime_lib=true/false!

Background:
https://github.com/llvm/llvm-project/issues/51657

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137750
2022-11-09 20:54:27 -08:00
Aart Bik
a61a9a700a [mlir][sparse] first end-to-end matmul with codegen
(1) also fixes memory leak in sparse2dense rewriting
(2) still needs fix in dense2sparse by skipping zeros

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D137736
2022-11-09 13:40:05 -08:00
Peiming Liu
f45c21da0c [mlir][sparse] fix SSA chain issue in dense2sparse conversion.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137683
2022-11-09 02:03:33 +00:00
Peiming Liu
7175f9dde1 [mlir][sparse] extend foreach operation to iterator over sparse constant.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137679
2022-11-09 01:50:34 +00:00
Peiming Liu
4fa00ce15c [mlir][sparse] extend foreach operation to accept reduction arguments; fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
This patch re-commit D137468 and D137463, which were reverted by mistakes.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137579
2022-11-07 21:40:30 +00:00
Stella Stamenova
99171078bb Revert "[mlir][sparse] extend foreach operation to accept reduction arguments."
This reverts commit 53d5d3401120f2aa741a73a5a9ba0ce012ca532c.

This is causing a build failure on the windows mlir bot that was previously hidden by another sparse tensor change that caused failures:

https://lab.llvm.org/buildbot/#/builders/13/builds/28006
2022-11-07 09:34:10 -08:00
Stella Stamenova
ec224e3b68 Revert "[mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly."
This reverts commit 70508b614e6478ba2c3fc79e935e2c68e2d79b71.

This change depends on a reverted change that broke the windows mlir buildbot; reverting to bring remaining mlir bots to green
2022-11-07 09:00:08 -08:00
Peiming Liu
70508b614e [mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137468
2022-11-04 23:43:54 +00:00
Peiming Liu
53d5d34011 [mlir][sparse] extend foreach operation to accept reduction arguments.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137463
2022-11-04 23:34:16 +00:00
bixia1
3e724e529b [mlir][sparse] Fix windows build.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137210
2022-11-01 17:00:02 -07:00
bixia1
eb877006a6 [mlir][sparse] Add rewriting rule for the convert operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136301
2022-11-01 15:57:34 -07:00
Peiming Liu
63baab8b39 [mlir][sparse] Fold invariant op only when it has only one use.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136990
2022-10-28 23:42:28 +00:00
Alexander Belyaev
b4db15a949 [mlir] Rename getInputs->getDpsInputs and getOutputs->getDpsInits in DPS interface.
https://discourse.llvm.org/t/rfc-interface-for-destination-style-ops/64056

Differential Revision: https://reviews.llvm.org/D136943
2022-10-28 15:41:12 +02:00
Peiming Liu
b0f8057e4c [mlir][sparse] use loop emitter to generate loop in sparsification
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136185
2022-10-26 00:27:56 +00:00
Jeff Niu
dc63ca78f7 [mlir][sparse] Fix build warning (NFC) 2022-10-21 13:13:07 -07:00
bixia1
e445349d2c [mlir][sparse] Add rewrite rule for the sparse_tensor.out operator.
Also fix the rewrite rule for sparse_tensor.new to reflect the recent change of
the runtime C interface and to use utilities for memref.alloca.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135891
2022-10-21 12:20:53 -07:00
Peiming Liu
35b3a0ce8d [mlir][sparse] support foreach on dense tensor.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136384
2022-10-21 00:12:37 +00:00
bixia1
cf7906cbbd [mlir][sparse] Fix getUnorderedCOOFromType for rank 1 tensor.
Previously, it used DimLevelType::SingletonNo to represent an unorder COO
tensor of rank 1 while it should use DimLevelType::CompressedNuNo.

Reviewed By: Peiming, wrengr

Differential Revision: https://reviews.llvm.org/D136387
2022-10-20 16:33:01 -07:00
wren romano
0e77b63bc0 [mlir][sparse] Use the runtime DimLevelType instead of a separate tablegen enum
This differential replaces all uses of SparseTensorEncodingAttr::DimLevelType with DimLevelType.  The next differential will break out a separate library for the DimLevelType enum, so that the Dialect code doesn't need to depend on the rest of the runtime

Depends On D135995

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135996
2022-10-18 13:45:26 -07:00
bixia1
c1864ab953 [mlir][sparse] Add options to sparse-tensor-rewrite to disable rewriting rules for operators foreach and convert.
This is to help simplify FileCheck tests for sparse-tensor-rewrite.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136093
2022-10-18 09:27:32 -07:00
bixia1
933e7e2393 [mlir][sparse] Change getUnorderedCOOFromType to propagate the overhead types.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136112
2022-10-18 08:01:16 -07:00
Alexander Belyaev
a7cccb9cbb [mlir] Simplify DestinationStyleOpInterface.
Differential Revision: https://reviews.llvm.org/D135348
2022-10-17 12:43:41 +02:00
Oleg Shyshkov
c38d9cf20e [mlir] Remove iterator_types() method from LinalgStructuredInterface.
`getIteratorTypesArray` should be used instead. It's a better substitute for all the current usages of the interface.

The current `ArrayAttr iterator_types()` has a few problems:
* It creates an assumption operation has iterators types as an attribute, but it's not always the case. Sometime iterator types can be inferred from other attribute, or they're just static.
* ArrayAttr is an obscure contained and required extracting values in the client code.
* Makes it hard to migrate iterator types from strings to enums ([RFC](https://discourse.llvm.org/t/rfc-enumattr-for-iterator-types-in-linalg/64535/9)).

Concrete ops, like `linalg.generic` will still have iterator types as an attribute if needed.

As a side effect, this change helps a bit with migration to prefixed accessors.

Differential Revision: https://reviews.llvm.org/D135765
2022-10-13 07:52:43 +00:00
Kazu Hirata
b2674de375 [mlir] Fix a warning
This patch fixes:

  mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorRewriting.cpp:587:27:
  error: comparison of integers of different signs: 'int64_t' (aka
  'long') and 'uint64_t' (aka 'unsigned long')
  [-Werror,-Wsign-compare]
2022-10-12 15:25:17 -07:00
bixia1
67f61b0863 [mlir][sparse] Add rewriting rules for the sparse_tensor.new operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135692
2022-10-11 15:23:11 -07:00
bixia1
330d48c4aa [mlir][sparse] Add rewrite rules for sparse-to-sparse reshape operators.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135077
2022-10-06 08:50:30 -07:00
Aart Bik
c48e90877f [mlir][sparse] introduce a higher-order tensor mapping
This extension to the sparse tensor type system in MLIR
opens up a whole new set of sparse storage schemes, such as
block sparse storage (e.g. BCSR) and ELL (aka jagged diagonals).

This revision merely introduces the type extension and
initial documentation. The actual interpretation of the type
(reading in tensors, lowering to code, etc.) will follow.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D135206
2022-10-05 09:40:51 -07:00
Peiming Liu
00ad065548 [mlir][sparse] Add rewriting rules for concatente using foreach operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134895
2022-09-30 21:56:55 +00:00
Peiming Liu
550288cbc3 [mlir][sparse] Add new utility class to help generates loop structures over sparse tensors; Implement foreach operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134782
2022-09-30 21:42:42 +00:00
Oleg Shyshkov
1227b8ab54 [mlir] Rename getTied* methods to getMatching* in LinalgInterface.
Summary:
As mentioned in the comment to https://reviews.llvm.org/D134444, the term `tied`
is a misnomer in this context and `matching` sounds much better.

Differential Revision: https://reviews.llvm.org/D134534
2022-09-30 10:05:45 +00:00
Jakub Kuderski
abc362a107 [mlir][arith] Change dialect name from Arithmetic to Arith
Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples`

and `bazel build --config=generic_clang @llvm-project//mlir:all`.

Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D134762
2022-09-29 11:23:28 -04:00
Peiming Liu
eb65327fe9 [mlir][sparse] Add new option (enable-runtime-library) to sparse compiler pipeline
Add new option (enable-runtime-library) to sparse compiler pipeline, it allows us to decide whether we need to rewrite operations (e.g., concatenate, reshape) within sparsification (when using codegen) or convert them after sparsification (when using runtime library).

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133597
2022-09-09 20:54:47 +00:00
Mehdi Amini
0449b6a07a Apply clang-tidy fixes for llvm-else-after-return in SparseTensorRewriting.cpp (NFC) 2022-09-07 19:30:54 +00:00
Aart Bik
0d4e7fba9a [mlir][sparse] minor zero test refactoring in rewriting
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D133382
2022-09-07 10:07:11 -07:00
Peiming Liu
ec495b53f8 [mlir][sparse] Folding operations that try to insert zero into an all-zero sparse tensor
The operations to fill zero into newly allocated sparse tensor are redundant, plus it failed
to lowering the test cases provided in the patch as well.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D132500
2022-08-25 17:00:04 +00:00
Kazu Hirata
06b551c944 Use llvm::is_contained (NFC) 2022-08-20 21:18:27 -07:00
Jacques Pienaar
d3b3f7653d [mlir] Flip to prefixed accessors (NFC) 2022-08-07 04:55:58 -07:00
Aart Bik
c7bb69bc75 [mlir][sparse] replace zero yield generic op with copy in allocation
This prepares patterns that sometimes are generated by the front-end
and would prohibit fusion of SDDMM flavored kernels.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D131126
2022-08-04 09:33:57 -07:00
Aart Bik
ce3d0e87ac [mlir][sparse] enable SDDMM-flavored fusion
This rewriting was no longer functional after recent migration to one shot
bufferization. However, this revision makes it work again, with a CHECK test
to ensure fusion happens. Note that functionality is tested by several
integration tests.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D130996
2022-08-02 12:40:04 -07:00