58 Commits

Author SHA1 Message Date
Mehdi Amini
5e92f7edd2 Apply clang-tidy fixes for performance-unnecessary-value-param in SparseTensorCodegen.cpp (NFC) 2023-01-06 10:38:54 +00:00
bixia1
81e3079d0f [mlir][sparse] Replace sparse_tensor.sort with sparse_tensor.sort_coo for sorting COO tensors.
Add codegen pattern for sparse_tensor.indices_buffer.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140871
2023-01-05 15:42:57 -08:00
bixia1
3fdd85da06 [mlir][sparse] Add AOS optimization.
Use an array of structures to represent the indices for the tailing COO region
of a sparse tensor.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140870
2023-01-04 18:16:04 -08:00
bixia1
63d31a4d15 [mlir][sparse] Move some member functions from SparseTensorDescriptorImpl to MutSparseTensorDescriptor.
This is to prepare for implementing AOS optimization.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D141002
2023-01-04 13:05:43 -08:00
bixia1
90aa436291 [mlir][sparse] Add layout to the memref for the indices buffers to prepare for the AOS storage optimization for COO regions.
Fix relevant FileCheck tests.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140742
2023-01-04 07:36:11 -08:00
bixia1
840e2ba336 [mlir][sparse] Use DLT in the mangled function names for insertion.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140484
2022-12-28 08:21:22 -08:00
Peiming Liu
988733c600 [mlir][sparse] use sparse_tensor::StorageSpecifier to store dim/memSizes
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140130
2022-12-23 00:47:36 +00:00
Ramkumar Ramachandra
0de16fafa5 mlir/DialectConversion: use std::optional (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch touches DialectConversion, and modifies
existing conversions and tests appropriately.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Differential Revision: https://reviews.llvm.org/D140303
2022-12-19 18:48:59 +01:00
Ramkumar Ramachandra
22426110c5 mlir/tblgen: use std::optional in generation
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch changes the way mlir-tblgen generates .inc
files, and modifies tests and documentation appropriately. It is a "no
compromises" patch, and doesn't leave the user with an unpleasant mix of
llvm::Optional and std::optional.

A non-trivial change has been made to ControlFlowInterfaces to split one
constructor into two, relating to a build failure on Windows.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Differential Revision: https://reviews.llvm.org/D138934
2022-12-17 11:13:26 +01:00
bixia1
ea4be70cea [mlir][sparse] Fix problems in creating complex zero for initialization.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139591
2022-12-08 07:49:27 -08:00
Peiming Liu
191c43f60e Revert "Revert "[mlir][sparse] Refactoring: abstract sparse tensor memory scheme into a SparseTensorDescriptor class.""
This reverts commit 10033a179f0c73f28f051ac70b058a0c61882e3a. Plus, it fixed windows warnings and gcc errors

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139384
2022-12-06 17:12:06 +00:00
Stella Stamenova
10033a179f Revert "[mlir][sparse] Refactoring: abstract sparse tensor memory scheme into a SparseTensorDescriptor class."
This reverts commit 8a7e69d145ff72e7e4fc10ce6b81c3aa4794201c.

This broke the windows mlir buildbot: https://lab.llvm.org/buildbot/#/builders/13/builds/29257
2022-12-05 17:20:01 -08:00
Peiming Liu
8a7e69d145 [mlir][sparse] Refactoring: abstract sparse tensor memory scheme into a SparseTensorDescriptor class.
This patch abstracts sparse tensor memory scheme into a SparseTensorDescriptor class. Previously, the field accesses are performed in a relatively error-prone way, this patch hides the hairy details behind a SparseTensorDescriptor class to allow users access sparse tensor fields in a more cohesive way.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138627
2022-12-05 22:11:53 +00:00
Kazu Hirata
70c73d1b72 [mlir] Use std::nullopt instead of None in comments (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-04 17:23:50 -08:00
Kazu Hirata
1a36588ec6 [mlir] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-03 18:50:27 -08:00
Peiming Liu
dd33481f48 [mlir][sparse] add getPointerType/getIndexType to SparseTensorEncodingAttr.
add new interfaces to SparseTensorEncodingAttr to construct the pointer/index types based on pointer/index bitwidth.

Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139141
2022-12-01 22:01:52 +00:00
bixia1
2aceadda78 [mlir][sparse] Put the implementation for the insertion operation to subroutines.
Previously, we generated inlined implementation for insert operation and
observed MLIR compile time increase due to the size of the main routine. We now
put the insert operation implementation in subroutines and leave the inlining
decision to the MLIR compiler.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138957
2022-12-01 13:23:59 -08:00
Aliia Khasanova
399638f98c Merge kDynamicSize and kDynamicSentinel into one constant.
resolve conflicts

Differential Revision: https://reviews.llvm.org/D138282
2022-11-21 13:01:26 +00:00
Mehdi Amini
7c5f06a7dc Apply clang-tidy fixes for llvm-else-after-return in SparseTensorCodegen.cpp (NFC) 2022-11-18 06:22:53 +00:00
Peiming Liu
709e23a1c1 [mlir][sparse] fix CHECK test error on windows.
Reviewed By: aartbik, bixia

Differential Revision: https://reviews.llvm.org/D138147
2022-11-16 20:29:21 +00:00
Aart Bik
c4f1af2bdf [mlir][sparse] avoid single small vector, set exact number 3
Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D138071
2022-11-15 21:04:43 -08:00
Aart Bik
0e1708ff64 [mlir][sparse] cleanup small vector constant hints
Following advise from

https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallvector-h

This revision removes the size hints from SmallVector (unless we are
certain of the resulting number of elements). Also, this replaces
SmallVector references with SmallVectorImpl references.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138063
2022-11-15 15:09:06 -08:00
Mehdi Amini
ca9916e984 Apply clang-tidy fixes for readability-identifier-naming in SparseTensorCodegen.cpp (NFC) 2022-11-15 21:20:39 +00:00
Mehdi Amini
ef99c27d3b Apply clang-tidy fixes for llvm-else-after-return in SparseTensorCodegen.cpp (NFC) 2022-11-15 21:20:39 +00:00
Peiming Liu
5ab1a8aef4 [mlir][sparse] fix crash when calling getTuple on non-sparse tensors.
This enables full sparse convolution codegen in D137298

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137853
2022-11-11 21:15:44 +00:00
bixia1
7276b643f8 [mlir][sparse] Add option enable-buffer-initialization to the sparse-tensor-codegen pass.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137733
2022-11-10 07:23:33 -08:00
Aart Bik
129177eaf0 [mlir][sparse] add sparse matmul codegen check tests
This found an insertion point bug in access pattern expansion

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137675
2022-11-08 15:58:30 -08:00
Aart Bik
70633a8d55 [mlir][sparse] first general insertion implementation with pure codegen
This revision generalizes lowering the sparse_tensor.insert op into actual code that directly operates on the memrefs of a sparse storage scheme. The current insertion strategy does *not* rely on a cursor anymore, with introduces some testing overhead for each insertion (but still proportional to the rank, as before). Over time, we can optimize the code generation, but this version enables us to finish the effort to migrate from library to actual codegen.

Things to do:
(1) carefully deal with (un)ordered and (not)unique
(2) omit overhead when not needed
(3) optimize and specialize
(4) try to avoid the pointer "cleanup" (at HasInserts), and make sure the storage scheme is consistent at every insertion point (so that it can "escape" without concerns).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137457
2022-11-08 13:10:05 -08:00
Aart Bik
10db57b7ea [mlir][sparse] replace magic constant with symbol
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137177
2022-11-01 11:13:20 -07:00
Aart Bik
5661647e85 [mlir][sparse] build proper insertion chain
The alloc->insert/compress->load chain needs to be
properly represented with an SSA chain now in loops
and if statements to properly reflect the modifying
behavior (runtime support lib is forgiving on breaking
this, but the new codegen is not).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136966
2022-10-28 15:58:51 -07:00
Jordan Rupprecht
71ac3ce938 [NFC] Remove unused variables 2022-10-27 12:23:22 -07:00
Aart Bik
80b08b68f2 [mlir][sparse] add a cursor to sparse storage scheme
This prepare a subsequent revision that will generalize
the insertion code generation. Similar to the support lib,
insertions become much easier to perform with some "cursor"
bookkeeping. Note that we, in the long run, could perhaps
avoid storing the "cursor" permanently and use some
retricted-scope solution (alloca?) instead. However,
that puts harder restrictions on insertion-chain operations,
so for now we follow the more straightforward approach.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136800
2022-10-27 11:18:50 -07:00
Aart Bik
0f3e4d1afa [mlir][sparse] lower number of entries op to actual code
works both along runtime path and pure codegen path

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136389
2022-10-21 10:48:37 -07:00
bixia1
0128f80167 [mlir][sparse] Fix the codegen for the convert operator to handle hidden nop convert.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136291
2022-10-20 12:30:21 -07:00
Peiming Liu
d12d4857c5 [mlir][sparse] add source materizalization callback for sparse tensor codegen type converter.
Required by scf.for to achieve 1:N type conversion (See D136314).

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136367
2022-10-20 18:04:57 +00:00
Aart Bik
96cab659a1 [mlir][sparse] end-to-end sparse vector insertion codegen
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136275
2022-10-19 12:32:20 -07:00
bixia1
58b449c3bb [mlir][sparse] Replace the folding of nop convert with a codegen rule.
This is to allow the use of a nop convert to express that the sparse tensor
allocated through bufferization::AllocTensorOp will be expanded to sparse
tensor storage by sparse tensor codegen.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136214
2022-10-19 10:20:47 -07:00
Aart Bik
d22df0ebba [mlir][sparse] refine insertion code
builds SSA cycle for compress insertion loop
adds casting on index mismatch during push_back

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136186
2022-10-18 14:16:38 -07:00
Aart Bik
9f596a7c67 [mlir][sparse] implement simple codegen for insertion (and related ops)
This is a proof of concept insertion implementation that sets up
the basic framework and implements it with push backs for just
sparse vectors. It adds insertion/compression through SSA values,
so that we properly update the memref after after pushback operation.

Note that properly using SSA values in sparsification is still TBD
but I will wait until Peiming's loop emitter is in to avoid conflicts.

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D136008
2022-10-17 18:02:08 -07:00
Mehdi Amini
d7802c4811 Apply clang-tidy fixes for llvm-qualified-auto in SparseTensorCodegen.cpp (NFC) 2022-10-13 02:58:34 +00:00
bixia1
654bbbde55 [mlir][sparse] Move the implementation of sparse_tensor.push_back to the buffer rewriter.
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D134777
2022-09-29 15:06:00 -07:00
Aart Bik
f231821e6e [mlir][sparse] provide convenience methods for toOrig/toStoredDim
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D134833
2022-09-28 16:32:03 -07:00
Aart Bik
4d06861950 [mlir][sparse] add "sort" to the compress op codegen
This revision also adds convenience methods to test the
dim level type/property (with the codegen being first client)

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D134776
2022-09-28 10:41:40 -07:00
bixia1
4132bce9e5 [mlir][sparse] Add codegen rule for the push_back operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134372
2022-09-22 09:09:49 -07:00
Aart Bik
3986c86986 [mlir][sparse] partially implement codegen for sparse_tensor.compress
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133912
2022-09-15 10:32:33 -07:00
Aart Bik
6607fdf749 [mlir][sparse] add memSizes array to sparse storage format
Rationale:
For every dynamic memref (memref<?xtype>), the stored size really
indicates the capacity and the entry in the memSizes indicates
the actual size. This allows us to use memref's as "vectors".

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133724
2022-09-12 14:04:01 -07:00
Peiming Liu
938f419cf1 [mlir][sparse] Avoid generating DimOp in conversion passes.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133592
2022-09-09 18:08:05 +00:00
bixia1
8a583bd53d [mlir][sparse] Add codegen for expand op.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133454
2022-09-08 14:06:01 -07:00
Aart Bik
610b09074a [mlir][sparse] change variable dimension to fixed attribute pointers/indices
The "sparsification" pass does not need the ability to use runtime values for
the dimension, so the only source for variability would have been user code.
Restricting the dimension to constants simplifies code generation.

Reviewed By: Peiming, wrengr

Differential Revision: https://reviews.llvm.org/D133458
2022-09-07 16:27:24 -07:00
Peiming Liu
edca72f5bc [mlir][sparse] Refactoring: remove dependence on tuple type when lowering sparse tensors.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133390
2022-09-07 17:53:48 +00:00