43 Commits

Author SHA1 Message Date
Peiming Liu
dd33481f48 [mlir][sparse] add getPointerType/getIndexType to SparseTensorEncodingAttr.
add new interfaces to SparseTensorEncodingAttr to construct the pointer/index types based on pointer/index bitwidth.

Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139141
2022-12-01 22:01:52 +00:00
bixia1
2aceadda78 [mlir][sparse] Put the implementation for the insertion operation to subroutines.
Previously, we generated inlined implementation for insert operation and
observed MLIR compile time increase due to the size of the main routine. We now
put the insert operation implementation in subroutines and leave the inlining
decision to the MLIR compiler.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138957
2022-12-01 13:23:59 -08:00
Aliia Khasanova
399638f98c Merge kDynamicSize and kDynamicSentinel into one constant.
resolve conflicts

Differential Revision: https://reviews.llvm.org/D138282
2022-11-21 13:01:26 +00:00
Mehdi Amini
7c5f06a7dc Apply clang-tidy fixes for llvm-else-after-return in SparseTensorCodegen.cpp (NFC) 2022-11-18 06:22:53 +00:00
Peiming Liu
709e23a1c1 [mlir][sparse] fix CHECK test error on windows.
Reviewed By: aartbik, bixia

Differential Revision: https://reviews.llvm.org/D138147
2022-11-16 20:29:21 +00:00
Aart Bik
c4f1af2bdf [mlir][sparse] avoid single small vector, set exact number 3
Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D138071
2022-11-15 21:04:43 -08:00
Aart Bik
0e1708ff64 [mlir][sparse] cleanup small vector constant hints
Following advise from

https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallvector-h

This revision removes the size hints from SmallVector (unless we are
certain of the resulting number of elements). Also, this replaces
SmallVector references with SmallVectorImpl references.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138063
2022-11-15 15:09:06 -08:00
Mehdi Amini
ca9916e984 Apply clang-tidy fixes for readability-identifier-naming in SparseTensorCodegen.cpp (NFC) 2022-11-15 21:20:39 +00:00
Mehdi Amini
ef99c27d3b Apply clang-tidy fixes for llvm-else-after-return in SparseTensorCodegen.cpp (NFC) 2022-11-15 21:20:39 +00:00
Peiming Liu
5ab1a8aef4 [mlir][sparse] fix crash when calling getTuple on non-sparse tensors.
This enables full sparse convolution codegen in D137298

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137853
2022-11-11 21:15:44 +00:00
bixia1
7276b643f8 [mlir][sparse] Add option enable-buffer-initialization to the sparse-tensor-codegen pass.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137733
2022-11-10 07:23:33 -08:00
Aart Bik
129177eaf0 [mlir][sparse] add sparse matmul codegen check tests
This found an insertion point bug in access pattern expansion

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137675
2022-11-08 15:58:30 -08:00
Aart Bik
70633a8d55 [mlir][sparse] first general insertion implementation with pure codegen
This revision generalizes lowering the sparse_tensor.insert op into actual code that directly operates on the memrefs of a sparse storage scheme. The current insertion strategy does *not* rely on a cursor anymore, with introduces some testing overhead for each insertion (but still proportional to the rank, as before). Over time, we can optimize the code generation, but this version enables us to finish the effort to migrate from library to actual codegen.

Things to do:
(1) carefully deal with (un)ordered and (not)unique
(2) omit overhead when not needed
(3) optimize and specialize
(4) try to avoid the pointer "cleanup" (at HasInserts), and make sure the storage scheme is consistent at every insertion point (so that it can "escape" without concerns).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137457
2022-11-08 13:10:05 -08:00
Aart Bik
10db57b7ea [mlir][sparse] replace magic constant with symbol
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137177
2022-11-01 11:13:20 -07:00
Aart Bik
5661647e85 [mlir][sparse] build proper insertion chain
The alloc->insert/compress->load chain needs to be
properly represented with an SSA chain now in loops
and if statements to properly reflect the modifying
behavior (runtime support lib is forgiving on breaking
this, but the new codegen is not).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136966
2022-10-28 15:58:51 -07:00
Jordan Rupprecht
71ac3ce938 [NFC] Remove unused variables 2022-10-27 12:23:22 -07:00
Aart Bik
80b08b68f2 [mlir][sparse] add a cursor to sparse storage scheme
This prepare a subsequent revision that will generalize
the insertion code generation. Similar to the support lib,
insertions become much easier to perform with some "cursor"
bookkeeping. Note that we, in the long run, could perhaps
avoid storing the "cursor" permanently and use some
retricted-scope solution (alloca?) instead. However,
that puts harder restrictions on insertion-chain operations,
so for now we follow the more straightforward approach.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136800
2022-10-27 11:18:50 -07:00
Aart Bik
0f3e4d1afa [mlir][sparse] lower number of entries op to actual code
works both along runtime path and pure codegen path

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136389
2022-10-21 10:48:37 -07:00
bixia1
0128f80167 [mlir][sparse] Fix the codegen for the convert operator to handle hidden nop convert.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136291
2022-10-20 12:30:21 -07:00
Peiming Liu
d12d4857c5 [mlir][sparse] add source materizalization callback for sparse tensor codegen type converter.
Required by scf.for to achieve 1:N type conversion (See D136314).

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136367
2022-10-20 18:04:57 +00:00
Aart Bik
96cab659a1 [mlir][sparse] end-to-end sparse vector insertion codegen
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136275
2022-10-19 12:32:20 -07:00
bixia1
58b449c3bb [mlir][sparse] Replace the folding of nop convert with a codegen rule.
This is to allow the use of a nop convert to express that the sparse tensor
allocated through bufferization::AllocTensorOp will be expanded to sparse
tensor storage by sparse tensor codegen.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136214
2022-10-19 10:20:47 -07:00
Aart Bik
d22df0ebba [mlir][sparse] refine insertion code
builds SSA cycle for compress insertion loop
adds casting on index mismatch during push_back

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136186
2022-10-18 14:16:38 -07:00
Aart Bik
9f596a7c67 [mlir][sparse] implement simple codegen for insertion (and related ops)
This is a proof of concept insertion implementation that sets up
the basic framework and implements it with push backs for just
sparse vectors. It adds insertion/compression through SSA values,
so that we properly update the memref after after pushback operation.

Note that properly using SSA values in sparsification is still TBD
but I will wait until Peiming's loop emitter is in to avoid conflicts.

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D136008
2022-10-17 18:02:08 -07:00
Mehdi Amini
d7802c4811 Apply clang-tidy fixes for llvm-qualified-auto in SparseTensorCodegen.cpp (NFC) 2022-10-13 02:58:34 +00:00
bixia1
654bbbde55 [mlir][sparse] Move the implementation of sparse_tensor.push_back to the buffer rewriter.
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D134777
2022-09-29 15:06:00 -07:00
Aart Bik
f231821e6e [mlir][sparse] provide convenience methods for toOrig/toStoredDim
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D134833
2022-09-28 16:32:03 -07:00
Aart Bik
4d06861950 [mlir][sparse] add "sort" to the compress op codegen
This revision also adds convenience methods to test the
dim level type/property (with the codegen being first client)

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D134776
2022-09-28 10:41:40 -07:00
bixia1
4132bce9e5 [mlir][sparse] Add codegen rule for the push_back operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134372
2022-09-22 09:09:49 -07:00
Aart Bik
3986c86986 [mlir][sparse] partially implement codegen for sparse_tensor.compress
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133912
2022-09-15 10:32:33 -07:00
Aart Bik
6607fdf749 [mlir][sparse] add memSizes array to sparse storage format
Rationale:
For every dynamic memref (memref<?xtype>), the stored size really
indicates the capacity and the entry in the memSizes indicates
the actual size. This allows us to use memref's as "vectors".

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133724
2022-09-12 14:04:01 -07:00
Peiming Liu
938f419cf1 [mlir][sparse] Avoid generating DimOp in conversion passes.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133592
2022-09-09 18:08:05 +00:00
bixia1
8a583bd53d [mlir][sparse] Add codegen for expand op.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133454
2022-09-08 14:06:01 -07:00
Aart Bik
610b09074a [mlir][sparse] change variable dimension to fixed attribute pointers/indices
The "sparsification" pass does not need the ability to use runtime values for
the dimension, so the only source for variability would have been user code.
Restricting the dimension to constants simplifies code generation.

Reviewed By: Peiming, wrengr

Differential Revision: https://reviews.llvm.org/D133458
2022-09-07 16:27:24 -07:00
Peiming Liu
edca72f5bc [mlir][sparse] Refactoring: remove dependence on tuple type when lowering sparse tensors.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133390
2022-09-07 17:53:48 +00:00
Peiming Liu
4c46a5d54d [mlir][sparse] Refactoring: renaming StorageNewOp to StorageOp
To address comment in https://reviews.llvm.org/D133241

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133363
2022-09-06 17:02:25 +00:00
Aart Bik
0c7abd3924 [mlir][sparse] codegen for sparse alloc
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133241
2022-09-06 09:37:54 -07:00
Aart Bik
2ddfacd95c [mlir][sparse] codegen for sparse dealloc
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D133171
2022-09-01 22:21:20 -07:00
Aart Bik
f27b806df5 [mlir][sparse] codegen for trivial tensor cast
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D133176
2022-09-01 21:55:18 -07:00
Aart Bik
3ae98fd259 [mlir][sparse] added codegen for dimop, pointers, indices, values
Demonstrates how sparse tensor type -> tuple -> getter
will eventually yield actual code on the memrefs directly

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133143
2022-09-01 16:36:10 -07:00
Aart Bik
1be09496bf [mlir][sparse] improved tensor type lowering
Also includes a first codegen example (although full support need tuple access)

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133080
2022-09-01 09:24:20 -07:00
Aart Bik
f767f09252 [mlir][sparse] sparse storage scheme type conversion
This builds a compound type for the buffers required for the sparse storage scheme defined by the MLIR sparse tensor types. The use of a tuple allows for a simple 1:1 type conversion. A subsequent pass can expand this tuple into its component with an isolated 1:N type conversion.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D133050
2022-08-31 15:12:55 -07:00
Aart Bik
86b22d3120 [mlir][sparse] start a sparse codegen conversion pass
This new pass provides an alternative to the current conversion pass
that converts sparse tensor types and sparse primitives to opaque pointers
and calls into a runtime support library. This pass will map sparse tensor
types to actual data structures and primitives to actual code. In the long
run, this new pass will remove our dependence on the support library, avoid
the need to link in fully templated and expanded code, and provide much better
opportunities for optimization on the generated code.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D132766
2022-08-29 09:39:33 -07:00