838 Commits

Author SHA1 Message Date
Peiming Liu
269685545e
[mlir][sparse] remove filter-loop based algorithm support to handle a… (#71840)
…ffine subscript expressions.
2023-11-13 11:36:49 -08:00
Aart Bik
af8428c0d9
[mlir][sparse] unify support of (dis)assemble between direct IR/lib path (#71880)
Note that the (dis)assemble operations still make some simplfying
assumptions (e.g. trailing 2-D COO in AoS format) but now at least both
the direct IR and support library path behave exactly the same.

Generalizing the ops is still TBD.
2023-11-13 10:05:00 -08:00
Peiming Liu
bfe08c094d
[mlir][sparse] support sparsifying 2:4 block sparsity (#71749) 2023-11-10 12:25:53 -08:00
Peiming Liu
c99951d491
[mlir][sparse] end-to-end matmul between Dense and BSR tensors (#71448) 2023-11-08 11:28:00 -08:00
Aart Bik
5ef446790f
[mlir][sparse][gpu] cleanup GPUDataTransferStrategy (#71615)
The flag seems to be doing practically the same thing for zero cost and
pinned dma. In addition, the register host is not truly the right zero
cost mechanism according to Thomas. So we are simplifying the setup for
now, until we have a better definition for what to implement and test.
    
https://github.com/llvm/llvm-project/issues/64316
2023-11-08 09:45:11 -08:00
Tim Harvey
c43e627457
Changed the phrase sparse-compiler to sparsifier in comments (#71578)
When the Powers That Be decided that the name "sparse compiler" should
be changed to "sparsifier", we negected to change some of the comments
in the code; this pull request completes the name change.
2023-11-07 20:55:00 +00:00
Aart Bik
160d483b1f
[mlir][sparse] implement loose-compressed/2:4 on direct IR codegen path (#71461)
Fills in the missing cases for direct IR codegen.
Note that non-permutation handling is still TBD.
2023-11-06 17:30:56 -08:00
Aart Bik
0500c9348c
[mlir][sparse] update doc of sparse tensor storage-layout/descriptor (#71249)
This prepares actual "direct IR codegen" for loose compressed and 2:4.
Also bit of cleanup of stale TODOs
2023-11-03 16:20:58 -07:00
Aart Bik
bc61122a71
[mlir][sparse] reformat SparseTensorCodegen file (#71231) 2023-11-03 14:35:43 -07:00
Aart Bik
2b67942139
[mlir][sparse] remove (some) deprecated dim/lvl methods (#71125)
This removes the most obvious ones. The others are still TBD.
2023-11-02 16:29:27 -07:00
Peiming Liu
53ffafb24d
[mlir][sparse] support sparse constant to BSR conversion. (#71114)
support direct convert from a constant tensor defined by
SparseArrayElements to BSR
2023-11-02 14:45:39 -07:00
Aart Bik
ee3ee1315a
[mlir][sparse] cleanup of enums header (#71090)
Some DLT related methods leaked into sparse_tensor.h, and this moves it
back to the right header. Also, the asserts were incomplete and some DLT
methods duplicated.
2023-11-02 13:00:27 -07:00
Peiming Liu
deedf554fb
[mlir][sparse] Cleanup sparse_tensor::LvlOp's folder (#71085)
Reuse the util function instead.
2023-11-02 11:31:16 -07:00
Aart Bik
22212ca745
[mlir][sparse] simplify some header code (#70989)
This is a first revision in a small series of changes that removes
duplications between direct encoding methods and sparse tensor type
wrapper methods (in favor of the latter abstraction, since it provides
more safety). The goal is to simply end up with "just" SparseTensorType
2023-11-02 09:31:11 -07:00
Peiming Liu
c0d78c4232
[mlir][sparse] Implement rewriters to reinterpret maps on alloc_tenso… (#70993)
…r operation
2023-11-01 18:15:11 -07:00
Peiming Liu
3426d330a7
[mlir][sparse] Implement rewriters to reinterpret maps on foreach (#70868) 2023-11-01 12:11:47 -07:00
Aart Bik
b19c40c579
[mlir][sparse] first end-to-end linalg.generic op on BSR (#70880) 2023-11-01 10:01:22 -07:00
Aart Bik
e599978760
[mlir][sparse] first proof-of-concept non-permutation rewriter (#70863)
Rather than extending sparsifier codegen with higher order
non-permutations, we follow the path of rewriting linalg generic ops
into higher order operations. That way, code generation will simply work
out of the box. This is a very first proof-of-concept rewriting of that
idea.
2023-10-31 16:19:27 -07:00
Peiming Liu
c81a2c058e
[mlir][sparse] add helper class to implement common rewriter to re/demap sparse tensors. (#70750) 2023-10-31 12:35:52 -07:00
Christian Ulmann
dcae289d3a
[MLIR][SparseTensor] Introduce opaque pointers in LLVM dialect lowering (#70570)
This commit changes the SparseTensor LLVM dialect lowering from using
`llvm.ptr<i8>` to `llvm.ptr`. This change ensures that the lowering now
properly relies on opaque pointers, instead of working with already type
erased i8 pointers.
2023-10-31 07:34:49 +01:00
Peiming Liu
ef100c228a
[mlir][sparse] implements tensor.insert on sparse tensors. (#70737) 2023-10-30 16:04:41 -07:00
Peiming Liu
f82bee1367
[mlir][sparse] split post-sparsification-rewriting into two passes. (#70727) 2023-10-30 15:22:21 -07:00
Vlad Serebrennikov
7c2ef38c36 [mlir][NFC] Use llvm::to_underlying in sparse tensor IR detail 2023-10-30 22:34:50 +03:00
Peiming Liu
7d608ee2bb
[mlir][sparse] unify sparse_tensor.out rewriting rules (#70518) 2023-10-27 16:46:58 -07:00
Peiming Liu
6a93da9900
[mlir][sparse] add ReinterpretMapScopeOption for the pass (#70486) 2023-10-27 14:14:09 -07:00
Aart Bik
7cfac1bedd
[mlir][sparse] add boilterplate code for a new reintepret map pass (#70393)
The interesting stuff is of course still coming ;-)
2023-10-26 17:57:46 -07:00
Peiming Liu
ef222988b4
[mlir][sparse] implements sparse_tensor.reinterpret_map (#70388) 2023-10-26 16:00:32 -07:00
Yinying Li
b165650aee
[mlir][sparse] Return actual identity map instead of null map (#70365)
Changes:

1. For both dimToLvl and lvlToDim, always returns the actual map instead
of AffineMap() for identity map.
2. Updated custom builder for encoding to have default values.
3. Non-inferable lvlToDim will still return AffineMap() during
inference, so it will be caught by verifier.
2023-10-26 18:30:34 -04:00
Peiming Liu
d808d922b4
[mlir][sparse] introduce sparse_tensor.reinterpret_map operation. (#70378) 2023-10-26 15:04:09 -07:00
Aart Bik
b1bb23991a
[mlir][sparse] merger cleanup (#70371)
Implemented some TODOs and removed unlikely ones.
Comment cleanup
2023-10-26 12:46:23 -07:00
Aart Bik
2e2011da38
[mlir][sparse] avoid excessive macro magic (#70276)
The shorthands are not even always shorter and the code is less clear
than when simply written out.
2023-10-25 20:48:37 -07:00
Aart Bik
740582fa4c
[mlir][sparse] test for linalg tensor semantics (#70254)
This test used to be here, but somehow got lost while linalg rewrote
their interfaces. It is essential to test this on entry of
sparsification, however, since all subsequent analysis simply assumes
tensor types.

Fixes:
https://github.com/llvm/llvm-project/issues/64325
2023-10-25 14:07:38 -07:00
Aart Bik
7e83a1af5d
[mlir][sparse] add verification of absent value in sparse_tensor.unary (#70248)
This value should always be a plain contant or something invariant
computed outside the surrounding linalg operation, since there is no
co-iteration defined on anything done in this branch.

Fixes:
https://github.com/llvm/llvm-project/issues/69395
2023-10-25 13:56:43 -07:00
Peiming Liu
c780352de9
[mlir][sparse] implement sparse_tensor.lvl operation. (#69993) 2023-10-24 13:23:28 -07:00
Peiming Liu
43961264c8
[mlir][sparse] hoists alloca outside the outermost loop. (#70085) 2023-10-24 11:22:50 -07:00
Peiming Liu
f0f5fdf73d
[mlir][sparse] introduce sparse_tensor.lvl operation. (#69978) 2023-10-23 15:49:39 -07:00
Yinying Li
34ed07e6a1
[mlir][sparse] Parser cleanup (#69792)
Removed TODOs, FIXMEs and long notes that are more suited for design
doc.
2023-10-23 14:30:02 -04:00
Peiming Liu
e9fa1fdec9
[mlir][sparse] support CSR/BSR conversion (#69800) 2023-10-20 17:24:34 -07:00
Peiming Liu
6456e0bbbb
[mlir][sparse] implement sparse_tensor.crd_translate operation (#69653) 2023-10-20 16:18:02 -07:00
Peiming Liu
6243d7d28b
[mlir][sparse] fix stack overflow due to memref.alloca in loops (#69786) 2023-10-20 16:01:36 -07:00
Kazu Hirata
6461a824e4 [Transforms] Use llvm::erase_if (NFC) 2023-10-20 00:08:56 -07:00
Aart Bik
3d89c088af
[mlir][sparse] support BSR for cuSPARSE (libgen path only) (#69646) 2023-10-19 16:56:52 -07:00
Peiming Liu
ff21a90e51
[mlir][sparse] introduce sparse_tensor.crd_translate operation (#69630) 2023-10-19 15:42:09 -07:00
Yinying Li
fb5047f524
[mlir][sparse] Remove old syntax (#69624) 2023-10-19 17:33:28 -04:00
Yinying Li
7b9fb1c228
[mlir][sparse] Update verifier for block sparsity and singleton (#69389)
Updates:
1. Verification of block sparsity.
2. Verification of singleton level type can only follow compressed or
loose_compressed levels. And all level types after singleton should be
singleton.
3. Added getBlockSize function.
4. Added an invalid encoding test for an incorrect lvlToDim map that
user provides.
2023-10-19 12:34:18 -04:00
Aart Bik
f16cb0eade
[mlir][sparse] connect MapRef's lvl2dim with latest AffineMap computation (#69540)
This makes sure

- GEN MAP dim=2 lvl=4
  (d0, d1) -> (d0 floordiv 2, d1 floordiv 2, d0 mod 2, d1 mod 2)
--
  (d0, d1, d2, d3) -> (d0 * 2 + d2, d1 * 2 + d3)

is indeed encoded as

MAP-REF (dim=2, lvl=4) isperm=0
  d2l = [ d0/2 d1/2 d0%2 d1%2 ]
  ld2 = [ l2+2*l0 l3+2*l1 ]
2023-10-18 18:00:56 -07:00
Aart Bik
306f4c306a
[mlir][sparse] implement non-permutation MapRef encoding (#69406)
This enables reading block sparse from file using libgen! (and soon also
direct IR codegen)
2023-10-18 13:01:12 -07:00
Yinying Li
d4088e7d5f
[mlir][sparse] Populate lvlToDim (#68937)
Updates:
1. Infer lvlToDim from dimToLvl
2. Add more tests for block sparsity
3. Finish TODOs related to lvlToDim, including adding lvlToDim to python
binding

Verification of lvlToDim that user provides will be implemented in the
next PR.
2023-10-17 16:09:39 -04:00
Peiming Liu
71c97c735c
[mlir][sparse] avoid tensor to memref conversion in sparse tensor rewri… (#69362)
…ting rules.
2023-10-17 11:34:06 -07:00
Peiming Liu
761c9dd927
[mlir][sparse] implementating stageSparseOpPass as an interface (#69022) 2023-10-17 10:54:44 -07:00