8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
wren romano
|
84cd51bb97 |
[mlir][sparse] Renaming "pointer/index" to "position/coordinate"
The old "pointer/index" names often cause confusion since these names clash with names of unrelated things in MLIR; so this change rectifies this by changing everything to use "position/coordinate" terminology instead.
In addition to the basic terminology, there have also been various conventions for making certain distinctions like: (1) the overall storage for coordinates in the sparse-tensor, vs the particular collection of coordinates of a given element; and (2) particular coordinates given as a `Value` or `TypedValue<MemRefType>`, vs particular coordinates given as `ValueRange` or similar. I have striven to maintain these distinctions
as follows:
* "p/c" are used for individual position/coordinate values, when there is no risk of confusion. (Just like we use "d/l" to abbreviate "dim/lvl".)
* "pos/crd" are used for individual position/coordinate values, when a longer name is helpful to avoid ambiguity or to form compound names (e.g., "parentPos"). (Just like we use "dim/lvl" when we need a longer form of "d/l".)
I have also used these forms for a handful of compound names where the old name had been using a three-letter form previously, even though a longer form would be more appropriate. I've avoided renaming these to use a longer form purely for expediency sake, since changing them would require a cascade of other renamings. They should be updated to follow the new naming scheme, but that can be done in future patches.
* "coords" is used for the complete collection of crd values associated with a single element. In the runtime library this includes both `std::vector` and raw pointer representations. In the compiler, this is used specifically for buffer variables with C++ type `Value`, `TypedValue<MemRefType>`, etc.
The bare form "coords" is discouraged, since it fails to make the dim/lvl distinction; so the compound names "dimCoords/lvlCoords" should be used instead. (Though there may exist a rare few cases where is is appropriate to be intentionally ambiguous about what coordinate-space the coords live in; in which case the bare "coords" is appropriate.)
There is seldom the need for the pos variant of this notion. In most circumstances we use the term "cursor", since the same buffer is reused for a 'moving' pos-collection.
* "dcvs/lcvs" is used in the compiler as the `ValueRange` analogue of "dimCoords/lvlCoords". (The "vs" stands for "`Value`s".) I haven't found the need for it, but "pvs" would be the obvious name for a pos-`ValueRange`.
The old "ind"-vs-"ivs" naming scheme does not seem to have been sustained in more recent code, which instead prefers other mnemonics (e.g., adding "Buf" to the end of the names for `TypeValue<MemRefType>`). I have cleaned up a lot of these to follow the "coords"-vs-"cvs" naming scheme, though haven't done an exhaustive cleanup.
* "positions/coordinates" are used for larger collections of pos/crd values; in particular, these are used when referring to the complete sparse-tensor storage components.
I also prefer to use these unabbreviated names in the documentation, unless there is some specific reason why using the abbreviated forms helps resolve ambiguity.
In addition to making this terminology change, this change also does some cleanup along the way:
* correcting the dim/lvl terminology in certain places.
* adding `const` when it requires no other code changes.
* miscellaneous cleanup that was entailed in order to make the proper distinctions. Most of these are in CodegenUtils.{h,cpp}
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D144773
|
||
|
Markus Böck
|
9048ea28da |
Reland "[mlir] Make the vast majority of intgration and runner tests work on Windows"
This reverts commit 5561e174117ff395d65b6978d04b62c1a1275138 The logic was moved from cmake into lit fixing the issue that lead to the revert and potentially others with multi-config cmake generators Differential Revision: https://reviews.llvm.org/D143925 |
||
|
Aart Bik
|
5561e17411 |
Revert "[mlir] Make the vast majority of integration and runner tests work on Windows"
This reverts commit 161b9d741a3c25f7bd79620598c5a2acf3f0f377. REASON: cmake --build . --target check-mlir-integration Failed Tests (186): MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-addi-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-cmpi-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-compare-results-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-constants-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-max-min-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-muli-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shli-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shrsi-i16.mlir MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shrui-i16.mlir MLIR :: Integration/Dialect/Async/CPU/microbench-linalg-async-parallel-for.mlir MLIR :: Integration/Dialect/Async/CPU/microbench-scf-async-parallel-for.mlir MLIR :: Integration/Dialect/Async/CPU/test-async-parallel-for-1d.mlir MLIR :: Integration/Dialect/Async/CPU/test-async-parallel-for-2d.mlir MLIR :: Integration/Dialect/Complex/CPU/correctness.mlir MLIR :: Integration/Dialect/LLVMIR/CPU/X86/test-inline-asm-vector.mlir MLIR :: Integration/Dialect/LLVMIR/CPU/X86/test-inline-asm.mlir MLIR :: Integration/Dialect/LLVMIR/CPU/test-vector-reductions-fp.mlir MLIR :: Integration/Dialect/LLVMIR/CPU/test-vector-reductions-int.mlir MLIR :: Integration/Dialect/Linalg/CPU/matmul-vs-matvec.mlir MLIR :: Integration/Dialect/Linalg/CPU/rank-reducing-subview.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-collapse-tensor.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-1d-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-1d-nwc-wcf-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-2d-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-2d-nhwc-hwcf-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-3d-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-conv-3d-ndhwc-dhwcf-call.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-elementwise.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-expand-tensor.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-one-shot-bufferize.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-padtensor.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-subtensor-insert-multiple-uses.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-subtensor-insert.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-tensor-e2e.mlir MLIR :: Integration/Dialect/Linalg/CPU/test-tensor-matmul.mlir MLIR :: Integration/Dialect/Memref/cast-runtime-verification.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/concatenate.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output_bf16.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output_f16.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_abs.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_binary.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_cast.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_codegen_dim.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_codegen_foreach.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex32.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex64.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex_ops.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_constant_to_sparse_tensor.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_1d_nwc_wcf.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nhwc_hwcf.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d_ndhwc_dhwcf.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_dyn.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_ptr.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2dense.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2sparse.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_dot.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_expand.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_file_io.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_filter_conv2d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_flatten.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_foreach_slices.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_index.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_index_dense.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_1d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_2d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_3d.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matmul.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matrix_ops.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matvec.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_mttkrp.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_mult_elt.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_reduction.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_simple.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_pack.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_quantized_matmul.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_re_im.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reduce_custom.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reduce_custom_prod.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reductions.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reductions_prod.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reshape.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_push_back.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_sort.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_sort_coo.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sampled_matmul.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sampled_mm_fusion.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_scale.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_scf_nested.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_select.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sign.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sorted_coo.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_spmm.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_storage.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_bf16.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_c32.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_f16.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tanh.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tensor_mul.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tensor_ops.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_transpose.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_unary.mlir MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_vector_ops.mlir MLIR :: Integration/Dialect/SparseTensor/python/test_SDDMM.py MLIR :: Integration/Dialect/SparseTensor/python/test_SpMM.py MLIR :: Integration/Dialect/SparseTensor/python/test_elementwise_add_sparse_output.py MLIR :: Integration/Dialect/SparseTensor/python/test_output.py MLIR :: Integration/Dialect/SparseTensor/python/test_stress.py MLIR :: Integration/Dialect/SparseTensor/taco/test_MTTKRP.py MLIR :: Integration/Dialect/SparseTensor/taco/test_SDDMM.py MLIR :: Integration/Dialect/SparseTensor/taco/test_SpMM.py MLIR :: Integration/Dialect/SparseTensor/taco/test_SpMV.py MLIR :: Integration/Dialect/SparseTensor/taco/test_Tensor.py MLIR :: Integration/Dialect/SparseTensor/taco/test_scalar_tensor_algebra.py MLIR :: Integration/Dialect/SparseTensor/taco/test_simple_tensor_algebra.py MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_complex.py MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_types.py MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_unary_ops.py MLIR :: Integration/Dialect/SparseTensor/taco/test_true_dense_tensor_algebra.py MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_core.py MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_io.py MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_utils.py MLIR :: Integration/Dialect/Standard/CPU/test-ceil-floor-pos-neg.mlir MLIR :: Integration/Dialect/Standard/CPU/test_subview.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-mulf-full.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-mulf.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli-ext.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli-full.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-tilezero-block.mlir MLIR :: Integration/Dialect/Vector/CPU/AMX/test-tilezero.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-dot.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-inline-asm-vector-avx512.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-mask-compress.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-rsqrt.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-sparse-dot-product.mlir MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-vp2intersect-i32.mlir MLIR :: Integration/Dialect/Vector/CPU/test-0-d-vectors.mlir MLIR :: Integration/Dialect/Vector/CPU/test-broadcast.mlir MLIR :: Integration/Dialect/Vector/CPU/test-compress.mlir MLIR :: Integration/Dialect/Vector/CPU/test-constant-mask.mlir MLIR :: Integration/Dialect/Vector/CPU/test-contraction.mlir MLIR :: Integration/Dialect/Vector/CPU/test-create-mask-v4i1.mlir MLIR :: Integration/Dialect/Vector/CPU/test-create-mask.mlir MLIR :: Integration/Dialect/Vector/CPU/test-expand.mlir MLIR :: Integration/Dialect/Vector/CPU/test-extract-strided-slice.mlir MLIR :: Integration/Dialect/Vector/CPU/test-flat-transpose-col.mlir MLIR :: Integration/Dialect/Vector/CPU/test-flat-transpose-row.mlir MLIR :: Integration/Dialect/Vector/CPU/test-fma.mlir MLIR :: Integration/Dialect/Vector/CPU/test-gather.mlir MLIR :: Integration/Dialect/Vector/CPU/test-index-vectors.mlir MLIR :: Integration/Dialect/Vector/CPU/test-insert-strided-slice.mlir MLIR :: Integration/Dialect/Vector/CPU/test-maskedload.mlir MLIR :: Integration/Dialect/Vector/CPU/test-maskedstore.mlir MLIR :: Integration/Dialect/Vector/CPU/test-matrix-multiply-col.mlir MLIR :: Integration/Dialect/Vector/CPU/test-matrix-multiply-row.mlir MLIR :: Integration/Dialect/Vector/CPU/test-outerproduct-f32.mlir MLIR :: Integration/Dialect/Vector/CPU/test-outerproduct-i64.mlir MLIR :: Integration/Dialect/Vector/CPU/test-print-int.mlir MLIR :: Integration/Dialect/Vector/CPU/test-realloc.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f32-reassoc.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f32.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f64-reassoc.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f64.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i32.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i4.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i64.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-si4.mlir MLIR :: Integration/Dialect/Vector/CPU/test-reductions-ui4.mlir MLIR :: Integration/Dialect/Vector/CPU/test-scan.mlir MLIR :: Integration/Dialect/Vector/CPU/test-scatter.mlir MLIR :: Integration/Dialect/Vector/CPU/test-shape-cast.mlir MLIR :: Integration/Dialect/Vector/CPU/test-shuffle.mlir MLIR :: Integration/Dialect/Vector/CPU/test-sparse-dot-matvec.mlir MLIR :: Integration/Dialect/Vector/CPU/test-sparse-saxpy-jagged-matvec.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-1d.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-2d.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-3d.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-to-loops.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transfer-write.mlir MLIR :: Integration/Dialect/Vector/CPU/test-transpose.mlir Testing Time: 0.29s Unsupported: 31 Passed : 5 Failed : 186 Differential Revision: https://reviews.llvm.org/D143970 |
||
|
Markus Böck
|
161b9d741a |
[mlir] Make the vast majority of integration and runner tests work on Windows
This patch contains the changes required to make the vast majority of integration and runner tests run on Windows. Historically speaking, the JIT support for Windows has been lacking behind, but recent versions of ORC JIT have now caught up and works for basically all examples in repo. Sadly due to these tests previously not working on Windows, basically all of them are making unix-like assumptions about things like filenames, paths, shell syntax etc. This patch fixes all these issues in one big swoop and enables Windows support for the vast majority of integration tests. More specifically, following changes had to be done: * The various JIT runners used paths to the runtime libraries that assumed a Unix toolchain layout and filenames. I abstracted the specific path and filename of these runtime libraries away by making the paths to the runtime libraries be passed from cmake into lit. This now also allows a much more convenient syntax: `--shared-libs=%mlir_c_runner_utils` instead of `--shared-libs=%mlir_lib_dir/lib/libmlir_c_runner_utils%shlibext` * Some tests using python set environment variables using the `ENV=VALUE cmd` format. This works on Unix, but on Windows it has to prefixed using `env ENV=VALUE cmd` * Some tests used C functions that are simply not available or exported on Windows (`fabsf`, `aligned_alloc`). These tests have either been adjusted or explicitly marked as `UNSUPPORTED` Some tests remain disabled on Windows as before: * In SparseTensor some tests have non-trivial logic for finding the runtime libraries which seems to be required for the use of emulators. I do not have the time to port these so I simply kept them disabled * Some tests requiring special hardware which I simply cannot test remain disabled on Windows. These include usage of AVX512 or AMX The tests for `mlir-vulkan-runner` and `mlir-spirv-runner` all work now as well and so do the vast majority of `mlir-cpu-runner`. Differential Revision: https://reviews.llvm.org/D143925 |
||
|
Peiming Liu
|
dc6427d687 |
[mlir][sparse] implement lowering rules for sparse_tensor::unpack
Reviewed By: aartbik Differential Revision: https://reviews.llvm.org/D143672 |
||
|
Andrzej Warzynski
|
bfe4ce3f83 |
[mlir][sparse] Port the remaining integration tests to use SVE
This patch updates the remaining SparseCompiler integration tests to target SVE when available. Two tests will require some investigation in the future: * sparse_matmul.mlir * sparse_tanh.mlir The former passes regardless - that's due to how `CHECK` lines are defined. The latter fails when SVE is enabled, but passes when it's disabled. I marked it as UNSUPPORTED as there is no mechanism to XFAIL a test conditionally. Also, see [1] for more details. [1] https://github.com/llvm/llvm-project/issues/60626 Differential Revision: https://reviews.llvm.org/D143514 |
||
|
Peiming Liu
|
7a8edea69d |
[mlir][sparse] fix bug when packing tensor with 32 bit pointer width.
Reviewed By: wrengr Differential Revision: https://reviews.llvm.org/D143450 |
||
|
Peiming Liu
|
a41672e16a |
[mlir][sparse] implement lowering rules for sparse_tensor.pack operation
Reviewed By: aartbik Differential Revision: https://reviews.llvm.org/D143230 |