Markus Böck
9048ea28da
Reland "[mlir] Make the vast majority of intgration and runner tests work on Windows"
...
This reverts commit 5561e174117ff395d65b6978d04b62c1a1275138
The logic was moved from cmake into lit fixing the issue that lead to the revert and potentially others with multi-config cmake generators
Differential Revision: https://reviews.llvm.org/D143925
2023-02-15 19:14:43 +01:00
Aart Bik
5561e17411
Revert "[mlir] Make the vast majority of integration and runner tests work on Windows"
...
This reverts commit 161b9d741a3c25f7bd79620598c5a2acf3f0f377.
REASON:
cmake --build . --target check-mlir-integration
Failed Tests (186):
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-addi-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-cmpi-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-compare-results-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-constants-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-max-min-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-muli-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shli-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shrsi-i16.mlir
MLIR :: Integration/Dialect/Arith/CPU/test-wide-int-emulation-shrui-i16.mlir
MLIR :: Integration/Dialect/Async/CPU/microbench-linalg-async-parallel-for.mlir
MLIR :: Integration/Dialect/Async/CPU/microbench-scf-async-parallel-for.mlir
MLIR :: Integration/Dialect/Async/CPU/test-async-parallel-for-1d.mlir
MLIR :: Integration/Dialect/Async/CPU/test-async-parallel-for-2d.mlir
MLIR :: Integration/Dialect/Complex/CPU/correctness.mlir
MLIR :: Integration/Dialect/LLVMIR/CPU/X86/test-inline-asm-vector.mlir
MLIR :: Integration/Dialect/LLVMIR/CPU/X86/test-inline-asm.mlir
MLIR :: Integration/Dialect/LLVMIR/CPU/test-vector-reductions-fp.mlir
MLIR :: Integration/Dialect/LLVMIR/CPU/test-vector-reductions-int.mlir
MLIR :: Integration/Dialect/Linalg/CPU/matmul-vs-matvec.mlir
MLIR :: Integration/Dialect/Linalg/CPU/rank-reducing-subview.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-collapse-tensor.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-1d-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-1d-nwc-wcf-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-2d-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-2d-nhwc-hwcf-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-3d-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-conv-3d-ndhwc-dhwcf-call.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-elementwise.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-expand-tensor.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-one-shot-bufferize.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-padtensor.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-subtensor-insert-multiple-uses.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-subtensor-insert.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-tensor-e2e.mlir
MLIR :: Integration/Dialect/Linalg/CPU/test-tensor-matmul.mlir
MLIR :: Integration/Dialect/Memref/cast-runtime-verification.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/concatenate.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output_bf16.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/dense_output_f16.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_abs.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_binary.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_cast.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_codegen_dim.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_codegen_foreach.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex32.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex64.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_complex_ops.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_constant_to_sparse_tensor.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_1d_nwc_wcf.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_2d_nhwc_hwcf.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conv_3d_ndhwc_dhwcf.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_dyn.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_ptr.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2dense.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_conversion_sparse2sparse.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_dot.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_expand.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_file_io.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_filter_conv2d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_flatten.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_foreach_slices.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_index.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_index_dense.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_1d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_2d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_insert_3d.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matmul.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matrix_ops.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_matvec.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_mttkrp.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_mult_elt.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_reduction.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_out_simple.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_pack.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_quantized_matmul.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_re_im.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reduce_custom.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reduce_custom_prod.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reductions.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reductions_prod.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_reshape.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_push_back.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_sort.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_rewrite_sort_coo.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sampled_matmul.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sampled_mm_fusion.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_scale.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_scf_nested.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_select.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sign.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sorted_coo.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_spmm.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_storage.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_bf16.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_c32.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_sum_f16.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tanh.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tensor_mul.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_tensor_ops.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_transpose.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_unary.mlir
MLIR :: Integration/Dialect/SparseTensor/CPU/sparse_vector_ops.mlir
MLIR :: Integration/Dialect/SparseTensor/python/test_SDDMM.py
MLIR :: Integration/Dialect/SparseTensor/python/test_SpMM.py
MLIR :: Integration/Dialect/SparseTensor/python/test_elementwise_add_sparse_output.py
MLIR :: Integration/Dialect/SparseTensor/python/test_output.py
MLIR :: Integration/Dialect/SparseTensor/python/test_stress.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_MTTKRP.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_SDDMM.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_SpMM.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_SpMV.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_Tensor.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_scalar_tensor_algebra.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_simple_tensor_algebra.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_complex.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_types.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_tensor_unary_ops.py
MLIR :: Integration/Dialect/SparseTensor/taco/test_true_dense_tensor_algebra.py
MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_core.py
MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_io.py
MLIR :: Integration/Dialect/SparseTensor/taco/unit_test_tensor_utils.py
MLIR :: Integration/Dialect/Standard/CPU/test-ceil-floor-pos-neg.mlir
MLIR :: Integration/Dialect/Standard/CPU/test_subview.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-mulf-full.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-mulf.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli-ext.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli-full.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-muli.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-tilezero-block.mlir
MLIR :: Integration/Dialect/Vector/CPU/AMX/test-tilezero.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-dot.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-inline-asm-vector-avx512.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-mask-compress.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-rsqrt.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-sparse-dot-product.mlir
MLIR :: Integration/Dialect/Vector/CPU/X86Vector/test-vp2intersect-i32.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-0-d-vectors.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-broadcast.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-compress.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-constant-mask.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-contraction.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-create-mask-v4i1.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-create-mask.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-expand.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-extract-strided-slice.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-flat-transpose-col.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-flat-transpose-row.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-fma.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-gather.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-index-vectors.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-insert-strided-slice.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-maskedload.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-maskedstore.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-matrix-multiply-col.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-matrix-multiply-row.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-outerproduct-f32.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-outerproduct-i64.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-print-int.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-realloc.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f32-reassoc.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f32.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f64-reassoc.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-f64.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i32.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i4.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-i64.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-si4.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-reductions-ui4.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-scan.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-scatter.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-shape-cast.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-shuffle.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-sparse-dot-matvec.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-sparse-saxpy-jagged-matvec.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-1d.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-2d.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read-3d.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-read.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-to-loops.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transfer-write.mlir
MLIR :: Integration/Dialect/Vector/CPU/test-transpose.mlir
Testing Time: 0.29s
Unsupported: 31
Passed : 5
Failed : 186
Differential Revision: https://reviews.llvm.org/D143970
2023-02-13 18:30:52 -08:00
Markus Böck
161b9d741a
[mlir] Make the vast majority of integration and runner tests work on Windows
...
This patch contains the changes required to make the vast majority of integration and runner tests run on Windows.
Historically speaking, the JIT support for Windows has been lacking behind, but recent versions of ORC JIT have now caught up and works for basically all examples in repo.
Sadly due to these tests previously not working on Windows, basically all of them are making unix-like assumptions about things like filenames, paths, shell syntax etc.
This patch fixes all these issues in one big swoop and enables Windows support for the vast majority of integration tests.
More specifically, following changes had to be done:
* The various JIT runners used paths to the runtime libraries that assumed a Unix toolchain layout and filenames. I abstracted the specific path and filename of these runtime libraries away by making the paths to the runtime libraries be passed from cmake into lit. This now also allows a much more convenient syntax: `--shared-libs=%mlir_c_runner_utils` instead of `--shared-libs=%mlir_lib_dir/lib/libmlir_c_runner_utils%shlibext`
* Some tests using python set environment variables using the `ENV=VALUE cmd` format. This works on Unix, but on Windows it has to prefixed using `env ENV=VALUE cmd`
* Some tests used C functions that are simply not available or exported on Windows (`fabsf`, `aligned_alloc`). These tests have either been adjusted or explicitly marked as `UNSUPPORTED`
Some tests remain disabled on Windows as before:
* In SparseTensor some tests have non-trivial logic for finding the runtime libraries which seems to be required for the use of emulators. I do not have the time to port these so I simply kept them disabled
* Some tests requiring special hardware which I simply cannot test remain disabled on Windows. These include usage of AVX512 or AMX
The tests for `mlir-vulkan-runner` and `mlir-spirv-runner` all work now as well and so do the vast majority of `mlir-cpu-runner`.
Differential Revision: https://reviews.llvm.org/D143925
2023-02-13 22:24:20 +01:00
Aart Bik
c48e90877f
[mlir][sparse] introduce a higher-order tensor mapping
...
This extension to the sparse tensor type system in MLIR
opens up a whole new set of sparse storage schemes, such as
block sparse storage (e.g. BCSR) and ELL (aka jagged diagonals).
This revision merely introduces the type extension and
initial documentation. The actual interpretation of the type
(reading in tensors, lowering to code, etc.) will follow.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D135206
2022-10-05 09:40:51 -07:00
Christian Sigg
0f2ec35691
[MLIR] Switch lit tests to %mlir_lib_dir and %mlir_src_dir replacements.
...
The old replacements will be removed soon:
- `%linalg_test_lib_dir`
- `%cuda_wrapper_library_dir`
- `%spirv_wrapper_library_dir`
- `%vulkan_wrapper_library_dir`
- `%mlir_runner_utils_dir`
- `%mlir_integration_test_dir`
Reviewed By: herhut
Differential Revision: https://reviews.llvm.org/D133270
2022-09-06 12:34:14 +02:00
River Riddle
2310ced874
[mlir][NFC] Update textual references of func to func.func in examples+python scripts
...
The special case parsing of `func` operations is being removed.
2022-04-20 22:17:26 -07:00
Aart Bik
28063a281b
[mlir][sparse] refactored python setup of sparse compiler
...
Reviewed By: bixia
Differential Revision: https://reviews.llvm.org/D123419
2022-04-12 11:58:41 -07:00
Aart Bik
8b83b8f131
[mlir][sparse] refactor sparse compiler pipeline to single place
...
Reviewed By: bixia
Differential Revision: https://reviews.llvm.org/D120347
2022-02-22 16:23:56 -08:00
Aart Bik
515c617003
[mlir][linalg][sparse] add linalg optimization passes "upstream"
...
It is time to compose Linalg related optimizations with SparseTensor
related optimizations. This is a careful first start by adding some
general Linalg optimizations "upstream" of the sparse compiler in the
full sparse compiler pipeline. Some minor changes were needed to make
those optimizations aware of sparsity.
Note that after this, we will add a sparse specific fusion rule,
just to demonstrate the power of the new composition.
Reviewed By: bixia
Differential Revision: https://reviews.llvm.org/D119971
2022-02-17 08:55:50 -08:00
wren romano
4998b1a6cd
[mlir][sparse] Updating sparse-compiler pipeline for python usage
...
Explicitly nests passes for FuncOp, adds more options to the sparse-compiler pipeline, and updates python integration tests. This should be sufficient to close https://github.com/llvm/llvm-project/issues/51751
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D118658
2022-02-04 11:39:48 -08:00
River Riddle
dec8af701f
[mlir] Move SelectOp from Standard to Arithmetic
...
This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.
Differential Revision: https://reviews.llvm.org/D118648
2022-02-02 14:45:12 -08:00
Matthias Springer
ab47418df6
[mlir][bufferize] Merge tensor-constant-bufferize into arith-bufferize
...
The bufferization of arith.constant ops is also switched over to BufferizableOpInterface-based bufferization. The old implementation is deleted. Both implementations utilize GlobalCreator, now renamed to just `getGlobalFor`.
GlobalCreator no longer maintains a set of all created allocations to avoid duplicate allocations of the same constant. Instead, `getGlobalFor` scans the module to see if there is already a global allocation with the same constant value.
For compatibility reasons, it is still possible to create a pass that bufferizes only `arith.constant`. This pass (createConstantBufferizePass) could be deleted once all users were switched over to One-Shot bufferization.
Differential Revision: https://reviews.llvm.org/D118483
2022-01-30 21:37:48 +09:00
Aart Bik
92c1c63dae
[mlir][sparse] integration test for sparse output operation
...
Reviewed By: bixia
Differential Revision: https://reviews.llvm.org/D118079
2022-01-24 17:41:09 -08:00
