This patch disables 2 bf16 tests that are currently not supported on
AArch64. I've triaged these failures and opened [1] to track this. I
don't have a simple reproducer for dense_output_bf16.mlir, but it's
rather clear that both tests fail due to missing support for `bfloat`
operations in the AArch64 backend.
I'm not sure what the path forward to enable these tests on AArch64
should be. I think that there are two options:
* AArch64 backened gains capability to legalize these nodes containing
`bfloat` operands, or
* MLIR (similarly to Clang) is taught not to emit such nodes in the
first place.
[1] https://github.com/llvm/llvm-project/issues/58465
Differential Revision: https://reviews.llvm.org/D136273
Context: https://discourse.llvm.org/t/psa-retire-linalg-filter-based-patterns/63785
Uses of `LinalgTilingPattern::returningMatchAndRewrite` are replaced by a top-level `tileWithLinalgTilingOptions` function that is marked obsolete and serves
as a temporary means to transition away from `LinalgTilingOptions`-based tiling.
LinalgTilingOptions supports too many options that have been orthogonalized with the use of the transform dialect.
Additionally, the revision introduces a `transform.structured.tile_to_scf_for` structured transform operation that is needed to properly tile `tensor.pad`
via the TilingInterface. Uses of `transform.structured.tile` will be deprecated and replaced by this new op.
This will achieve the deprecation of `linalg::tileLinalgOp`.
Context: https://discourse.llvm.org/t/psa-retire-tileandfuselinalgops-method/63850
In the process of transitioning, tests that were performing tile and distribute on tensors are retired: transformations should be orthogonalized better in the future.
In particular, tiling to specific loop types and tileAndDistribute behavior are not available via the transform ops.
The behavior is still available as part of the `tileWithLinalgTilingOptions` method to allow downstream clients to transition without breakages but is meant to be retired soon.
As more tests are ported to the transform dialect, it became necessary to introduce a test-transform-dialect-erase-schedule-pass to discard the transform specification
once applied so that e2e lowering and execution is possible.
Lastly, a number of redundant tests that were testing composition of patterns are retired as they are available with a better mechanism via the transform dialect.
Differential Revision: https://reviews.llvm.org/D135573
Handle more cases of singleton DLT including direct sparse2sparse conversion. (Followup to D134096)
Depends On D134926
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D134933
This extension to the sparse tensor type system in MLIR
opens up a whole new set of sparse storage schemes, such as
block sparse storage (e.g. BCSR) and ELL (aka jagged diagonals).
This revision merely introduces the type extension and
initial documentation. The actual interpretation of the type
(reading in tensors, lowering to code, etc.) will follow.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D135206
tensor.empty/linalg.init_tensor produces an uninititalized tensor that can be used as a destination operand for destination-style ops (ops that implement `DestinationStyleOpInterface`).
This change makes it possible to implement `TilingInterface` for non-destination-style ops without depending on the Linalg dialect.
RFC: https://discourse.llvm.org/t/rfc-add-tensor-from-shape-operation/65101
Differential Revision: https://reviews.llvm.org/D135129
The region within sparse_tensor.select is used as the runtime criteria
for whether to keep the existing value in the sparse tensor.
While the sparse element is provided to the comparison, indices may also
be used to decide on whether to keep the original value. This allows, for
example, to only keep the upper triangle of a matrix.
Reviewed by: aartbik
Differential Revision: https://reviews.llvm.org/D134761
This is a first step towards fully implementing the new dimension
level types and properties, illustrating with a fully functional
sorted COO of any dimension. Note that the sparsification part is
pretty complete. The required parts in the runtime support library
have been kept to a minimum, to avoid huge conflicts with Wren's
ongoing refactoring. The missing parts will be filled in later.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D134096
Add sparse-buffer-rewrite pass to rewrite sparse primitives on buffers to MLIR
implementation.
Add sparse rewrite rule for the sort operator.
Add FileCheck test and integration test.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D134627
Add memref.realloc and canonicalization of the op. Add conversion patterns for
lowering the op to LLVM using unaligned alloc or aligned alloc based on the
conversion option.
Add filecheck tests for parsing and converting the op. Add an integration test.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D133424
This includes tests with the exact expected values and comparison-based tests.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D134321
Introduces a simple framework for runtime tests of the wide integer emulation.
In these tests, we are only interested in checking that both wide and narrow calculation
produce the same results, and do not check for exact results. This allows us to cover
more of the input space, as we do not have to hardcode each of the expected outputs.
Introduce common helper functions to check the results, print a message on mismatch,
and sample the input space.
Implement runtime comparrison tests for `arith.muli` and `arith.shrui`.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D134184
The new test cases focus on known edge cases in the current implementation.
Specifically, we check for low (0, 1), mid (7, 8, 9) and high (15) shift amounts with i16 operands.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D134182
The new test pass allows for running wide integer emulation conversion
within specified functions only.
I intend to use it in integration tests in a way that allows me print both
original and emulated results in the same format, or even compare both results
at runtime and print on mismatch only.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D134120
All relevant operations have been switched to primarily use the strided
layout, but still support the affine map layout. Update the relevant
tests to use the strided format instead for compatibility with how ops
now print by default.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D134045
Emulate multiplication by splitting each input element of type i2N into 4
digits of type iN and bit width i(N/2). This is so that the intermediate
multiplications and additions do not overflow. We extract these i(N/2)
digits from iN vector elements by masking (low digit) and shifting right
(high digit).
The multiplication algorithm used is the standard (long) multiplication.
Multiplying two i2N integers produces (at most) a i4N result, but because
the calculation of top i2N is not necessary, we omit it.
In total, this implementations performs 10 intermediate multiplications
and 16 additions. The number of multiplications could be decreased by
switching to a more efficient algorithm like Karatsuba. This would,
however, require being able to perform (intermediate) wide additions and
subtractions, so it is not clear that such implementation would be more
efficient.
I tested this on all 16-bit inut pairs, when emulating i16 with i8.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D133629
The goal is to have a set of runtime tests for further extercise the
wide integer emulation pass and its conversion patterns. This was
suggested by @Mogball in D133629.
Add a minimal runtime test to demonstrate that printing and pass
pipeline works as expected.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D134004
Previously, the iteration graph is computed without priority. This patch add a heuristic when computing the iteration graph by starting with Reduction iterator when doing topo sort, which makes Reduction iterators (likely) appear as late in the sorted array as possible.
The current sparse compiler also failed to compile the newly added case.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D133738
Add new option (enable-runtime-library) to sparse compiler pipeline, it allows us to decide whether we need to rewrite operations (e.g., concatenate, reshape) within sparsification (when using codegen) or convert them after sparsification (when using runtime library).
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D133597
Access pattern expansion is always done along the innermost stored
dimension, but this was incorrectly reordered due to using a
general utility typically used by original dimensions only.
Reviewed By: bixia
Differential Revision: https://reviews.llvm.org/D133472
The "sparsification" pass does not need the ability to use runtime values for
the dimension, so the only source for variability would have been user code.
Restricting the dimension to constants simplifies code generation.
Reviewed By: Peiming, wrengr
Differential Revision: https://reviews.llvm.org/D133458
Replace the following config attributes with `mlir_lib_dir`:
- `mlir_runner_utils_dir`
- `linalg_test_lib_dir`
- `spirv_wrapper_library_dir`
- `vulkan_wrapper_library_dir`
- `mlir_integration_test_dir`
I'm going to clean up substitutions in separate changes.
Reviewed By: aartbik, mehdi_amini
Differential Revision: https://reviews.llvm.org/D133217
