This commit ensures the validation pass is not run on operations from
other dialects. In doing so, operations from other dialects that, for
example, use types not supported by TOSA don't result in an error.
Signed-off-by: Luke Hutton <luke.hutton@arm.com>
Relands #118583, with a fix for Python 3.8 compatibility. It was not
possible to set the buffer protocol accessers via slots in Python 3.8.
Why? https://nanobind.readthedocs.io/en/latest/why.html says it better
than I can, but my primary motivation for this change is to improve MLIR
IR construction time from JAX.
For a complicated Google-internal LLM model in JAX, this change improves
the MLIR
lowering time by around 5s (out of around 30s), which is a significant
speedup for simply switching binding frameworks.
To a large extent, this is a mechanical change, for instance changing
`pybind11::` to `nanobind::`.
Notes:
* this PR needs Nanobind 2.4.0, because it needs a bug fix
(https://github.com/wjakob/nanobind/pull/806) that landed in that
release.
* this PR does not port the in-tree dialect extension modules. They can
be ported in a future PR.
* I removed the py::sibling() annotations from def_static and def_class
in `PybindAdapters.h`. These ask pybind11 to try to form an overload
with an existing method, but it's not possible to form mixed
pybind11/nanobind overloads this ways and the parent class is now
defined in nanobind. Better solutions may be possible here.
* nanobind does not contain an exact equivalent of pybind11's buffer
protocol support. It was not hard to add a nanobind implementation of a
similar API.
* nanobind is pickier about casting to std::vector<bool>, expecting that
the input is a sequence of bool types, not truthy values. In a couple of
places I added code to support truthy values during casting.
* nanobind distinguishes bytes (`nb::bytes`) from strings (e.g.,
`std::string`). This required nb::bytes overloads in a few places.
Give the properties from tablegen a `predicate` field that holds the
predicate that the property needs to satisfy, if one exists, and hook
that field up to verifier generation.
In many cases the emptyTensorElimination can not transform or eliminate
the empty tensor which is being inserted into the
`SubsetInsertionOpInterface`.
Two major reasons for that:
1- Failing when trying to find a legal/suitable insertion point for the
`subsetExtract` which is about to replace the empty tensor. However, we
may try to handle this issue by moving the needed values which
responsible on building the `subsetExtract` nearby the empty tensor
(which is about to be eliminated). Thus increasing the probability to
find a legal insertion point.
2-The EmptyTensorElimination transform replaces the tensor.empty's uses
all at once in one apply, rather than replacing only the specific use
which was visited in the use-def chain (when traversing from the
tensor.insert_slice). This scenario of replacing all the uses of the
tensor.empty may lead into additional read effects after bufferization
of the specific subset extract/subview which should not be the case.
Both cases may result in many copies in the coming bufferization which
can not be canonicalized.
The first case can be noticed when having a `tensor.empty` followed by
`SubsetInsertionOpInterface` (or in simple words `tensor.insert_slice`),
which have been lowered from `tensor/tosa.concat`.
The second case can be noticed when having a `tensor.empty`, with many
uses and leading to applying the transformation only once, since the
whole uses have been replaced at once.
The first commit in the PR only adds the lit tests for the cases shown
above (NFC), to emphasize how the transform works, in the coming MRs
will upload a slight changes to handle these case.
The second commit in this PR, we want to replace only the specific use
which was visited in the `use-def` chain (when traversing from the
`tensor.insert_slice`'s source).
Why? https://nanobind.readthedocs.io/en/latest/why.html says it better
than I can, but my primary motivation for this change is to improve MLIR
IR construction time from JAX.
For a complicated Google-internal LLM model in JAX, this change improves
the MLIR
lowering time by around 5s (out of around 30s), which is a significant
speedup for simply switching binding frameworks.
To a large extent, this is a mechanical change, for instance changing
`pybind11::`
to `nanobind::`.
Notes:
* this PR needs Nanobind 2.4.0, because it needs a bug fix
(https://github.com/wjakob/nanobind/pull/806) that landed in that
release.
* this PR does not port the in-tree dialect extension modules. They can
be ported in a future PR.
* I removed the py::sibling() annotations from def_static and def_class
in `PybindAdapters.h`. These ask pybind11 to try to form an overload
with an existing method, but it's not possible to form mixed
pybind11/nanobind overloads this ways and the parent class is now
defined in nanobind. Better solutions may be possible here.
* nanobind does not contain an exact equivalent of pybind11's buffer
protocol support. It was not hard to add a nanobind implementation of a
similar API.
* nanobind is pickier about casting to std::vector<bool>, expecting that
the input is a sequence of bool types, not truthy values. In a couple of
places I added code to support truthy values during casting.
* nanobind distinguishes bytes (`nb::bytes`) from strings (e.g.,
`std::string`). This required nb::bytes overloads in a few places.
This patch unifies the tiling implementation for tileUsingFor and
tileReductionUsingFor. This is done by passing an addition option to
SCFTilingOptions, allowing it to set how reduction dimensions should be
tiled. Currently, there are 3 different options for reduction tiling:
FullReduction (old tileUsingFor), PartialReductionOuterReduction (old
tileReductionUsingFor) and PartialReductionOuterParallel
(linalg::tileReductionUsingForall, this isn't implemented in this
patch).
The patch makes tileReductionUsingFor use the tileUsingFor
implementation with the new reduction tiling options.
There are no test changes because the implementation was doing almost
the exactly same thing. This was also tested in IREE (which uses both
these APIs heavily) and there were no test changes.
This commit makes the following changes:
1. Previously certain pipeline options could cause the options parser to
get stuck in an an infinite loop. An example is:
```
mlir-opt %s -verify-each=false
-pass-pipeline='builtin.module(func.func(test-options-super-pass{list={list=1,2},{list=3,4}}))''
```
In this example, the 'list' option of the `test-options-super-pass`
is itself a pass options specification (this capability was added in
https://github.com/llvm/llvm-project/issues/101118).
However, while the textual format allows `ListOption<int>` to be given
as `list=1,2,3`, it did not allow the same format for
`ListOption<T>` when T is a subclass of `PassOptions` without extra
enclosing `{....}`. Lack of enclosing `{...}` would cause the infinite
looping in the parser.
This change resolves the parser bug and also allows omitting the
outer `{...}` for `ListOption`-of-options.
2. Previously, if you specified a default list value for your
`ListOption`, e.g. `ListOption<int> opt{*this, "list",
llvm:🆑:list_init({1,2,3})}`,
it would be impossible to override that default value of `{1,2,3}` with
an *empty* list on the command line, since `my-pass{list=}` was not
allowed.
This was not allowed because of ambiguous handling of lists-of-strings
(no literal marker is currently required).
This change makes it explicit in the ListOption construction that we
would like to treat all ListOption as having a default value of "empty"
unless otherwise specified (e.g. using `llvm::list_init`).
It removes the requirement that lists are not printed if empty. Instead,
lists are not printed if they do not have their default value.
It is now clarified that the textual format
`my-pass{string-list=""}` or `my-pass{string-list={}}`
is interpreted as "empty list". This makes it imposssible to specify
that ListOption `string-list` should be a size-1 list containing the
empty string. However, `my-pass{string-list={"",""}}` *does* specify
a size-2 list containing the empty string. This behavior seems
preferable
to allow for overriding non-empty defaults as described above.
The `parseX()` functions that are defined to support `custom<X>` in
`assemblyFormat` should return `ParseResult` rather than
`LogicalResult`. The `ParseResult` type is necessary due to tablegen
generating code that expects this type within an Op `parseX()` function.
This PR adds an `AsmPrinter` option `-mlir-use-nameloc-as-prefix` which
uses trailing `NameLoc`s, if the source IR provides them, as prefixes
when printing SSA IDs.
Clean up `populateVectorToLLVMConversionPatterns` so that it populates
only conversion patterns. All rewrite patterns that do not lower to LLVM
should be populated into a separate greedy pattern rewrite.
The current combination of rewrite patterns and conversion patterns
triggered an edge case when merging the 1:1 and 1:N dialect conversions.
Depends on #119973.
The mask materialization patterns during `VectorToLLVM` are rewrite
patterns. They should run as part of the greedy pattern rewrite and not
the dialect conversion. (Rewrite patterns and conversion patterns are
not generally compatible.)
The current combination of rewrite patterns and conversion patterns
triggered an edge case when merging the 1:1 and 1:N dialect conversions.
- Extend the SPIRV::LowerABIAttributesPass to detect when the target env
is using SPIR-V ver >= 1.4, and in this case add all the functions'
interface storage variables to the spirv.EntryPoint calls, as required
by the spec of OpEntryPoint:
"_Before version 1.4, the interface’s storage classes are limited to the
Input and Output storage classes. Starting with version 1.4, the
interface’s storage classes are all storage classes used in declaring
all global variables referenced by the entry point’s call tree_."
- Fix: generate the replacement ops (spirv.AddressOf and .AccessChain)
in the order in which the associated variable appears in the function
signature
Signed-off-by: Fabrizio Indirli <Fabrizio.Indirli@arm.com>
This is a follow-up of #117246.
I thought then it would be easy to edit a DictionaryAttr but it turns
out that these attributes are immutable and need to be passed during the
construction of the gpu.binary Op.
The first commit was using the NVVMTargetAttr to pass the information.
After feedback from @fabianmcg, this PR now passes the information
through a new option of the gpu-module-to-binary pass.
Please add reviewers, as you see fit.
When running `convert-to-llvm`, `ceildiv` and `floordiv` ops, which do not
have direct llvm conversion pattern, would not get lowered to llvm
dialect. This patch adds CeilFloorDivExpandOpsPatterns to both
`convert-to-llvm` and `arith-to-llvm` (deprecated) lowering those ops to
lower level arith ops which can be lowered to llvm using LLVM
conversion.
Reland of https://github.com/llvm/llvm-project/pull/117305 after
buildbot failures.
See:
https://lab.llvm.org/buildbot/#/builders/80/builds/7168https://lab.llvm.org/buildbot/#/builders/130/builds/7036https://lab.llvm.org/buildbot/#/builders/138/builds/7290
Added dependence to ArithTransforms in ArithToLLVM. In previous
discussion, it has been suggested to move the
CeilFloorDivExpandOpsPatterns to ArithUtils but I think linking
ArithTransforms makes more sense as otherwise :
* ArithToLLVM needs a new dependency to ArithUtils
* ArithUtils needs new dependency to ArithTransforms or move the
patterns as well which will create more dependencies
* It creates lots of code motion which makes it hard to review.
The printer for prop-dict would elide properties that had their default
value, such as optional properties that were not present. The parser
would similarly not raise an error if such a key was missing. However,
after not raising an error, the parser would attempt to convert the null
attribute to a property anyway, causing failures.
This commit fixes the issue and adds tests.
This allows for inlining to be somewhat controlled by the user instead
of always inlining everything. External heuristics may be used to place
`no_inline` attributes on invidiual calls or functions to prevent
inlining.
In the transform dialect use-after-free chcker pass, account for pattern
descriptor operations that intentionally have no declared side effects.
They are not destroying any handles.
Closes#118761.
Update integer range narrowing to handle negative values.
The previous restriction to only narrowing known-non-negative values
wasn't needed, as both the signed and unsigned ranges represent bounds
on the values of each variable in the program ... except that one might
be more accurate than the other. So, if either the signed or unsigned
interpretetation of the inputs and outputs allows for integer narrowing,
the narrowing is permitted.
This commit also updates the integer optimization rewrites to preserve
the stae of constant-like operations and those that are narrowed so that
rewrites of other operations don't lose that range information.
* Move the negative tests from nvvmir.mlir to nvvm-invalid.mlir. With
this, all the error-handling tests are moved to the nvvm-invalid.mlir file.
* Move the tma_prefetch tests to a separate file, as there are many
tests, and fix the FileCheck prefix for these.
* Since undef is discouraged, we use an 'i64 0' as the placeholder value
for cache-hint when unused.
Signed-off-by: Durgadoss R <durgadossr@nvidia.com>
Constrains Vector lowering to apply boundary checks only to data
transfers operating on block shapes.
This further aligns lowering with the current Xe instructions'
restrictions.
A complex multiplication should lower simply to the familiar 4 real
multiplications, 1 real addition, 1 real subtraction. No special-casing
of infinite or NaN values should be made, instead the complex numbers
should be thought as just vectors of two reals, naturally bottoming out
on the reals' semantics, IEEE754 or otherwise. That is what nearly
everybody else is doing ("nearly" because at the end of this PR
description we pinpoint the actual source of this in C99 `_Complex`),
and this pattern, by trying to do something different, was generating
much larger code, which was much slower and a departure from the
naturally expected floating-point behavior.
This code had originally been introduced in
https://reviews.llvm.org/D105270, which stated this rationale:
> The lowering handles special cases with NaN or infinity like C++.
I don't think that the C++ standard is a particularly important thing to
follow in this instance. What matters more is what people actually do in
practice with complex numbers, which rarely involves the C++
`std::complex` library type.
But out of curiosity, I checked, and the above statement seems
incorrect. The [current C++
standard](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/n4928.pdf)
library specification for `std::complex` does not say anything about the
implementation of complex multiplication: paragraph `[complex.ops]`
falls back on `[complex.member.ops]` which says:
> Effects: Multiplies the complex value rhs by the complex value *this
and stores the product in *this.
I also checked cppreference which often has useful information in case
something changed in a c++ language revision, but likewise, nothing at
all there:
https://en.cppreference.com/w/cpp/numeric/complex/operator_arith3
Finally, I checked in Compiler Explorer what Clang 19 currently
generates:
https://godbolt.org/z/oY7Ks4j95
That is just the familiar 4 multiplications.... and then there is some
weird check (`fcmp`) and conditionally a call to an external `__mulsc3`.
Googled that, found this StackOverflow answer:
https://stackoverflow.com/a/49438578
Summary: this is not about C++ (this post confirms my reading of the C++
standard not mandating anything about this). This is about C, and it
just happens that this C++ standard library implementation bottoms out
on code shared with the C `_Complex` implementation.
Another nuance missing in that SO answer: this is actually
[implementation-defined
behavior](https://en.cppreference.com/w/c/preprocessor/impl). There are
two modes, controlled by
```c
#pragma STDC CX_LIMITED_RANGE {ON,OFF,DEFAULT}
```
It is implementation-defined which is the default. Clang defaults to
OFF, but that's just Clang. In that mode, the check is required:
https://en.cppreference.com/w/c/language/arithmetic_types#Complex_floating_types
And the specific point in the [C99
standard](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf)
is: `G.5.1 Multiplicative operators`.
But set it to ON and the check is gone:
https://godbolt.org/z/aG8fnbYoP
Summary: the argument has moved from C++ to C --- and even there, to
implementation-defined behavior with a standard opt-out mechanism.
Like with C++, I maintain that the C standard is not a particularly
meaningful thing for MLIR to follow here, because people doing business
with complex numbers tend to lower them to real numbers themselves, or
have their own specialized complex types, either way not relying on
C99's `_Complex` type --- and the very poor performance of the
`CX_LIMITED_RANGE OFF` behavior (default in Clang) is certainly a key
reason why people who care prefer to stay away from `_Complex` and
`std::complex`.
A good example that's relevant to MLIR's space is CUDA's `cuComplex`
type (used in the cuBLAS CGEMM interface). Here is its multiplication
function. The comment about competitiveness is interesting: it's not a
quirk of this particular function, it's the spirit underpinning
numerical code that matters.
1bf5cd15c5/v8.0/include/cuComplex.h (L106-L120)
```c
/* This implementation could suffer from intermediate overflow even though
* the final result would be in range. However, various implementations do
* not guard against this (presumably to avoid losing performance), so we
* don't do it either to stay competitive.
*/
__host__ __device__ static __inline__ cuFloatComplex cuCmulf (cuFloatComplex x,
cuFloatComplex y)
{
cuFloatComplex prod;
prod = make_cuFloatComplex ((cuCrealf(x) * cuCrealf(y)) -
(cuCimagf(x) * cuCimagf(y)),
(cuCrealf(x) * cuCimagf(y)) +
(cuCimagf(x) * cuCrealf(y)));
return prod;
}
```
Another instance in CUTLASS:
https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/complex.h#L231-L236
Signed-off-by: Benoit Jacob <jacob.benoit.1@gmail.com>
## Description
This PR updates the `ConvertGpuOpsToLLVMSPVOps`'s option by replacing
the `index-bitwidth` with a boolean option `use-64bit-index` (similar to
the `ConvertGPUToSPIRV` option).
The reason for this modification is because the
`ConvertGpuOpsToLLVMSPVOps`:
> Generate LLVM operations to be ingested by a SPIR-V backend for gpu
operations
In the context of SPIR-V specifications only two physical addressing
models are allowed: `Physical32` and `Physical64`.
This change guarantees output sanity by preventing invalid or
unsupported index bitwidths from being specified.
The validation pass is used to check alignment of the IR against the
TOSA specification. This commit updates the `isValidElement` check to
more strictly align with the specifications supported element types.
Signed-off-by: Luke Hutton <luke.hutton@arm.com>
Tests in "vectorize-tensor-extract.mlir" are inconsistent and would
benefit from refactoring to:
* Clearly categorize tests into "contiguous load," "gather load," and
"scalar load + broadcast" cases, reflecting the structure of
tensor.extract vectorization.
* Unify variable naming (both MLIR and FileCheck).
* Ensure all tests exercise unmasked vectorization (masked vectorization
is covered in "vectorize-tensor-extract-masked.mlir").
* Improve and standardize formatting.
These changes will make it easier to identify the test cases being
exercised and simplify future maintenance or refactoring.
This is patch 3/N in the series. Below is a summary of the changes in
this patch.
----------------------------------------------------------------------
Summary of patch 3/N
----------------------------------------------------------------------
* Cluster all tests for "scalar load + broadcast" together
* Unify MLIR and FileCheck variable names, e.g. `%input`, `%output` ->
`%src`, `%init`.
Note, I haven't changed test function names to make it easier to track
changes (this PR is mostly about moving code). I will send a seperate PR
to rename the tests.
----------------------------------------------------------------------
Previous patches
----------------------------------------------------------------------
* https://github.com/llvm/llvm-project/pull/118977
* https://github.com/llvm/llvm-project/pull/119080
PR #116854 adds intrinsics for TMA Store with reduction.
This patch adds an NVVM Dialect Op for the same.
* Lit tests are added to verify the lowering to LLVM intrinsics and
invalid cases.
* The common verifier method is updated to handle im2col modes without
offsets.
This helps Ops like TMA Store, TMA StoreReduce etc.
* The nvvmir.mlir test file is already large. So, this patch adds the
tests for this Op
in a new file under a separate "nvvm/" directory.
[mlir/test/Target/LLVMIR/"nvvm"/tma_store_reduce.mlir]
PTX Spec reference:
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions-cp-reduce-async-bulk-tensor
Signed-off-by: Durgadoss R <durgadossr@nvidia.com>
Tests in "vectorize-tensor-extract.mlir" are inconsistent and would
benefit from refactoring to:
* Clearly categorize tests into "contiguous load," "gather load," and
"scalar load + broadcast" cases, reflecting the structure of
tensor.extract vectorization.
* Unify variable naming (both MLIR and FileCheck).
* Ensure all tests exercise unmasked vectorization (masked vectorization
is covered in "vectorize-tensor-extract-masked.mlir").
* Improve and standardize formatting.
These changes will make it easier to identify the test cases being
exercised and simplify future maintenance or refactoring.
This is patch 2/N in the series; below is a summary of the specific
changes
in this patch.
----------------------------------------------------------------------
Summary of patch 2/N
----------------------------------------------------------------------
Makes all tests re-use the same TD sequence. This TD sequence has been
extracted to a deducted file:
* "td/vectorize-with-patterns.mlir".
----------------------------------------------------------------------
Previous patches:
----------------------------------------------------------------------
* https://github.com/llvm/llvm-project/pull/118977
This patch removes an assert in `vectorizeTensorExtract` that was
blocking
the vectorization of 0-D tensor.extract operations, e.g.:
```mlir
%1 = tensor.extract %src[] : tensor<f32>
```
As demonstrated by the included tests, this case is already effectively
supported.
**Context**
The removed assert was introduced in #109580 as a guard, pending proper
support
and testing for 0-D tensors. This PR addresses that previously
undocumented
TODO. Apologies for the oversight!
**Updates and Tests**
* Revised the existing test `@negative_no_loop` to ensure the
`vectorize_nd_extract` attribute is included, allowing the vectorizer
to process it. The test was renamed and variables updated for clarity.
* Added a new test `@extract_scalar_from_0d_into_1d` to cover "mixed"
0-D/1-D tensor extraction, e.g.:
```mlir
%res = linalg.generic {
indexing_maps = [#map],
iterator_types = ["parallel"]
} outs(%init : tensor<1xf32>) {
^bb0(%in: f32):
%1 = tensor.extract %src[] : tensor<f32>
linalg.yield %1 : f32
} -> tensor<1xf32>
return %res : tensor<1xf32>
```
**Additional updates**
I also took the liberty and improved test coverage for 0-D tensor in the
vectorizer tests:
* Added a specific test for "0D linalg.generic" in
"vectorization-with-patterns.mlir".
* Renamed several tests in "vectorization-with-patterns.mlir" to clarify
that the 0-D case is now covered.
Tests in "vectorize-tensor-extract.mlir" are inconsistent and would
benefit from refactoring to:
* Clearly categorize tests into "contiguous load," "gather load," and
"scalar load + broadcast" cases, reflecting the structure of
tensor.extract vectorization.
* Unify variable naming (both MLIR and FileCheck).
* Ensure all tests exercise unmasked vectorization (masked vectorization
is covered in "vectorize-tensor-extract-masked.mlir").
* Improve and standardize formatting.
These changes will make it easier to identify the test cases being
exercised and simplify future maintenance or refactoring.
This is patch 1/N in the series; below is a summary of the specific
changes
in this patch.
----------------------------------------------------------------------
Summary for patch 1/N
----------------------------------------------------------------------
This PR updates the `@vectorize_scalar_broadcast_column_tensor` test in
"vectorize-tensor-extract.mlir", which exercises:
* Vectorization of tensor.extract.
* A scalar read followed by a broadcast.
* Reading from a constant column tensor.
Currently, the test uses "masked" vectorization, but the file
exclusively tests unmasked vectorization paths. To address this
inconsistency, this PR removes masking, aligning the test with the rest
of the file. Masked vectorization scenarios remain covered in
"vectorize-tensor-extract-masked.mlir". This update switches from:
* `transform.structured.vectorize`, to
* `transform.structured.vectorize_children_and_apply_patterns`.
The latter approach applies canonicalization patterns, significantly
simplifying the generated output.
Additional improvements for readability:
* Renamed the test function for clarity.
* Updated variable names and removed unused variables.
* Added empty lines for better formatting.
This PR adds default option below. The new options will come as default
to true and not change the original lowering behavior of pack and unpack
op.
- lowerPadLikeWithInsertSlice to packOp (with default = true)
- lowerUnpadLikeWithExtractSlice to unPackOp (with default = true)
The motivation of the PR is finer granular control of the lowering of
pack and unpack Ops. This is useful in particular when we want to
guarantee that there's no additional insertslice and extractslice that
interfere with tiling. With the original lowering pipeline, packOp and
unPackOp may be lowered to insertslice and extractslice when the high
dimensions are unit dimensions and no transpose is invovled. Under such
circumstances, such insert and extract slice ops will block
producer/consumer fusion tile + fuse transforms. With this PR, we will
be able to disable such lowering path and allow consumer fusion to go
through as expected.
Adds a new mlir-opt test-only pass, -test-vulkan-runner-pipeline, which
runs a set of passes needed for mlir-vulkan-runner, and removes them
from the runner. The tests are changed to invoke mlir-opt with this flag
before invoking the runner. The passes moved are ones concerned with
lowering of the device code prior to serialization to SPIR-V. This is an
incremental step towards moving the entire pipeline to mlir-opt, to
align with other runners (see #73457).
The acc data clause operations hold an operand named `varPtr`. This was
intended to hold a pointer to a variable - where the element type of
that pointer specifies the type of the variable. However, for both
memref and llvm dialects, this assumption is not true. This is because
memref element type for cases like memref<10xf32> is simply f32 and for
LLVM, after opaque pointers, the variable type is no longer recoverable.
Thus, introduce varType to ensure that appropriate semantics are kept.
Both the parser and printer for this new type attribute allow it to not
be specified in cases where a dialect's getElementType() applied to
`varPtr`'s type has a recoverable type. And more specifically, for FIR,
no changes are needed in the MLIR unit tests.
Lowering `math.powf` to `llvm.intr.powf` will result in `pow(x, 0) =
1`, even for `x=0`. When using the Math dialect expansion patterns,
`pow(0, 0)` will result in `-nan`, however, This change adds two
additional instructions to the lowering to ensure the `pow(x, 0)` case
lowers to to `1` regardless of the value of `x`.
Resolves https://github.com/llvm/llvm-project/issues/118945.
This reapplies PR #118463 after introducing a fix for a bug uncovere by
the test suite. The problem is that when the alloca block is terminated
with a conditional branch, this violates a pre-condition of
`allocatePrivateVars` (which assumes the alloca block has a single
successor). This new PR includes a test that reproduces the issue.
Extend MLIR to LLVM lowering by adding support for `omp.wsloop` for
delayed privatization. This also refactors a few bit of code to isolate
the logic needed for `firstprivate` initialization in a shared util that
can be used across constructs that need it. The same is done for
`dealloc` regions.
This PR adds translation support for task detach. Essentially, if the
`detach` clause is present on a task, emit a
`__kmpc_task_allow_completion_event` on it, and store its return (of
type `kmp_event_t*`) into the `event_handle`.
* Adds extra comments to group Ops
* Unifies the test function naming, i.e.
* `@vector_{op_name}_{variant}` -> `@{op_name}_{variant}`
* Unifies input variable names (`%input` -> `%arg0`)
* Capitalises LIT variable names (e.g. `%[[insert]]` --> `%[[INSERT]]`)
* Moves `@step_scalable()` _below_ its "fixed-width" counterpart
(to follow the existing consistency within this file).
There's still some inconsistencies within this file - I'm happy to send
more updates if folks find it useful. But I'd definitely recommend
splitting across multiple PRs (otherwise it's hard to review).
This commit adds additional tests and documentation for
`DecomposeOuterUnitDimsUnPackOpPattern` to ensure symmetry with its
counterpart for `tensor.pack`, `DecomposeOuterUnitDimsPackOpPattern`.
The new tests aim to improve implementation, documentation, and test
coverage for tensor.unpack. They cover the following scenarios:
* Static tile sizes: A simple `tensor.unpack` case
(`@simple_unpack_static_tiles`).
* Dynamic tile size: `tensor.unpack` with a single dynamic tile size
(`@simple_unpack_dynamic_tile`).
* Transpose: `tensor.unpack` with dynamic tile size and transpose
(`@simple_unpack_dynamic_tile_transpose`), currently commented out due
to some missing logic (see below)
* Scalable tile size: `tensor.unpack` with a scalable inner tile size
(@simple_unpack_scalable_tile).
Notes:
The test `@simple_unpack_dynamic_tile_transpose` is commented out
because the logic for capturing dynamic sizes for `tensor::EmptyOp` when
some tile sizes are dynamic is incomplete. This missing functionality
will be addressed in a follow-up patch.
