When the affine.parallel op was introduced, affine utilities weren't
extended to handle it. Extending these is straightforward and natural
given that addAffineParallelOpDomain has also been added.
Update/complete memref region compute to account for affine.parallel
ops. Handle failure cleanly.
Add and expose utilities missing for affine.parallel to be consistent
with affine.for.
All of these allow various affine passes to work with a combination of
affine.parallel and affine.for ops.
Differential Revision: https://reviews.llvm.org/D145669
This is a clean up before fixing issues identified in this pass by
https://github.com/llvm/llvm-project/issues/61380 and similar issues.
- Move patterns definitions closer to declarations.
- Simplify pattern definitions.
- Drop hand-written pass constructor in favor of an auto-generated on.
- Fix typos in pass description.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D146077
Previously, we generate function calls to compare values for sorting. It turns
out that the compiler doesn't inline those function calls. We now directly
generate inlined code. Also, modify the code for comparing values to use less
number of branches.
This improves all sort implementation in general. For arabic-2005.mtx CSR, the
improvement is around 25%.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D145442
Add conversion for integer multiplication in scf reductions in the
SCF to OpenMP dialect conversion.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D145948
This commit swaps back the `tosa.reshape` op used at the end of the
lowering for reduce ops with the op `tensor.expand_shape`. This is
needed to properly support dynamically-sized tensors. In such cases,
lowering directly to `tensor.expand_shape` allows us to control which
dimension gets expanded at the end using the knowledge of the
reduction. This would not be possible when using `tosa.reshape`, since
the op does not have a way of knowing that we are only unsqueezing a
single dimension.
Note: this change had previously been performed in
https://reviews.llvm.org/D133877.
Reviewed By: rsuderman
Differential Revision: https://reviews.llvm.org/D145986
This change does a bunch of renaming to clear up confusions in these files. In particular, this change:
* Renames variables and methods to clarify the "dim"/"lvl" distinction, and changes them to use the `Dimension`/`Level` types as appropriate.
* Introduces new typedefs
* `ExprId`, `LatPointId`, `LatSetId`: to clarify the interning design of the Merger.
* `LoopId`, `LoopOrd`: to clarify the distinction between arbitrary names for loop-variables, vs numeric identifiers based on the actual order of loop generation.
* `TensorId`
* (Future CLs will change these from typedefs to structs/classes, so that the typechecker can help avoid mixups.)
* Updates documentation to match the new terminology
* Adds additional assertions
* Adds `const` to local variables along the way
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D145756
The initial implementaiton of value handle update when the payload
operation defining the values associated with value handles was being
replaced required the replacement operation to have the same number of
results. This is not strictly necessary. The replacement operation may
have more results, or less results provided that there are no handles to
the results that have no equivalent in the replacement op.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D145254
This is for targets that do not support gather-like ops, e.g., SPIR-V.
Gather is expanded into lower-level vector ops with memory accesses
guarded with `scf.if`.
I also considered generating `vector.maskedload`s, but decided against
it to keep the `memref` and `tensor` codepath closer together. There's a
good chance that if a target doesn't support gather it does not support
masked loads either.
Issue: https://github.com/llvm/llvm-project/issues/60905
Reviewed By: ThomasRaoux
Differential Revision: https://reviews.llvm.org/D145942
Care is taken to order operands from least hoistable to most hoistable and to process subexpressions in the same
order.
This allows exposing more oppportunities for licm, cse and strength reduction.
Such a step should typically be applied while we still have loops in the IR and just before lowering affine ops to arith.
This is because the affine.apply canonicalization currently tries to maximally compose chains of affine.apply operations
and could undo the effects of these decompositions.
Depends on: D145784
Differential Revision: https://reviews.llvm.org/D145685
tosa.concat is lowered to tensor.insert_slice thus it should be in
TosaToTensor rather than in TosaToLinalg.
Reviewed By: rsuderman
Differential Revision: https://reviews.llvm.org/D145952
Reword the vector gather and scatter op description to make it
well-defined to have out-of-bounds indices when the corresponding mask
bits are false.
Update the code sample to avoid relying on C UB semantics to provide
informal semantics for vector.gather.
This change should be consistent with the existing interpretation of the
semantics in the codebase.
Issue: https://github.com/llvm/llvm-project/issues/60905
Reviewed By: dcaballe
Differential Revision: https://reviews.llvm.org/D145824
This is a convenient flag for context where we intend to summarize a top-level
operation without the full-blown regions it may hold.
Differential Revision: https://reviews.llvm.org/D145889
It happens from time to time that one may run mlir-opt expecting something to
happen, but the process is waiting on stdin. Print a message when reading from
stdin to warn developers.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D145469
This iterator is similar to `ForwardIterator` but enumerates blocks according to their successor relationship. As a first use case, this new iterator is utilized in the dialect conversion framework.
Differential Revision: https://reviews.llvm.org/D144888
This component acts as an action handler that can be registered in the
MLIRContext. It is the main orchestration of the infrastructure, and implements
support for clients to hook there and snoop on or control the execution.
This is the basis to build tracing as well as a "gdb-like" control of the
compilation flow.
The ExecutionContext acts as a handler in the MLIRContext for executing an
Action. When an action is dispatched, it'll query its set of Breakpoints
managers for a breakpoint matching this action. If a breakpoint is hit, it
passes the action and the breakpoint information to a callback. The callback
is responsible for controlling the execution of the action through an enum
value it returns. Optionally, observers can be registered to be notified
before and after the callback is executed.
Differential Revision: https://reviews.llvm.org/D144812
This commit introduces the LLVM's visibility attribute and adds it to
both globals and functions.
Furthermore, this commit ensures that "thread_local" is printed in the
correct place and adds a test for that.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D145790
The inliner pass performs canonicalization when created programtically, run with `mlir-opt` with default options, or when explicitly specified. However, when running the pipeline resulting from a `-dump-pass-pipeline` on a default inline pass, the canonicalization is not performed as part of the inlining. This is because the default value for the `default-pipeline` option of the inline pass is an empty string, and this is selected during the dumping. When `InlinerPass::initializeOptions` detects the empty string, it sets the `defaultPipeline` to `nullptr`, which was previously set to canonicalize in the `InlinerPass` constructor, thus the canonicalization is not performed.
The added test checks if the inline pass performs canonicalization by default, and that the dumped `default-pipeline` is set to `canonicalize`.
Fixes: https://github.com/llvm/llvm-project/issues/60960
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D145066
A dialect can opt-in to handle versioning through the
`BytecodeDialectInterface`. Few hooks are exposed to the dialect to allow
managing a version encoded into the bytecode file. The version is loaded
lazily and allows to retrieve the version information while parsing the input
IR, and gives an opportunity to each dialect for which a version is present
to perform IR upgrades post-parsing through the `upgradeFromVersion` method.
Custom Attribute and Type encodings can also be upgraded according to the
dialect version using readAttribute and readType methods.
There is no restriction on what kind of information a dialect is allowed to
encode to model its versioning. Currently, versioning is supported only for
bytecode formats.
Reviewed By: rriddle, mehdi_amini
Differential Revision: https://reviews.llvm.org/D143647
This converts a specific form of `vector.reduction` to SPIR-V integer
dot product ops.
Add a new test pass to excercise this outside of the main vector to
spirv conversion pass.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D145760
Rationale:
The LLVM dialect supports passing fastmath flags from floating point ops to LLVMIR instructions. However, not all LLVM ops have the required attribute. This change adds support for fastmath flags to `llvm.intr.vector.reduce.{fmin,fmax}`. One scenario where this is useful is in lowering llvm.intr.vector.reduce.{fmax,fmin} to LLVMIR with `nnan` (NoNans) flag so it may be [[ 115c7beda7/llvm/lib/CodeGen/ExpandReductions.cpp (L159) | lowered to a shuffle reduction ]].
Changes:
- Make `LLVM_VecReductionF` implement the `FastmathFlagsInterface`; change is modeled on `LLVM_UnaryIntrOpF`
- Add an assembly format for `LLVM_VecReductionF` ops. The purpose is to keep existing functionality: avoid printing the fastmath flags attribute when it has its default value (`none`). Change is modeled on `LLVM_UnaryIntrOpBase`
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D145692
This change adds a new helper function `mlir::reifyResultShapes` that calls the corresponding interface method and also checks the result produced by the implementation when running in debug mode. Bugs due to incorrect interface implementations can be difficult to debug.
This helper function also reduces the amount of code needed at call sites: the cast to `ReifyRankedShapedTypeOpInterface` is done in the helper function.
Differential Revision: https://reviews.llvm.org/D145777
`reifyResultShapes` should return an IntegerAttr if and only if the corresponding dimension is static.
Differential Revision: https://reviews.llvm.org/D145702
This has caused build failures when enabling opaque pointers for the GPU integration tests as could be seen here:
https://lab.llvm.org/buildbot/#/builders/220/builds/16946 and here https://lab.llvm.org/buildbot/#/builders/61/builds/40822
The gist of the issue was the use of a wrong pointer base type within a GEP. There sadly was no test coverage for either the generating of that GEP, nor is LLVM Dialects GEP verifier currently capable of catching such issues, so it went unnoticed until the integration tests actually attempted to convert it to LLVM IR.
Differential Revision: https://reviews.llvm.org/D145774
Part of https://discourse.llvm.org/t/rfc-switching-the-llvm-dialect-and-dialect-lowerings-to-opaque-pointers/68179
When this patch lands any downstream users with custom LLVM conversion passes not yet using opaque pointers will start either experiencing assertions being triggered, null pointer dereferences or at the very least verifier errors. These can be either fixed by switching to opaque pointers or simply disabling opaque pointers in both pass options of any upstream conversion passes and any uses of `LLVMTypeConverter` via the `LowerToLLVMOptions`.
Users using just MLIRs conversion passes to the LLVM Dialect should not experience any change in functionality except when inspecting the output from the passes.
Differential Revision: https://reviews.llvm.org/D145585
This change allows using fp8 pointers when exporting to LLVM, because we anyway export them as opaque pointers, however full support of fp8 types is not yet implemented on the LLVM side.
Differential Revision: https://reviews.llvm.org/D143008
