As was correctly pointed out by @azteca1998, the element type for a
`llvm.getelementptr` was only read when using an attribute and not when
using a type. As pointed out in
https://github.com/llvm/llvm-project/issues/63832#issuecomment-1643751039,
the translation to LLVM would work for
```mlir
llvm.func @main(%0 : !llvm.ptr) -> !llvm.ptr {
%1 = llvm.getelementptr %0[0] { elem_type = !llvm.ptr } : (!llvm.ptr) -> !llvm.ptr
llvm.return %1 : !llvm.ptr
}
```
but not for
```mlir
llvm.func @main(%0 : !llvm.ptr) -> !llvm.ptr<ptr> {
%1 = llvm.getelementptr %0[0] : (!llvm.ptr) -> !llvm.ptr<ptr>
llvm.return %1 : !llvm.ptr<ptr>
}
```
This was caused by the `LLVM_GEPOp` builder only reading the type from
the attribute (`{ elem_type = !llvm.ptr }`), but not from the pointer
type (`!llvm.ptr<ptr>`).
Fixes#63832.
EDIT: During review Markus Böck pointed out that this bugfix adds new functionality for typed pointers, but this functionality shouldn't be there in the first place. In response, Oleksandr "Alex" Zinenko pointed out that this is okay for now since the typed pointers will be removed in an upcoming release anyway, so it's best to merge this PR and spend time on the removal instead.
In order to support indirect vararg calls, we need to have information about the
callee type - this patch adds a `callee_type` attribute that holds that.
The attribute is required for vararg calls, else, it is optional and the callee
type is inferred by the operands and results of the operation if not present.
The syntax for non-vararg calls remains the same, whereas for vararg calls, it
is changed to this:
```
llvm.call %p(%arg0, %arg0) vararg(!llvm.func<void (i32, ...)>) : !llvm.ptr, (i32, i32) -> ()
llvm.call @s(%arg0, %arg0) vararg(!llvm.func<void (i32, ...)>) : (i32, i32) -> ()
```
The vscale_range is used for scalabale vector functionality in Arm
Scalable Vector Extension to select the size of vector operation (and I
thnk RISCV has something similar).
This patch adds the base support for the vscale_range attribute to the
LLVM::FuncOp, and the marshalling for translation to LLVM-IR and import
from LLVM-IR to LLVM dialect.
This attribute is intended to be used at higher level MLIR, specified
either by command-line options to the compiler or using compiler
directives (e.g. pragmas or function attributes in the source code) to
indicate the desired range.
This revision replaces the LLVM dialect NullOp by the recently
introduced ZeroOp. The ZeroOp is more generic in the sense that it
represents zero values of any LLVM type rather than null pointers only.
This is a follow to https://github.com/llvm/llvm-project/pull/65508
Use the recently introduced llvm.mlir.zero operation for values with
LLVM target extension type. Replaces the previous workaround that uses a
single zero-valued integer attribute constant operation.
Signed-off-by: Lukas Sommer <lukas.sommer@codeplay.com>
According to the LLVM language reference, both volatile memory
operations and atomic operations (except unordered) do not simply read
memory but also perform write operations on arbitrary memory[0][1].
In the case of volatile memory operations, this is the case due to the
read possibly having target specific properties. A common real-world
situation where this happens is reading memory mapped registers on an
MCU for example. Atomic operations are more special. They form a kind of
memory barrier which from the perspective of the optimizer/lang-ref
makes writes from other threads visible in the current thread. Any kind
of synchronization can therefore conservatively be modeled as a
write-effect.
This PR therefore adjusts the side effects of `llvm.load` and
`llvm.store` to add unknown global read and write effects if they are
either atomic or volatile.
Regarding testing: I am not sure how to best test this change for
`llvm.store` and the "globalness" of the effect that isn't just a unit
test checking that the output matches exactly. For the time being, I
added a test making sure that `llvm.load` does not get DCEd in
aforementioned cases.
Related logic in LLVM proper:
3398744a61/llvm/lib/IR/Instruction.cpp (L638-L676)3398744a61/llvm/include/llvm/IR/Instructions.h (L258-L262)
[0] https://llvm.org/docs/LangRef.html#volatile-memory-accesses
[1] https://llvm.org/docs/Atomics.html#monotonic
Functions are always callable operations and thus every operation
implementing the `FunctionOpInterface` also implements the
`CallableOpInterface`. The only exception was the FuncOp in the toy
example. To make implementation of the `FunctionOpInterface` easier,
this commit lets `FunctionOpInterface` inherit from
`CallableOpInterface` and merges some of their methods. More precisely,
the `CallableOpInterface` has methods to get the argument and result
attributes and a method to get the result types of the callable region.
These methods are always implemented the same way as their analogues in
`FunctionOpInterface` and thus this commit moves all the argument and
result attribute handling methods to the callable interface as well as
the methods to get the argument and result types. The
`FuntionOpInterface` then does not have to declare them as well, but
just inherits them from the `CallableOpInterface`.
Adding the inheritance relation also required to move the
`FunctionOpInterface` from the IR directory to the Interfaces directory
since IR should not depend on Interfaces.
Reviewed By: jpienaar, springerm
Differential Revision: https://reviews.llvm.org/D157988
D157096 introduces a new verifier for debug location
on call operation. `isDeclaration()` is actually not defined
for `LLVMFuncOp` and default to the interface definition that always
return `false`. This leads to wrong diagnostic in some case as shown in the added test.
Use `callee.isExternal()` instead that returns the
desired information.
We have seen this error being triggered during flang codegen.
Reviewed By: Dinistro, vzakhari
Differential Revision: https://reviews.llvm.org/D157447
This commit introduces a debug location verifier for the LLVM dialect's
call operation. LLVM does not allow calls to have no debug location when
they reference an inlinable function with debug information. This
apparenlty breaks assumptions of LLVM's inliner.
So far, there was a hack in the LLVM export that avoided this case to
be triggered, but that hack causes issues when debug intrinsics are
involved. Link to the revision that inroduced the export hack:
https://reviews.llvm.org/D88135
LLVM's verifier as a reference: 2df05cd01c/llvm/lib/IR/Verifier.cpp (L3546)
Reviewed By: gysit, zero9178
Differential Revision: https://reviews.llvm.org/D157096
This commit adds a DIModuleAttr to the set of debug info attributes and
extends the LLVM IR import and export to support it.
DIModule metadata is missing in the LLVM LangRef and cannot be produced
from C or C++ input. So far, we only observed classic flang producing
such DI metadata.
Reviewed By: zero9178
Differential Revision: https://reviews.llvm.org/D156969
Add a method to the CallOpInterface to get a mutable operand range over
the function arguments. This allows to add, remove, or change the type
of call arguments in a generic manner without having to assume that the
argument operand range is at the end of the operand list, or having to
type switch on all supported concrete operation kinds.
Alternatively, a new OpInterface could be added which inherits from
CallOpInterface and appends it with the mutable variants of the base
interface.
There will be two users of this new function in the beginning:
(1) A few passes in the Arc dialect in CIRCT already use a downstream
implementation of the alternative case mentioned above: https://github.com/llvm/circt/blob/main/include/circt/Dialect/Arc/ArcInterfaces.td#L15
(2) The BufferDeallocation pass will be modified to be able to pass
ownership of memrefs to called private functions if the caller does not
need the memref anymore by appending the function argument list with a
boolean value per memref, thus enabling earlier deallocation of the
memref which can lead to lower peak memory usage.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D156675
This revision removes the metadata op, that to the best of our
knowledge, has no more uses after switching to a purely attribute based
metadata representation:
https://reviews.llvm.org/D155444https://reviews.llvm.org/D155285https://reviews.llvm.org/D155159
These changes got unlocked after landing distinct attribute support:
https://reviews.llvm.org/D153360,
which enables modeling distinct metadata using attributes. As a result,
all metadata kinds are now represented using attributes. Previously,
there has been a mix of attribute and op based representations.
Having attribute only metadata makes it possible to update the metadata
in-parallel, while updating the global metadata operation has been
a sequential process. The LLVM Dialect inliner already benefits from
this change and now creates new alias scopes and domains during
inlining rather than dropping the no alias information:
https://reviews.llvm.org/D155712
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D156217
This revision adds a branch weight op interface for the call / branch
operations that support branch weights. It can be used in the LLVM IR
import and export to simplify the branch weight conversion. An
additional mapping between call operations and instructions ensures
the actual conversion can be done in the module translation itself,
rather than in the dialect translation interface. It also has the
benefit that downstream users can amend custom metadata to the call
operation during the export to LLVM IR.
Reviewed By: zero9178, definelicht
Differential Revision: https://reviews.llvm.org/D155702
The current representation of TBAA is the very last in-tree user of the `llvm.metadata` operation.
Using ops to model metadata has a few disadvantages:
* Building a graph has to be done through some weakly typed indirection mechanism such as `SymbolRefAttr`
* Creating the metadata has to be done through a builder within a metadata op.
* It is not multithreading safe as operation insertion into the same block is not thread-safe
This patch therefore converts TBAA metadata into an attribute representation, in a similar manner as it has been done for alias groups and access groups in previous patches.
This additionally has the large benefit of giving us more "correctness by construction" as it makes things like cycles in a TBAA graph, or references to an incorrectly typed metadata node impossible.
Differential Revision: https://reviews.llvm.org/D155444
Using MLIR attributes instead of metadata has many advantages:
* No indirection: Attributes can simply refer to each other seemlessly without having to use the indirection of `SymbolRefAttr`. This also gives us correctness by construction in a lot of places as well
* Multithreading safe: The Attribute infrastructure gives us thread-safety for free. Creating operations and inserting them into a block is not thread-safe. This is a major use case for e.g. the inliner in MLIR which runs in parallel
* Easier to create: There is no need for a builder or a metadata region
This patch therefore does exactly that. It leverages the new distinct attributes to create distinct access groups in a deterministic and threadsafe manner.
Differential Revision: https://reviews.llvm.org/D155285
Using MLIR attributes instead of metadata has many advantages:
* No indirection: Attributes can simply refer to each other seemlessly without having to use the indirection of `SymbolRefAttr`. This also gives us correctness by construction in a lot of places as well
* Multithreading save: The Attribute infrastructure gives us thread-safety for free. Creating operations and inserting them into a block is not thread-safe. This is a major use case for e.g. the inliner in MLIR which runs in parallel
* Easier to create: There is no need for a builder or a metadata region
This patch therefore does exactly that. It leverages the new distinct attributes to create distinct alias domains and scopes in a deterministic and threadsafe manner.
Differential Revision: https://reviews.llvm.org/D155159
The goal of this pattern is to eliminate all GEPs that have more than two indices by splitting it into multiple GEPs.
The advantage of this change is that the resulting GEPs only ever index into one aggregate at the time. This enables handling sub-aggregates in other patterns and also creates IR with easier to deduce pointer element types.
As a minor note, `getResultPtrElementType` for `GEPOp` was rewritten since it did not properly handle dynamic indices. The way GEPOp is specified, the resulting pointer element type can *always* be deduced from its base type and indices.
Differential Revision: https://reviews.llvm.org/D154692
This revision introduces new rewrites to improve the type consistency of
a program expressed in the LLVM dialect.
Type consistency means that a given opaque pointer is consistently used
assuming the same pointee type, in a best effort basis. The introduced
rewrites modify the program to improve type consistency while preserving
the same semantics. This is useful for two main reasons:
- Transformation passes in the LLVM dialect like SROA or Mem2Reg can
analyse code better if type information and structure is used in a
consistent manner. Opaque pointers make this difficult to enforce, but
type consistency improvements increase the amount of occurences where
reasonable analysis can pick up on transformable patterns.
- While LLVM IR is not particularly picky about inconsistent type uses,
it may be of interest to lift LLVM IR into higher level dialects.
Having more instances of consistent type information would help
lifting into dialects that do care about consistent types.
In order to detect cases of inconsistent uses, operations returning an
LLVMPointer can implement the GetResultPtrElementType interface, which
allows getting a hint of which type the provided pointer should see its
pointee as, if such hint is available. The provided rewrites will then
use this hint to attempt to modify operations using the pointers so they
use the hinted type when dealing with the pointer.
Two transformations have been implemented in this revision:
- When a load/store uses a struct ptr directly to write to the first
element of the struct, introduce a GEP to the first element so the
type structure is preserved.
- When a GEP statically indexes in a pointer with a base type
inconsistent with the hint, try to find indices using the hint as a
base type that would result in the same offset, and replace the GEP
with this indexing.
More transformations are possible and I hope this is only a beginning
for this simplification effort.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153973
This revision adds support for the llvm.dbg.label.intrinsic
and the corresponding DILabel metadata.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153975
This commit changes the 'llvm.switch' parsing to not silently fail when
it encounters superfluous commas in the case list.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153841
This revision adds comdat support to functions. Additionally,
it ensures only comdats that have uses are imported/exported and
only non-empty global comdat operations are created.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153739
This revision introduces some constant folding features to the LLVM
dialect. This specific choice of operations to cover is intended to
allow the elimination of logic generated by mem2reg with memset in the
common case of memsets of constant values.
This also introduces new verifiers for integer extension operations.
This lead to a fix in SPIRV to LLVM conversion, as it would sometimes
generate invalid ZExt and SExt operations.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153135
Previously, llvm.atomicrmw only allowed operands that are pointers to
LLVM floats or integers. However, according to the LLVM IR Language
Reference, that op allows pointer to pointer operands in its `xchg`
mode. This patch allows those operands also in MLIR's LLVM dialect and
adapts the tests accordingly.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153747
This revision ensures SwitchOps with case and condition
bitwidths other than 32-bit are imported properly. It adds an
APInt based builder to the SwitchOp and implements
a verifier that checks that the condition and the case
value types match.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153438
The LLVM comdat operation specifies how to deduplicate globals with the
same key in two different object files. This is necessary on Windows
where e.g. two object files with linkonce globals will not link unless
a comdat for those globals is specified. It is also supported in the ELF
format.
Differential Revision: https://reviews.llvm.org/D150796
The revision adds the alias analysis and access group
interfaces to the call operation.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D151693
Add support for the `llvm::TargetExtType` to the MLIR LLVM dialect.
Target extension types were introduced to represent target-specific types, which are opaque to the compiler and optimizations.
The patch also enforces some of the constraints defined for the target extension type in the LLVM language reference manual.
Signed-off-by: Lukas Sommer <lukas.sommer@codeplay.com>
Reviewed By: ftynse, gysit, Dinistro
Differential Revision: https://reviews.llvm.org/D151446
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This patch updates all remaining uses of the deprecated functionality in
mlir/. This was done with clang-tidy as described below and further
modifications to GPUBase.td and OpenMPOpsInterfaces.td.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D151542
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
* https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…"
* Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This follows a previous patch that updated calls
`op.cast<T>()-> cast<T>(op)`. However some cases could not handle an
unprefixed `cast` call due to occurrences of variables named cast, or
occurring inside of class definitions which would resolve to the method.
All C++ files that did not work automatically with `cast<T>()` are
updated here to `llvm::cast` and similar with the intention that they
can be easily updated after the methods are removed through a
find-replace.
See https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
for the clang-tidy check that is used and then update printed
occurrences of the function to include `llvm::` before.
One can then run the following:
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-export-fixes /tmp/cast/casts.yaml mlir/*\
-header-filter=mlir/ -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D150348
Currently `CallOpInterface` has a method `getCallableForCallee` to have a consistent way to get the callee from an operation with `CallOpInterface`, but missing a consistent way to set a callee for an operation with `CallOpInterface`.
A set callee method is useful for transformations that operate on `CallOpInterface`, and change the callee, e.g., a pass that specialize function, which clone the callee, and change the `CallOpInterface`'s callee to the cloned version. Without such method, transformation would need to understand the implementation for every operations with `CallOpInterface`, and have a type switch to handle them.
This review adds a method to set callee for operation with `CallOpInterface`.
Reviewed By: gysit, zero9178o
Differential Revision: https://reviews.llvm.org/D149763
Following the steps of the LLVM verifier
(b2c48559c8/llvm/lib/IR/Verifier.cpp (L4195)),
checks in llvm.func verifier are added to ensure consistency of
llvm.landingpad operations' result types and llvm.resume operations'
input types.
As in LLVM, we will allow llvm.resume operations with input values
defined by operations other than llvm.landingpad.
Signed-off-by: Victor Perez <victor.perez@codeplay.com>
Reviewed By: gysit, Dinistro
Differential Revision: https://reviews.llvm.org/D146968
A fully fledged LLVM inliner will require a lot of logic. Since
`LLVMDialect.cpp` is large enough as it is, preemptively outline the
inlining logic into a separate `.cpp` file. This will also allow us to
add a `DEBUG_TYPE` for debugging the inliner.
The name `LLVMInlining` was chosen over `LLVMInlinerInterface` to keep
the option open for exposing inlining functionality even when not
invoked through the `DialectInlinerInterface`.
Depends on D146616
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D146628
Support inlining of function calls with the byval attribute on function
arguments by copying the pointee into a newly alloca'ed pointer at the
callsite before inlining.
The alignment attribute is not yet taken into account.
Reviewed By: ftynse, gysit
Differential Revision: https://reviews.llvm.org/D146616
Replace references to enumerate results with either result_pairs
(reference wrapper type) or structured bindings. I did not use
structured bindings everywhere as it wasn't clear to me it would
improve readability.
This is in preparation to the switch to zip semantics which won't
support non-const lvalue reference to elements:
https://reviews.llvm.org/D144503.
I chose to use values instead of const lvalue-refs because MLIR is
biased towards avoiding `const` local variables. This won't degrade
performance because currently `result_pair` is cheap to copy (size_t
+ iterator), and in the future, the enumerator iterator dereference
will return temporaries anyway.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D146006
This commit introduces the LLVM's visibility attribute and adds it to
both globals and functions.
Furthermore, this commit ensures that "thread_local" is printed in the
correct place and adds a test for that.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D145790
This revision adds the inalloca attribute to the alloca operation in the LLVMIR dialect.
It also adds tests for import and export.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D145483
The revision introduces two interfaces that provide access to
the alias analysis and access group metadata attributes. The
AliasAnalysis interface combines all alias analysis related
attributes (alias, noalias, and tbaa) similar to LLVM's getAAMetadata
method, while the AccessGroup interface is dedicated to the
access group metadata.
Previously, only the load and store operations supported alias analysis
and access group metadata. This revision extends this support to the
atomic operations. A follow up revision will also add support for the
memcopy, memset, and memove intrinsics. The interfaces then provide
convenient access to the metadata attributes and eliminate the need
of TypeSwitch or string based attribute access.
The revision still relies on string based attribute access for
the translation to LLVM IR (except for tbaa metadata). Only once
the the memory access intrinsics also implement the new interfaces,
the translation to LLVM IR can be fully switched to use interface
based attribute accesses.
Depends on D144875
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D144851
The revision renames LLVMOpsInterfaces.td since the the tablegen file
contains op and type interfaces.
Reviewed By: ftynse, Dinistro
Differential Revision: https://reviews.llvm.org/D144875
Part of https://discourse.llvm.org/t/rfc-switching-the-llvm-dialect-and-dialect-lowerings-to-opaque-pointers/68179
This patch adds the new pass option `use-opaque-pointers` to the GPU to LLVM lowerings (including ROCD and NVVM) and adapts the code to support using opaque pointers in addition to typed pointers.
The required changes mostly boil down to avoiding `getElementType` and specifying base types in GEP and Alloca.
In the future opaque pointers will be the only supported model, hence tests have been ported to using opaque pointers by default. Additional regression tests for typed-pointers have been added to avoid breaking existing clients.
Note: This does not yet port the `GpuToVulkan` passes.
Differential Revision: https://reviews.llvm.org/D144448
This revision adds atomic support to the StoreOp. It chooses
to print the atomic keywords together with the syncscope and
ordering arguments. The revision also implements verifiers to
ensure the constraints that apply to atomic store operations
are checked.
Depends on D144112
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D144200
