489 Commits

Author SHA1 Message Date
Lei Zhang
3c278e5e27 [mlir][spirv] Fix spirv.MatrixTimesScalar for cooperative matrix
spirv.MatrixTimesScalar is allowed to use cooperative matrix.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D139279
2022-12-05 22:13:23 +00:00
River Riddle
aef89c8b41 [mlir] Cleanup lingering problems surrounding attribute/type aliases
This commit refactors attribute/type alias generation to be similar to how
we do it for operations, i.e. we generate aliases determined on what is
actually necessary when printing the IR (using a dummy printer for alias
collection). This allows for generating aliases only when necessary, and
also allows for proper propagation of when a nested alias can be deferred.
This also necessitated a fix for location parsing to actually parse aliases
instead of ignoring them.

Fixes #59041

Differential Revision: https://reviews.llvm.org/D138886
2022-11-30 17:02:54 -08:00
Ivan Butygin
339c87a8a0 [mlir][spirv] Add group ops
Also, gen_spirv_dialect script was broken was broken due to SPV->SPIRV renamings.

Differential Revision: https://reviews.llvm.org/D138756
2022-11-29 15:03:31 +01:00
Tobias Gysi
36fa80b046 [mlir][llvm] Make additional int debug attribute parameters optional.
The revision makes additional integer debug info parameters such as
line or column numbers optional (e.g. the line and column numbers 0 are
elided when printing, which is in line with the LLVM implementation).

Depends on D138405

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D138437
2022-11-29 14:02:58 +02:00
Tobias Gysi
be4b49407e [mlir][llvm] Import debug intrinsics from LLVMIR.
Currently, the import of LLVMIR fails if the program contains debug
intrinsics. The revision adds support to import debug intrinsics that
have no debug expression attached and drops all debug intrinsics with a
non-empty debug expression. It also moves the existing debug intrinsics
into the "intr" namespace by deriving from LLVM_IntrOp.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D138405
2022-11-29 12:31:54 +02:00
Hanhan Wang
0a1569a400 [mlir][NFC] Remove trailing whitespaces from *.td and *.mlir files.
This is generated by running

```
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.td
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.mlir
```

Reviewed By: rriddle, dcaballe

Differential Revision: https://reviews.llvm.org/D138866
2022-11-28 15:26:30 -08:00
Slava Zakharin
29016d2830 [mlir][llvmir] Translate function result attributes to LLVM IR.
Translate align, noalias, noundef, signext and zeroext result
attributes from llvm.func to LLVM IR.

This is needed for https://github.com/llvm/llvm-project/issues/58579

Differential Revision: https://reviews.llvm.org/D137049
2022-11-18 12:03:42 -08:00
Tobias Gysi
bc270f9ed1 [mlir][llvm] Iterative constant import from LLVM IR.
Instead of importing constant expressions recursively, the revision
walks all dependencies of an LLVM constant iteratively. The actual
conversion then iterates over a list of constants and all intermediate
constant values are added to the value mapping. As a result, an LLVM IR
constant maps to exactly one MLIR operation per function. The revision
adapts the existing tests since the constant ordering changed for
aggregate types. Additionally, it adds extra tests that mix aggregate
constants and constant expressions.

Depends on D137416

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137559
2022-11-18 15:15:35 +01:00
River Riddle
446fc42d7c [mlir] Fix ordering of intermixed attribute/type aliases
We properly order dependencies between attribute/type aliases,
but we currently always print attribute aliases separately from type
aliases. This creates problems if an attribute wants to use a type
alias during printing.

This commit refactors alias collection such that attribute/type aliases
are collected together and printed together.

Differential Revision: https://reviews.llvm.org/D138162
2022-11-18 02:09:57 -08:00
Tobias Gysi
3f63889d58 [mlir][llvm] Import additional debug info from LLVMIR.
Add a DebugImporter to convert LLVMIR debug metadata into
MLIR debug attributes. It is the counterpart to the
DebugTranslation class and supports the same attributes.
The revision only supports the translation of instruction,
function, and module debug information. The import of
intrinsics is left to a later revision.

Depends on D138206

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D138209
2022-11-18 09:44:40 +01:00
Tobias Gysi
63502901ca [mlir][llvm] Make some debug info attribute parameters optional.
The revision makes specific debug information attribute parameters
optional since some of them can be omitted in LLVMIR. The additional
flexibility enables a later revision that will support importing
debug information from LLVMIR. A special case is the types parameter
of the SubroutineTypeAttr. For void functions, its first entry is
null in LLVMIR. This revision splits the type parameter in an optional
resultType parameter and an argumentTypes array to support this corner
case.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D138206
2022-11-18 09:35:44 +01:00
Tobias Gysi
2c9d264785 [mlir][llvm] Move call and gep LLVM IR import tests.
The revision moves call and gep tests to instructions.ll.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137636
2022-11-17 10:41:07 +01:00
Tobias Gysi
2920095d41 [mlir][llvm] Consolidate function attribute import tests (NFC).
The revision puts the function attributes tests previously distributed
accross multiple files int a single function-attributes.ll test file.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137627
2022-11-17 10:28:02 +01:00
Tobias Gysi
356f41705d [mlir][llvm] Test LLVM IR global variable import separately (NFC).
The revision moves tests related to LLVM IR global variable
import into a separate global-variables.ll file.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137622
2022-11-17 10:26:15 +01:00
River Riddle
1984f01acf [mlir][LLVM] Add base type and flag support to composite type
This requires defining the DIFlag enum from LLVM.

Differential Revision: https://reviews.llvm.org/D137667
2022-11-08 18:04:26 -08:00
River Riddle
7c849845e0 [mlir][LLVMDebugTranslation] Add inlinedAt to the key when caching locations
This avoids accidentally using the wrong inlined at scope when translating
locations.
2022-11-08 13:00:05 -08:00
rkayaith
e4889c0a04 [mlir][spirv] Change translation to use spirv.module
Update the SPIRV `mlir-translate` translations to translate to/from
`spirv.module` instead of `builtin.module`. This simplifies the
translation since the code no longer needs to walk the module looking
for a SPIRV module, however it requires passing `-no-implicit-module` to
all the tests.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D135819
2022-11-08 14:47:01 -05:00
Tobias Gysi
54fb17347b [mlir][llvm] Test LLVM IR constant import separately.
The revision moves tests related to LLVM IR constant import
into a separate constant.ll file and adds tests for integer,
null and undef immediates.

Depends on D136679

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137416
2022-11-08 10:35:20 +02:00
Tobias Gysi
9aae3dd94c [mlir][llvm] Update insertion point handling in LLVM import.
Insert constants and globals in order by maintaining the position
of the constant and global inserted last. Update the tests
to reflect the updated insertion order. Also make sure functions
are always inserted at the end of the module instead of at
the second last position and delete a spurious function in
the intrinsic.ll that seems to exist to avoid the first
function under test ends up at the end of the module.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136679
2022-11-08 10:07:52 +02:00
River Riddle
e83446f43b [mlir:LLVM] Fix translation of DISubRangeAttr to LLVM
We were missing DISubRangeAttr in the type switch. This
also updates a few of the range parameters to be optional.
2022-11-04 18:15:03 -07:00
Nikita Popov
304f1d59ca [IR] Switch everything to use memory attribute
This switches everything to use the memory attribute proposed in
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
The old argmemonly, inaccessiblememonly and inaccessiblemem_or_argmemonly
attributes are dropped. The readnone, readonly and writeonly attributes
are restricted to parameters only.

The old attributes are auto-upgraded both in bitcode and IR.
The bitcode upgrade is a policy requirement that has to be retained
indefinitely. The IR upgrade is mainly there so it's not necessary
to update all tests using memory attributes in this patch, which
is already large enough. We could drop that part after migrating
tests, or retain it longer term, to make it easier to import IR
from older LLVM versions.

High-level Function/CallBase APIs like doesNotAccessMemory() or
setDoesNotAccessMemory() are mapped transparently to the memory
attribute. Code that directly manipulates attributes (e.g. via
AttributeList) on the other hand needs to switch to working with
the memory attribute instead.

Differential Revision: https://reviews.llvm.org/D135780
2022-11-04 10:21:38 +01:00
Slava Zakharin
6f04011f15 [mlir][llvmir] Add support for llvm.signext/zeroext function attributes.
This change-set adds basic support for llvm.signext and llvm.zeroext
attributes, and makes sure that the attributes are translated to LLVM IR
when attached to arguments.

This is needed for https://github.com/llvm/llvm-project/issues/58579

Differential Revision: https://reviews.llvm.org/D137048
2022-11-03 09:19:02 -07:00
Slava Zakharin
473d001152 [mlir][llvmir] Convert attributes for functions without bodies.
So far the function argument attributes were only translated
for functions with bodies. This change makes sure that this
happens for functions without bodies (declarations) as well.

This is needed for https://github.com/llvm/llvm-project/issues/58579

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D137047
2022-11-03 08:52:19 -07:00
rkayaith
13bd410962 [mlir][Pass] Include anchor op in -pass-pipeline
In D134622 the printed form of a pass manager is changed to include the
name of the op that the pass manager is anchored on. This updates the
`-pass-pipeline` argument format to include the anchor op as well, so
that the printed form of a pipeline can be directly passed to
`-pass-pipeline`. In most cases this requires updating
`-pass-pipeline='pipeline'` to
`-pass-pipeline='builtin.module(pipeline)'`.

This also fixes an outdated assert that prevented running a
`PassManager` anchored on `'any'`.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D134900
2022-11-03 11:36:12 -04:00
Slava Zakharin
35c9085121 [mlir][llvmir] Support FastmathFlags for LLVM intrinsic operations.
This is required for D126305 code to propagate fastmath attributes
for Arith operations that are converted to LLVM IR intrinsics
operations.

LLVM IR intrinsic operations are using custom assembly format now
to avoid printing {fastmathFlags = #llvm.fastmath<none>}, which
is too verbose.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D136225
2022-11-02 12:41:47 -07:00
Dominik Adamski
25243d717d [mlir][OpenMP] Add aligned clause definition to simd construct
simd aligned construct is represented as pair of variable which needs
to be aligned and corresponding alignment value.

Added parser, printer and verifier of aligned clause. MLIR tests were
updated to test correctness of MLIR definition of aligned clause.

Differential Revision: https://reviews.llvm.org/D135865

Reviewed By: kiranchandramohan
2022-11-01 11:07:53 -05:00
Guray Ozen
3ac17449cf [mlir][nvvm] Introduce performance tuning directives
PTX programming models provides some performance tuning directives; see https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#performance-tuning-directives

The downstream compiler namely `ptxas` leverages these information for better register allocation or to handle other resource management that improves the performance.

This revision introduce all the kernel based directives to MLIR's NVVM dialect. The list is below
```
maxnreg			-> 	max register per thread in CTA
maxntid			-> 	max threads per CTA
reqntid			-> 	exact number of threads per CTA
minnctapersm		-> 	min CTA per SM
```

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136931
2022-10-28 14:02:40 +02:00
Tobias Gysi
606634967d [mlir][llvm] Tighten LLVM dialect intrinsic type constraints.
The revision specifies more precise argument and result type
constraints for many of the llvm intrinsics. Additionally, add
tests to verify intrinsics with invalid arguments/result result
in a verification error.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136360
2022-10-26 11:27:06 +03:00
River Riddle
9af92ed8a0 [mlir:LLVM] Rewrite the LLVMIR export to use the debug info attributes
This has been a long standing TODO, and actually enables users to generate
debug information for LLVM using the LLVM dialect; as opposed to our
dummy placeholder that generated just enough for line table information.

Differential Revision: https://reviews.llvm.org/D136543
2022-10-24 22:31:41 -07:00
Benjamin Maxwell
fc28971fb9 Add nocapture to pointer parameters of masked stores/loads
The lack of this attribute (particularly on the load intrinsics)
prevented InstCombine from optimizing away allocas and memcpys
for arrays that could be read directly from rodata.

This now also includes a new test to check the masked load/store
intrinsics have the expected attributes (specifically nocapture).

Differential Revision: https://reviews.llvm.org/D135656
2022-10-24 11:15:55 +00:00
Jeff Niu
8c8775e938 [mlir][llvm] Move LLVMArrayType to a TypeDef
This moves the `LLVMArrayType` to a `TypeDef`. The main side-effect of
this change is that the syntax `array<4xi32>` is no longer allowed. It
was previously parsed and then printed as `array<4 x i32>`. Now the
syntax must be the latter.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D136473
2022-10-21 12:57:16 -07:00
rkayaith
ed90f8026e [mlir-translate] Support parsing operations other than 'builtin.module' as top-level
This adds a '--no-implicit-module' option, which disables the insertion
of a top-level 'builtin.module' during parsing.

The translation APIs are also updated to take/return 'Operation*'
instead of 'ModuleOp', to allow other operation types to be used. To
simplify translations which are restricted to specific operation types,
'TranslateFromMLIRRegistration' has an overload which performs the
necessary cast and error checking.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D134237
2022-10-21 15:54:06 -04:00
Victor Perez
70e3f0e10e [mlir][llvm] Handle llvm.noundef attribute when converting to LLVM IR
Translate LLVMIR llvm.noundef attribute to its equivalent in LLVM IR.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136324
2022-10-20 12:59:47 +01:00
Tobias Gysi
3883615906 [mlir][llvm] Ordered traversal in LLVM IR import.
The revision performs a topological sort of the blocks to
ensure the operations are processed in dominance order.
After the change, we do not need to introduce dummy
instructions if an operand has not yet been processed.
Additionally, the revision also moves and simplifies the
control-flow related tests to a separate test file.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136230
2022-10-19 12:57:09 +03:00
Tobias Gysi
b3ed69d37c [mlir][llvm] Import matrix, vector, and assume intrinsics from LLVM IR.
The revision adds support to import:
- matrix intrinsics
- vector reduce fadd/fmul intrinsics
- assume intrinsics
from LLVM IR.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136137
2022-10-18 11:11:03 +03:00
Jeremy Furtek
dd38f89980 [mlir][LLVMIR] Update LLVMIR fastmath to use EnumAttr tblgen classes
This diff updates the `fastmath` attribute in the LLVMIR dialect to use `tblgen`
classes that were developed after the initial LLVMIR `fastmath` implementation.
Using the `EnumAttr` `tblgen` classes brings the LLVMIR `fastmath` attribute in
line with other dialects, and eliminates some of the custom printing and parsing
code in the LLVMIR dialect.

Subsequent commits will further reduce the custom processing code for the LLVMIR
`fastmath` attribute by unifying printing/parsing functionality between the
LLVMIR and `arith` `fastmath` attributes. (The actual attributes will remain
separate, but the printing and parsing will be made generic, and will be usable
by other dialects/attributes.)

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135289
2022-10-17 15:03:47 -07:00
Tobias Gysi
5942456a30 [mlir][llvm] Add support for importing masked intrinsics from LLVM IR.
The revision adds support for importing the masked load/store and
gather/scatter intrinsics from LLVM IR. To enable the import, the
revision also includes an extension of the mlirBuilder code generation
to support variadic arguments.

Depends on D136057

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D136058
2022-10-17 15:53:47 +03:00
Jeff Niu
7c3439b74c [mlir][llvm] Add llvm.intr.is.fpclass
This intrinsic allows testing for inf, nan, etc. on floating point
values.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D135969
2022-10-14 16:22:59 -07:00
Tobias Gysi
9f13900371 [mlir][llvm] Use tablegen to import atomic ops from LLVM IR.
The revision imports the atomic operations using
tablegen generated builders. Additionally, it moves their tests to
the instructions.ll test file.

Depends on D135880

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135944
2022-10-14 15:25:55 +03:00
Tobias Gysi
ea9cae03d7 [mlir][llvm] Use tablegen to import shufflevector from LLVM IR.
The revision imports the shuffle vector operation using
tablegen generated builders. Additionally, it moves its test to
the instructions.ll test file.

Depends on D135874

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135880
2022-10-14 14:46:05 +03:00
Tobias Gysi
7114185f4e [mlir][llvm] Use tablegen to import extract/insert ops from LLVM IR.
The revision imports the extract and insert value operations using
tablegen generated builders. Additionally, it moves the tests to
the instructions.ll test file.

Reviewed By: ftynse, dcaballe

Differential Revision: https://reviews.llvm.org/D135874
2022-10-14 10:09:21 +03:00
Tobias Gysi
cc49a74a7b [mlir][llvm] Use TableGen to import compare ops from LLVM IR.
The revision imports compare operations using TableGen generated
builders, instead of using the special handlers defined by the Importer.
It therefore adds a new llvmArgIndexes field that allows to specify
a mapping between MLIR argument and LLVM IR operand indexes if they do
not match. Additionally, the FCmp op is extended with an additional
builder and all compare operations are extended with verification
traits to ensure the operands types match. These extensions simplify
the logic of the newly introduced builders and are in line with the
compare operations define by the arithmetic dialect.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135855
2022-10-13 15:31:04 +03:00
Tobias Gysi
8446f24ef0 [mlir][llvm] Tablegen based operation import from LLVM IR.
The revision uses tablegen generated builders to convert the most common
LLVM IR instructions to MLIR LLVM dialect operations. All instructions
with special handlers, except for alloca and fence, still use manual
handlers. The revision also introduces an additional "instructions.ll"
test file to test the import of instructions that have tablegen builders
(except for the resume instruction whose test remains untouched). A part
of the test cases are new, for example the integer instruction test,
while others are migrated from the "basic.ll" test file.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135709
2022-10-12 14:36:41 +03:00
Tobias Gysi
d85f6e5d57 [mlir][llvmir] Import intrinsics with attributes from LLVMIR.
The revision adds support to specify custom import functions for
LLVM IR intrinsics with immediate arguments that translate to MLIR
attributes. It takes an approach similar to the MLIR to LLVM translation
that uses a tablegen defined build method. The default implementation
of this newly introduced "mlirBuilder" assumes all intrinsic arguments
translate to operands. Specific intrinsics, such as
llvm.lifetime.start/stop then define a custom builder that converts
their immediate arguments to MLIR attributes.

Depends on D135349

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135350
2022-10-07 14:30:30 +03:00
Tobias Gysi
12b68ad886 [mlir][llvmir] Add filename debug info when converting from LLVMIR.
The revision enriches the debug locations generated during LLVMIR to MLIR translation with file name information and adds a separate test to exercise the debug location translation.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D135069
2022-10-03 15:14:37 +03:00
Min-Yih Hsu
ddc67856ee [mlir][LLVMIR] Add translation of AtomicRMW/CmpXchg from LLVM IR
Add support for translating AtomicRMWInst and AtomicCmpXchgInst from
LLVM IR.

Differential Revision: https://reviews.llvm.org/D134450
2022-09-26 15:07:30 -07:00
Jakub Kuderski
ce82530cd0 Revert "[mlir][spirv] Change td def/class prefix from SPV to SPIRV"
This reverts commit a2052b8794cb5abac131cd62f68505eebcfaffcb.

This commit renamed some Vulkan identifiers that shouldn't have been
renamed, e.g., `SPV_KHR_storage_buffer_storage_class`.
2022-09-26 12:40:35 -04:00
Jakub Kuderski
a2052b8794 [mlir][spirv] Change td def/class prefix from SPV to SPIRV
Tested with `check-mlir` and `check-mlir-integration`.

Fixes: https://github.com/llvm/llvm-project/issues/56863

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D134649
2022-09-26 11:42:49 -04:00
Jakub Kuderski
5ab6ef758f [mlir][spirv] Change dialect name from 'spv' to 'spirv'
Tested with `check-mlir` and `check-mlir-integration`.

Issue: https://github.com/llvm/llvm-project/issues/56863

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D134620
2022-09-26 10:58:30 -04:00
Min-Yih Hsu
006a752a3c [mlir][LLVMIR] Do not create pseudo debug file name using llvm::Instruction
Previously in mlir-translate, if debug info was absent in a
llvm::Instruction, we tried to create one using the name of its defined
value in a textual LLVM IR file as the (pseudo) debug file name.
However, in order to get that name, we need to call out to LLVM's
SlotTracker, which, surprisingly, took a lot of time. Judging from
the usefulness of such pseudo debug file name and the performance penalty
during translation, this patch simply use "imported-bitcode" as the
debug file name in these case. Eliminating the need of using (expensive)
LLVM value numbering.

Differential Revision: https://reviews.llvm.org/D134305
2022-09-21 09:46:58 -07:00