67 Commits

Author SHA1 Message Date
Jakub Kuderski
08425def7d
[mlir][spirv] Improve coop matrix attribute handling (#66020)
- Fix values of Matrix Operand bit enums.
- Add verification for the aligned Memory Operand attributes. Mark the
'Aligned' enumerant as not supported.

The target test passes validation with `spirv-val`.
2023-09-12 11:05:08 -04:00
Jakub Kuderski
1258c3f214
[mlir][spirv] Support spirv.coopmatrix type (de-)serialization (#65831)
Extend SPIR-V target serialization and deserialization to handle coop
matrix types. Add a roundtrip test. In addition to `FileCheck` checks,
the resulting spirv binary also passes `spir-val` (external tool).

Also fix a type attribute bug surfaced by the `CooperativeMatrixLength`
op.

Multiple matrix operand attributes will be handled in a future patch to
reduce the scope.
2023-09-11 12:23:18 -04:00
Jakub Kuderski
1d51597870 [mlir][spirv] Add missing NV prefix/suffix for coop matrix
This is in preparation for adding the KHR version of the cooperative
matrix extension, `SPV_KHR_cooperative_matrix`, that comes with
equivalent ops and type. These are not cross-extension compatible,
so it's better to add prefixes/suffixes to the Nvidia one,
`SPV_NV_cooperative_matrix`, before adding the KHR counterparts.

In near future, I plan for these two extensions to co-exist in
the SPIR-V dialect, but we may want to remove the NV one at some point.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D154799
2023-07-10 10:06:24 -04:00
Md Abdullah Shahneous Bari
2dab551d64 [mlir][spirv] Change numeric constant's bit-extension decision to be based on type
Integer constants with bit width less than a word (e.g., i8, i16)
should be bit extended based on its type to be SPIR-V spec-compliant.
Previously, the decision was based on the most significant bit of the
value which ignores the signless semantics and causes problems when
interfacing with SPIR-V tools.

Dealing with numeric literals: the SPIR-V spec says, "If a numeric
type’s bit width is less than 32-bits, the value appears in the
low-order bits of the word, and the high-order bits must be 0 for
a floating-point type or integer type with Signedness of 0, or sign
extended for an integer type with a Signedness of 1 (similarly for the
remaining bits of widths larger than 32 bits but not a multiple of 32
bits)."

Therefore, signless integers (e.g., i8, i16) and unsigned integers
should be 0-extended, and signed integers (e.g., si8, si16) should be
sign-extended.

Patch By: mshahneo
Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D151767
2023-06-05 10:52:09 -04:00
Md Abdullah Shahneous Bari
edfd360d78 [mlir][spirv] Add OpConvertPtrToU and OpConvertUToPtr ops
Add the definitions and necessary verifications of these Ops.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D151074
2023-05-24 21:35:30 -07:00
Md Abdullah Shahneous Bari
6d578669f3 [mlir][spirv] Support import/export Functions and GlobalVariables
"LinkageAttributes" decoration allow a SPIR-V module to import
external functions and global variables, or export functions or
global variables for other SPIR-V modules to link against and use.

Import/export capability is extremely important when using outside
libraries (e.g., intrinsic libraries).

Added decorations:
- LinkageAttributes

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D148749
2023-05-24 17:08:46 -07:00
Md Abdullah Shahneous Bari
d2a559ffc0 [mlir][spirv] Add OpExtension "SPV_INTEL_bfloat16_conversion"
Add Intel-specific "SPV_INTEL_bfloat16_conversion" extension and
capability (Bfloat16ConversionINTEL), and
two ops (OpConvertFToBF16INTEL, OpConvertBF16ToFINTEL)
that are introduced by this extension.
These ops allow BF16 to Float conversion and vice-versa.

Reference Specification:
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/INTEL/SPV_INTEL_bfloat16_conversion.asciidoc

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D147087
2023-03-31 14:02:59 -07:00
Fabian
09fd9ef4f4 [mlir] Execute all requested translations in MlirTranslateMain
Currently, MlirTranslateMain only executes one of the requested translations, and does not error if multiple are specified. This commit enables translations to be chained in the specified order.

This makes round-trip tests easier, since existing import/export passes can be reused and no combined round-trip passes have to be registered (example: mlir-translate  -serialize-spirv -deserialize-spirv).

Additionally, by leveraging TranslateRegistration with file-to-file TranslateFunctions, generic pre- and post-processing can be added before/after conversion to/from MLIR.

Reviewed By: lattner, Mogball

Differential Revision: https://reviews.llvm.org/D143719
2023-02-18 21:31:59 +01:00
Benjamin Chetioui
a6c8f06f55 [mlir] Clean up typos in FileCheck directives in various tests.
Reviewed By: tpopp

Differential Revision: https://reviews.llvm.org/D139698
2022-12-12 09:29:14 +01:00
Lei Zhang
3c278e5e27 [mlir][spirv] Fix spirv.MatrixTimesScalar for cooperative matrix
spirv.MatrixTimesScalar is allowed to use cooperative matrix.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D139279
2022-12-05 22:13:23 +00:00
Ivan Butygin
339c87a8a0 [mlir][spirv] Add group ops
Also, gen_spirv_dialect script was broken was broken due to SPV->SPIRV renamings.

Differential Revision: https://reviews.llvm.org/D138756
2022-11-29 15:03:31 +01:00
Hanhan Wang
0a1569a400 [mlir][NFC] Remove trailing whitespaces from *.td and *.mlir files.
This is generated by running

```
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.td
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.mlir
```

Reviewed By: rriddle, dcaballe

Differential Revision: https://reviews.llvm.org/D138866
2022-11-28 15:26:30 -08:00
rkayaith
e4889c0a04 [mlir][spirv] Change translation to use spirv.module
Update the SPIRV `mlir-translate` translations to translate to/from
`spirv.module` instead of `builtin.module`. This simplifies the
translation since the code no longer needs to walk the module looking
for a SPIRV module, however it requires passing `-no-implicit-module` to
all the tests.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D135819
2022-11-08 14:47:01 -05:00
rkayaith
ed90f8026e [mlir-translate] Support parsing operations other than 'builtin.module' as top-level
This adds a '--no-implicit-module' option, which disables the insertion
of a top-level 'builtin.module' during parsing.

The translation APIs are also updated to take/return 'Operation*'
instead of 'ModuleOp', to allow other operation types to be used. To
simplify translations which are restricted to specific operation types,
'TranslateFromMLIRRegistration' has an overload which performs the
necessary cast and error checking.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D134237
2022-10-21 15:54:06 -04:00
Jakub Kuderski
ce82530cd0 Revert "[mlir][spirv] Change td def/class prefix from SPV to SPIRV"
This reverts commit a2052b8794cb5abac131cd62f68505eebcfaffcb.

This commit renamed some Vulkan identifiers that shouldn't have been
renamed, e.g., `SPV_KHR_storage_buffer_storage_class`.
2022-09-26 12:40:35 -04:00
Jakub Kuderski
a2052b8794 [mlir][spirv] Change td def/class prefix from SPV to SPIRV
Tested with `check-mlir` and `check-mlir-integration`.

Fixes: https://github.com/llvm/llvm-project/issues/56863

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D134649
2022-09-26 11:42:49 -04:00
Jakub Kuderski
5ab6ef758f [mlir][spirv] Change dialect name from 'spv' to 'spirv'
Tested with `check-mlir` and `check-mlir-integration`.

Issue: https://github.com/llvm/llvm-project/issues/56863

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D134620
2022-09-26 10:58:30 -04:00
Nirvedh Meshram
7f19e59a6e [mlir][spirv] Add casting ops to/from generic storage space
Reviwed By: antiagainst

Differential Revision: https://reviews.llvm.org/D134217
2022-09-20 14:57:06 -07:00
Stanley Winata
7d23d1e640 [mlir][spirv] Lower arith max/min ops to OpenCL ones
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D132881
2022-09-19 13:34:09 -04:00
Jakub Kuderski
817de304d5 [mlir][spirv] Change vendor op mnemonics to spv.VENDOR.name
Make vendor ops more consistent with the naming scheme within the SPIR-V
dialect.

Issue: https://github.com/llvm/llvm-project/issues/56863

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D133247
2022-09-06 13:35:08 -04:00
Lei Zhang
4c726d3bbf [mlir][spirv] Define various spv.GroupNonUniformShuffle ops
Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D133041
2022-09-02 17:07:16 -04:00
Nirvedh Meshram
6b8952554c [mlir][spirv] Modify intel joint matrix ops
Adding more supported storage classes in pointer type
Removing support for arithmetic ops as this is not available on the tensor hardware

Reviwed By: antiagainst

Differential Revision: https://reviews.llvm.org/D132039
2022-08-17 18:40:59 +00:00
Nirvedh Meshram
b8f62dc22a [MLIR][SPIRV] Add intel joint matrix ops
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D131586
2022-08-15 23:49:45 +00:00
Lei Zhang
6f4f9e316c [mlir][spirv] Add a test covering load/store with memory operands
Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131904
2022-08-15 17:48:33 -04:00
Lei Zhang
a29fffc475 [mlir][spirv] Migrate to use specalized enum attributes
Previously we are using IntegerAttr to back all SPIR-V enum
attributes. Therefore we all such attributes are showed like
IntegerAttr in IRs, which is barely readable and breaks
roundtripability of the IR. This commit changes to use
`EnumAttr` as the base directly so that we can have separate
attribute definitions and better IR printing.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131311
2022-08-09 14:14:54 -04:00
Lei Zhang
7668e58210 [mlir][spirv] Fix spv.CompositeConstruct assembly and validation
This commit fixes spv.CompositeConstruct to assembly to list
operand types to enable vector construction out of smaller vectors.
Validation is also fixed to properly check the cases for vector
construction.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D130669
2022-07-27 19:17:23 -04:00
Jakub Kuderski
52b630da02 [mlir][spirv] Rename spv.GLSL ops to spv.GL. NFC.
This is to improve consistency within the SPIR-V dialect and make these ops a bit shorter.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D130280
2022-07-21 13:03:30 -04:00
Jakub Kuderski
3930cc6847 [mlir][spirv] Rename spv.ocl to spv.cl. NFC.
This is to improve the consistency within the SPIR-V dialect and to make op names a bit shorter.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D130194
2022-07-20 17:46:45 -04:00
Lei Zhang
cc020a2236 [mlir][spirv] Convert math.ctlz to spv.GLSL.FindUMsb
Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D127582
2022-06-13 13:02:37 -04:00
Chengji Yao
e5a4cf6743 [mlir] Fix printer when it is a DenseElementsAttr of i1
A large DenseElementsAttr of i1could trigger a bug in printer/parser roundtrip.

Ex. A DenseElementsAttr of i1 with 200 elements will print as Hex format of length 400 before the fix. However, when parsing the printed text, an error will be triggered. After fix, the printed length will be 50.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D122925
2022-05-05 16:56:43 -07:00
Lei Zhang
cfb9e474ae [mlir][spirv] Define spv.VectorTimesScalar op
Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D121247
2022-03-08 15:58:31 -05:00
Ivan Butygin
32389d0c2e [mlir][spirv] Add OpenCL fma op and lowering
Also, it seems Khronos has changed html spec format so small adjustment to script was needed.
Base op parsing is also probably broken.

Differential Revision: https://reviews.llvm.org/D119678
2022-02-15 11:28:20 +03:00
Lei Zhang
5e55a20119 [mlir][spirv] Serialize selection with separate header block
The previous "optimization" that tries to reuse existing block for
selection header block can be problematic for deserialization
because it effectively pulls in previous ops in the selection op's
enclosing block into the selection op's header. When deserializing,
those ops will be placed in the selection op's region. If any of
the previous ops has usage after the section op, it will break. That
is, the following IR cannot round trip:

```mlir
^bb:
  %def = ...
  spv.mlir.selection { ... }
  %use = spv.SomeOp %def
```

This commit removes the "optimization" to always create new blocks
for the selection header.

Along the way, also made error reporting better in deserialization
by turning asserts into proper errors and add check of uses outside
of sinked structured control flow region blocks.

Reviewed By: Hardcode84

Differential Revision: https://reviews.llvm.org/D115582
2021-12-13 10:42:26 -05:00
Lei Zhang
731676b10d [mlir][spirv] Fix nested control flow serialization
If we have a `spv.mlir.selection` op nested in a `spv.mlir.loop`
op, when serializing the loop's block, we might need to jump
from the selection op's merge block, which might be different
than the immediate MLIR IR predecessor block. But we still need
to get the block argument from the MLIR IR predecessor block.

Also, if the `spv.mlir.selection` is in the `spv.mlir.loop`'s
header block, we need to make sure `OpLoopMerge` is emitted
in the current block before start processing the nested selection
op. Otherwise we'll see the LoopMerge in the wrong SPIR-V
basic block.

Reviewed By: Hardcode84

Differential Revision: https://reviews.llvm.org/D115560
2021-12-11 14:47:19 -05:00
Butygin
d8fce785de [mlir][spirv] math.erf OpenCL lowering
Differential Revision: https://reviews.llvm.org/D115335
2021-12-08 21:59:46 +03:00
Butygin
75a1bee05d [mlir][spirv] Add math to OpenCL conversion
Differential Revision: https://reviews.llvm.org/D113780
2021-11-24 02:31:21 +03:00
Butygin
6c48f6aafe [mlir][spirv] add AtomicFAddEXTOp
Differential Revision: https://reviews.llvm.org/D113764
2021-11-16 14:24:22 +03:00
Butygin
1cb13fddb9 [mlir] spirv: Add some atomic ops
Differential Revision: https://reviews.llvm.org/D112812
2021-11-03 14:47:12 +03:00
Caitlyn Cano
2ea5e7ba57 [mlir] SPIR-V: add sin, cos, log, sqrt OCL ops
Differential Revision: https://reviews.llvm.org/D111884
2021-10-18 20:48:59 +03:00
Lei Zhang
7a89444cd9 [mlir][spirv] Add ops and patterns for lowering standard max/min ops
Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D111143
2021-10-05 14:27:32 -04:00
Weiwei Li
1e4cfe5e4f [mlir][SPIRVToLLVM] Propagate location attribute from spv.GlobalVariable to llvm.mlir.global
This patch is mainly to propogate location attribute from spv.GlobalVariable to llvm.mlir.global.

It also contains three small changes.

1. Remove the restriction on UniformConstant In SPIRVToLLVM.cpp;
2. Remove the errorCheck on relaxedPrecision when deserializering SPIR-V in Deserializer.cpp
3. In SPIRVOps.cpp, let ConstantOp take signedInteger too.

Co-authered: Alan Liu <alanliu.yf@gmail.com> and Xinyi Liu <xyliuhelen@gmail.com>

Reviewed by:antiagainst

Differential revision: https://reviews.llvm.org/D110207
2021-10-05 00:09:09 +08:00
Lei Zhang
233a3a75ac [mlir][spirv] Add support for i8 serialization
Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D107122
2021-07-30 18:06:11 -04:00
Weiwei Li
108a320a58 [mlir][spirv] Add support for GLSL FMix
Add spv.GLSL.FMix opertaion.

co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D104153
2021-07-16 08:29:46 +08:00
Weiwei Li
cd0eeb52ad [mlir][spirv] Define spv.ImageQuerySize operation
Support OpImageQuerySize in spirv dialect

co-authored-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D102029
2021-05-13 13:17:08 -04:00
KareemErgawy-TomTom
e4dee7e730 [MLIR][SPIRV] Properly (de-)serialize BranchConditionalOp.
Implements proper (de-)serialization logic for BranchConditionalOp when
such ops have true/false target operands.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D101602
2021-05-07 09:00:50 +02:00
Weiwei Li
12ffc26067 [mlir][spirv] Define spv.ImageDrefGather operation
This patch doesn't support the optional operands of ImageDrefGather. The support of optional operands will be implemented later.

co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Differential Revision: https://reviews.llvm.org/D100128
2021-04-08 20:15:54 -04:00
Weiwei Li
619c1505f9 [mlir][spirv] Define spv.Image Operation
co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D98270
2021-03-10 15:48:04 -05:00
KareemErgawy-TomTom
3fb384d50e [MLIR][SPIRV] Rename spv.selection to spv.mlir.selection.
To unify the naming scheme across all ops in the SPIR-V dialect, we are
moving from spv.camelCase to spv.CamelCase everywhere. For ops that
don't have a SPIR-V spec counterpart, we use spv.mlir.snake_case.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D98014
2021-03-06 16:05:31 +01:00
KareemErgawy-TomTom
d48ceb45e3 [MLIR][SPIRV] Rename spv.undef to spv.Undef.
To unify the naming scheme across all ops in the SPIR-V dialect, we are
moving from spv.camelCase to spv.CamelCase everywhere. For ops that
don't have a SPIR-V spec counterpart, we use spv.mlir.snake_case.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D98016
2021-03-05 15:49:44 -05:00
KareemErgawy-TomTom
29812a6195 [MLIR][SPIRV] Rename spv.loop to spv.mlir.loop.
To unify the naming scheme across all ops in the SPIR-V dialect,
we are moving from spv.camelCase to spv.CamelCase everywhere.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D97918
2021-03-05 15:44:30 -05:00