448 Commits

Author SHA1 Message Date
River Riddle
195730a650 [mlir][NFC] Replace references to Identifier with StringAttr
This is part of the replacement of Identifier with StringAttr.

Differential Revision: https://reviews.llvm.org/D113953
2021-11-16 17:36:26 +00:00
Butygin
6c48f6aafe [mlir][spirv] add AtomicFAddEXTOp
Differential Revision: https://reviews.llvm.org/D113764
2021-11-16 14:24:22 +03:00
Butygin
92fc60bc62 [mlir][spirv] Regenerate SPIRVBase.td from recent spec
* Some long names were added and script decided to change whitespaces in a lot of places
* `ImageOperand` was renamed to `ImageOperands` in spec
* Some *NV enums were renamed to *KHR (spec actually maintains both variants with same value, but script pulled only *KHR versions)

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D113667
2021-11-11 17:07:52 -05:00
River Riddle
ae40d62541 [mlir] Refactor ElementsAttr's value access API
There are several aspects of the API that either aren't easy to use, or are
deceptively easy to do the wrong thing. The main change of this commit
is to remove all of the `getValue<T>`/`getFlatValue<T>` from ElementsAttr
and instead provide operator[] methods on the ranges returned by
`getValues<T>`. This provides a much more convenient API for the value
ranges. It also removes the easy-to-be-inefficient nature of
getValue/getFlatValue, which under the hood would construct a new range for
the type `T`. Constructing a range is not necessarily cheap in all cases, and
could lead to very poor performance if used within a loop; i.e. if you were to
naively write something like:

```
DenseElementsAttr attr = ...;
for (int i = 0; i < size; ++i) {
  // We are internally rebuilding the APFloat value range on each iteration!!
  APFloat it = attr.getFlatValue<APFloat>(i);
}
```

Differential Revision: https://reviews.llvm.org/D113229
2021-11-09 00:15:08 +00:00
Butygin
1cb13fddb9 [mlir] spirv: Add some atomic ops
Differential Revision: https://reviews.llvm.org/D112812
2021-11-03 14:47:12 +03:00
River Riddle
015192c634 [mlir:DialectConversion] Restructure how argument/target materializations get invoked
The current implementation invokes materializations
whenever an input operand does not have a mapping for the
desired type, i.e. it requires materialization at the earliest possible
point. This conflicts with goal of dialect conversion (and also the
current documentation) which states that a materialization is only
required if the materialization is supposed to persist after the
conversion process has finished.

This revision refactors this such that whenever a target
materialization "might" be necessary, we insert an
unrealized_conversion_cast to act as a temporary materialization.
This allows for deferring the invocation of the user
materialization hooks until the end of the conversion process,
where we actually have a better sense if it's actually
necessary. This has several benefits:

* In some cases a target materialization hook is no longer
   necessary
When performing a full conversion, there are some situations
where a temporary materialization is necessary. Moving forward,
these users won't need to provide any target materializations,
as the temporary materializations do not require the user to
provide materialization hooks.

* getRemappedValue can now handle values that haven't been
   converted yet
Before this commit, it wasn't well supported to get the remapped
value of a value that hadn't been converted yet (making it
difficult/impossible to convert multiple operations in many
situations). This commit updates getRemappedValue to properly
handle this case by inserting temporary materializations when
necessary.

Another code-health related benefit is that with this change we
can move a majority of the complexity related to materializations
to the end of the conversion process, instead of handling adhoc
while conversion is happening.

Differential Revision: https://reviews.llvm.org/D111620
2021-10-27 02:09:04 +00:00
xndcn
8c1553f0d7 [mlir][spirv] Add memory semantics verify for atomic operations
Differential Revision: https://reviews.llvm.org/D111510
2021-10-14 00:00:55 +08:00
Lei Zhang
83e074a0c6 [mlir] Add an 'cppNamespace' field to availability
This allows us to generate interfaces in a namespace,
following other TableGen'erated code.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108311
2021-10-05 09:38:09 -04:00
Weiwei Li
1e4cfe5e4f [mlir][SPIRVToLLVM] Propagate location attribute from spv.GlobalVariable to llvm.mlir.global
This patch is mainly to propogate location attribute from spv.GlobalVariable to llvm.mlir.global.

It also contains three small changes.

1. Remove the restriction on UniformConstant In SPIRVToLLVM.cpp;
2. Remove the errorCheck on relaxedPrecision when deserializering SPIR-V in Deserializer.cpp
3. In SPIRVOps.cpp, let ConstantOp take signedInteger too.

Co-authered: Alan Liu <alanliu.yf@gmail.com> and Xinyi Liu <xyliuhelen@gmail.com>

Reviewed by:antiagainst

Differential revision: https://reviews.llvm.org/D110207
2021-10-05 00:09:09 +08:00
Mehdi Amini
2da3facd86 Fix memory leak in MLIR SPIRV ModuleCombiner 2021-10-02 23:55:25 +00:00
Chris Lattner
fb093c8314 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Recommit 4b32f8bac4 after fixing a dependency.

Differential Revision: https://reviews.llvm.org/D110796
2021-09-30 05:10:28 +00:00
Mehdi Amini
3310e0020c Revert "[ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser."
This reverts commit 4b32f8bac40dcd1535bfe95757c3de0911bf6d1a.

Seems like the build is broken with -DDBUILD_SHARED_LIBS=ON
2021-09-30 05:01:17 +00:00
Chris Lattner
4b32f8bac4 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Differential Revision: https://reviews.llvm.org/D110796
2021-09-29 21:36:05 -07:00
River Riddle
b54c724be0 [mlir:OpConversionPattern] Add overloads for taking an Adaptor instead of ArrayRef
This has been a TODO for a long time, and it brings about many advantages (namely nice accessors, and less fragile code). The existing overloads that accept ArrayRef are now treated as deprecated and will be removed in a followup (after a small grace period). Most of the upstream MLIR usages have been fixed by this commit, the rest will be handled in a followup.

Differential Revision: https://reviews.llvm.org/D110293
2021-09-24 17:51:41 +00:00
Chris Lattner
58abc8c34b [OpAsmParser] Add a parseCommaSeparatedList helper and beef up Delimeter.
Lots of custom ops have hand-rolled comma-delimited parsing loops, as does
the MLIR parser itself.  Provides a standard interface for doing this that
is less error prone and less boilerplate.

While here, extend Delimiter to support <> and {} delimited sequences as
well (I have a use for <> in CIRCT specifically).

Differential Revision: https://reviews.llvm.org/D110122
2021-09-20 20:59:11 -07:00
Weiwei Li
a79d7c2c85 [mlir][SPIRV] Add Image Operands for Image Instructions
This patch is to add Image Operands in SPIR-V Dialect and also let ImageDrefGather to use Image Operands.

Image Operands are used in many image instructions. "Image Operands encodes what oprands follow, as per Image Operands". And ususally, they are optional to image instructions.

The format of image operands looks like:

    %0 = spv.ImageXXXX %1, ... %3 : f32 ["Bias|Lod"](%4, %5 : f32, f32) -> ...

This patch doesn’t implement all operands (see Section 3.14 in SPIR-V Spec) but provides a skeleton of it. There is TODO in verifyImageOperands function.

Co-authored: Alan Liu <alanliu.yf@gmail.com>

Reviewed by: antiagainst

Differential Revision: https://reviews.llvm.org/D108501
2021-09-02 04:14:17 +08:00
Mehdi Amini
c41b16c26b Change ASM Op printer to print the operation name in the framework instead of leaving it up to each individual operation
This aligns the printer with the parser contract: the operation isn't part of the user-controllable part of the syntax.

Differential Revision: https://reviews.llvm.org/D108804
2021-08-31 17:52:40 +00:00
Chris Lattner
faf1c22408 [Builder] Eliminate the StringRef/StringAttr forms of getSymbolRefAttr.
The StringAttr version doesn't need a context, so we can just use the
existing `SymbolRefAttr::get` form.  The StringRef version isn't preferred
so we want to encourage people to use StringAttr.

There is an additional form of getSymbolRefAttr that takes a (SymbolTrait
implementing) operation.  This should also be moved, but I'll do that as
a separate patch.

Differential Revision: https://reviews.llvm.org/D108922
2021-08-30 16:05:36 -07:00
Chris Lattner
41d4aa7de6 [SymbolRefAttr] Revise SymbolRefAttr to hold a StringAttr.
SymbolRefAttr is fundamentally a base string plus a sequence
of nested references.  Instead of storing the string data as
a copies StringRef, store it as an already-uniqued StringAttr.

This makes a lot of things simpler and more efficient because:
1) references to the symbol are already stored as StringAttr's:
   there is no need to copy the string data into MLIRContext
   multiple times.
2) This allows pointer comparisons instead of string
   comparisons (or redundant uniquing) within SymbolTable.cpp.
3) This allows SymbolTable to hold a DenseMap instead of a
   StringMap (which again copies the string data and slows
   lookup).

This is a moderately invasive patch, so I kept a lot of
compatibility APIs around.  It would be nice to explore changing
getName() to return a StringAttr for example (right now you have
to use getNameAttr()), and eliminate things like the StringRef
version of getSymbol.

Differential Revision: https://reviews.llvm.org/D108899
2021-08-29 21:54:47 -07:00
Lei Zhang
a5621e26db [mlir][spirv] Use type dyn_cast when scanning spv.GlobalVariable
This avoids crashes when there are spv.GlobalVariable without
pointer type.
2021-08-29 12:01:19 -04:00
Butygin
1e35a7690d [mlir][spirv] Initial support for 64 bit index type and builtins
Differential Revision: https://reviews.llvm.org/D108516
2021-08-27 01:38:53 +03:00
Butygin
ddc3d51d58 [mlir][spirv] Add (InBounds)PtrAccessChain ops
Differential Revision: https://reviews.llvm.org/D108070
2021-08-18 17:59:21 +03:00
Lei Zhang
23326b9f17 [mlir][spirv] Fix a few issues in ModuleCombiner
- Fixed symbol insertion into `symNameToModuleMap`. Insertion
  needs to happen whether symbols are renamed or not.
- Added check for the VCE triple and avoid dropping it.
- Disabled function deduplication. It requires more careful
  rules. Right now it can remove different functions.
- Added tests for symbol rename listener.
- And some other code/comment cleanups.

Reviewed By: ergawy

Differential Revision: https://reviews.llvm.org/D106886
2021-07-28 10:31:01 -04:00
Stella Laurenzo
485cc55edf [mlir] Generare .cpp.inc files for dialects.
* Previously, we were only generating .h.inc files. We foresee the need to also generate implementations and this is a step towards that.
* Discussed in https://llvm.discourse.group/t/generating-cpp-inc-files-for-dialects/3732/2
* Deviates from the discussion above by generating a default constructor in the .cpp.inc file (and adding a tablegen bit that disables this in case if this is user provided).
* Generating the destructor started as a way to flush out the missing includes (produces a link error), but it is a strict improvement on its own that is worth doing (i.e. by emitting key methods in the .cpp file, we root vtables in one translation unit, which is a non-controversial improvement).

Differential Revision: https://reviews.llvm.org/D105070
2021-06-29 20:10:30 +00:00
Lei Zhang
56f60a1ce7 [mlir][spirv] Use SingleBlock + NoTerminator for spv.module
This allows us to remove the `spv.mlir.endmodule` op and
all the code associated with it.

Along the way, tightened the APIs for `spv.module` a bit
by removing some aliases. Now we use `getRegion` to get
the only region, and `getBody` to get the region's only
block.

Reviewed By: mravishankar, hanchung

Differential Revision: https://reviews.llvm.org/D103265
2021-06-09 14:00:06 -04:00
KareemErgawy
2def12ebc6 [MLIR][SPIRV] Use getAsmResultName(...) hook for AddressOfOp.
Implements better naming for results of spv.mlir.addressof ops by making it
inherit from OpAsmOpInterface and implementing the associated
getAsmResultName(...) hook.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D103594
2021-06-07 13:58:26 +02:00
KareemErgawy
e493abcf55 [MLIR][SPIRV] Use getAsmResultName(...) hook for ConstantOp.
Implements better naming for results of `spv.Constant` ops by making it
inherit from OpAsmOpInterface and implementing the associated
getAsmResultName(...) hook.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D103152
2021-05-28 09:28:02 +02:00
Weiwei Li
cd0eeb52ad [mlir][spirv] Define spv.ImageQuerySize operation
Support OpImageQuerySize in spirv dialect

co-authored-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D102029
2021-05-13 13:17:08 -04:00
River Riddle
53b946aa63 [mlir] Refactor the representation of function-like argument/result attributes.
The current design uses a unique entry for each argument/result attribute, with the name of the entry being something like "arg0". This provides for a somewhat sparse design, but ends up being much more expensive (from a runtime perspective) in-practice. The design requires building a string every time we lookup the dictionary for a specific arg/result, and also requires N attribute lookups when collecting all of the arg/result attribute dictionaries.

This revision restructures the design to instead have an ArrayAttr that contains all of the attribute dictionaries for arguments and another for results. This design reduces the number of attribute name lookups to 1, and allows for O(1) lookup for individual element dictionaries. The major downside is that we can end up with larger memory usage, as the ArrayAttr contains an entry for each element even if that element has no attributes. If the memory usage becomes too problematic, we can experiment with a more sparse structure that still provides a lot of the wins in this revision.

This dropped the compilation time of a somewhat large TensorFlow model from ~650 seconds to ~400 seconds.

Differential Revision: https://reviews.llvm.org/D102035
2021-05-07 19:32:31 -07:00
Lei Zhang
41bc54cc56 [mlir][spirv] NFC: Replace OwningSPIRVModuleRef with OwningOpRef
Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D102009
2021-05-06 17:17:44 -04:00
Nico Weber
297a5b7cbc [mlir] hopefully final round of iwyu fixes after ba7a92c01e8 2021-04-21 11:03:06 -04:00
River Riddle
4efb7754e0 [mlir][NFC] Add a using directive for llvm::SetVector
Differential Revision: https://reviews.llvm.org/D100436
2021-04-15 16:09:34 -07:00
Hanhan Wang
7c4de2e9b9 [mlir][StandardToSPIRV] Add support for lowering memref<?xi1> to SPIR-V
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D100452
2021-04-14 07:22:49 -07:00
Lei Zhang
5b15fe9334 [mlir][spirv] Only attach struct offset for required storage classes
Per the SPIR-V spec "2.16.2. Validation Rules for Shader Capabilities":

  Composite objects in the StorageBuffer, PhysicalStorageBuffer,
  Uniform, and PushConstant Storage Classes must be explicitly
  laid out.

For other cases we don't need to attach the struct offsets.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D100386
2021-04-13 15:30:30 -04:00
Lei Zhang
23b8264b52 [mlir][spirv] Fix runtime array stride when emulating bitwidth
The stride should be calculated with the converted array element
type, not the original input type.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D100337
2021-04-12 17:13:33 -04:00
Lei Zhang
fd91f81c85 [mlir][spirv] Put debug-only variable in LLVM_DEBUG
This avoids paying the cost when building in release.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D100325
2021-04-12 15:14:43 -04:00
Weiwei Li
12ffc26067 [mlir][spirv] Define spv.ImageDrefGather operation
This patch doesn't support the optional operands of ImageDrefGather. The support of optional operands will be implemented later.

co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Differential Revision: https://reviews.llvm.org/D100128
2021-04-08 20:15:54 -04:00
Hanhan Wang
c361435845 [mlir][StandardToSPIRV] Handle i1 case for lowering memref.load/store op
This patch unconditionally converts i1 types to i8 types on memrefs. If the
extensions or capabilities are not met, they will be converted to i32. Hence the
logic in IntLoadPattern and IntStorePattern are also updated.

Also added the implementation of SPIRVTypeConverter::getOptions().

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D99724
2021-04-08 12:15:25 -07:00
Lei Zhang
5299843c31 [mlir][spirv] Add control for non-32-bit scalar type emulation
Non-32-bit scalar types require special hardware support that may not
exist on all GPUs. This is reflected in SPIR-V as that non-32-bit scalar
types require special capabilities or extensions.

Previously when there is a non-32-bit type and no native support, we
unconditionally emulate it with 32-bit ones. This isn't good given that
it can have implications over ABI and data layout consistency.

This commit introduces an option to control whether to use 32-bit
types to emulate.

Differential Revision: https://reviews.llvm.org/D100059
2021-04-08 08:19:47 -04:00
Lei Zhang
004f29c0bb [mlir][spirv] Timely fail type conversion
Per the TypeConverter API contract, returning `llvm:None` means
other conversion rules should be tried. But we only have one
rule per input type. So there is no need to try others and we can
just directly fail, which should return `nullptr`. This avoids
unnecessary checks.

Differential Revision: https://reviews.llvm.org/D100058
2021-04-08 08:19:46 -04:00
Lei Zhang
6dd07fa513 [mlir][spirv] Add utilities for push constant value
This commit add utility functions for creating push constant
storage variable and loading values from it.

Along the way, performs some clean up:

* Deleted `setABIAttrs`, which is just a 4-liner function
  with one user.
* Moved `SPIRVConverstionTarget` into `mlir` namespace,
  to be consistent with `SPIRVTypeConverter` and
  `LLVMConversionTarget`.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D99725
2021-04-02 07:51:07 -04:00
Chris Lattner
79d7f618af Rename FrozenRewritePatternList -> FrozenRewritePatternSet; NFC.
This nicely aligns the naming with RewritePatternSet.  This type isn't
as widely used, but we keep a using declaration in to help with
downstream consumption of this change.

Differential Revision: https://reviews.llvm.org/D99131
2021-03-22 17:40:45 -07:00
Chris Lattner
dc4e913be9 [PatternMatch] Big mechanical rename OwningRewritePatternList -> RewritePatternSet and insert -> add. NFC
This doesn't change APIs, this just cleans up the many in-tree uses of these
names to use the new preferred names.  We'll keep the old names around for a
couple weeks to help transitions.

Differential Revision: https://reviews.llvm.org/D99127
2021-03-22 17:20:50 -07:00
Chris Lattner
3a506b31a3 Change OwningRewritePatternList to carry an MLIRContext with it.
This updates the codebase to pass the context when creating an instance of
OwningRewritePatternList, and starts removing extraneous MLIRContext
parameters.  There are many many more to be removed.

Differential Revision: https://reviews.llvm.org/D99028
2021-03-21 10:06:31 -07:00
Vladislav Vinogradov
fee9054232 [mlir][ODS] Support specialized Attribute class for Enums
Add a feature to `EnumAttr` definition to generate
specialized Attribute class for the particular enumeration.

This class will inherit `StringAttr` or `IntegerAttr` and
will override `classof` and `getValue` methods.

With this class the enumeration predicate can be checked with simple
RTTI calls (`isa`, `dyn_cast`) and it will return the typed enumeration
directly instead of raw string/integer.

Based on the following discussion:
https://llvm.discourse.group/t/rfc-add-enum-attribute-decorator-class/2252

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D97836
2021-03-17 16:44:24 +03:00
River Riddle
31bb8efd69 [mlir][StorageUniquer] Properly call the destructor on non-trivially destructible storage instances
This allows for storage instances to store data that isn't uniqued in the context, or contain otherwise non-trivial logic, in the rare situations that they occur. Storage instances with trivial destructors will still have their destructor skipped. A consequence of this is that the storage instance definition must be visible from the place that registers the type.

Differential Revision: https://reviews.llvm.org/D98311
2021-03-11 11:35:32 -08:00
KareemErgawy-TomTom
3fb384d50e [MLIR][SPIRV] Rename spv.selection to spv.mlir.selection.
To unify the naming scheme across all ops in the SPIR-V dialect, we are
moving from spv.camelCase to spv.CamelCase everywhere. For ops that
don't have a SPIR-V spec counterpart, we use spv.mlir.snake_case.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D98014
2021-03-06 16:05:31 +01:00
Lei Zhang
bb6f5c8314 [mlir][spirv] Convert tensor.extract for very small tensors
Normally tensors will be stored in buffers before converting to SPIR-V,
given that is how a large amount of data is sent to the GPU. However,
SPIR-V supports converting from tensors directly too. This is for the
cases where the tensor just contains a small amount of elements and it
makes sense to directly inline them as a small data array in the shader.
To handle this, internally the conversion might create new local
variables. SPIR-V consumers in GPU drivers may or may not optimize that
away. So this has implications over register pressure. Therefore, a
threshold is used to control when the patterns should kick in.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D98052
2021-03-06 08:03:36 -05:00
KareemErgawy-TomTom
29812a6195 [MLIR][SPIRV] Rename spv.loop to spv.mlir.loop.
To unify the naming scheme across all ops in the SPIR-V dialect,
we are moving from spv.camelCase to spv.CamelCase everywhere.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D97918
2021-03-05 15:44:30 -05:00
KareemErgawy-TomTom
c74eb466d2 [MLIR][SPIRV] Rename spv.globalVariable to spv.GlobalVariable.
To unify the naming scheme across all ops in the SPIR-V dialect, we are
moving from spv.camelCase to spv.CamelCase everywhere.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D97919
2021-03-04 16:24:59 -05:00