Review follow-up from https://github.com/llvm/llvm-project/pull/183325
No reason for these data structures to be ordered.
Minor annoyance when trying to use `DenseMap` because of the C++ code
for enums generated by TableGen, but not too bad.
Signed-off-by: Nick Sarnie <nick.sarnie@intel.com>
If you pass `-ExtName` to the `--spirv-ext` command line option. that
should disable the extension. However some vendors have some extensions
enabled by default when using a triple with that vendor, and disabling
an extension with the option did not effect the default extensions.
This PR makes it so disabling an extension with the `--spirv-ext` option
actually disables the extension.
The problem was we only considered the disabled extension when parsing
the arguments for `--spirv-ext`, but the default extensions are added
separately, so we need to store the disabled extensions and factor them
in when computing the final extension set to use.
Signed-off-by: Nick Sarnie <nick.sarnie@intel.com>
For compute we don't run structurizer hence we won't be able to preserve
loop metadata via LoopMerge instruction. So
SPV_INTEL_unstructured_loop_controls is the only way we can preserve the
info in unstructured control flow.
The extension adds support for the `OpFmaKHR` instruction, which
provides a native SPIR-V instruction for fused multiply-add operations
as an alternative to using OpenCL.std::Fma extended instruction.
Translate both LLVM fma intrinsics as well as OCL builtins to `OpFmaKHR`
if the extension is available.
Specification:
https://github.khronos.org/SPIRV-Registry/extensions/KHR/SPV_KHR_fma.html
--Added support for the extension SPV_KHR_non_semantic_info
--Added support for the extension SPV_KHR_relaxed_extended_instruction
--Added instructions from the documentation of the extension.
--Added supporting tests for the same.
Same as #165302
---------
Co-authored-by: Michal Paszkowski <michal@michalpaszkowski.com>
Added support for the SPV_ALTERA_arbitrary_precision_floating_point
extension, enabling all the arbitrary precision floating-point
operations with instruction definitions and test files.
This changes the extension parsing mechanism underpinning `--spirv-ext`
to be more explicit about what it is doing and not rely on a sort. More
specifically, we partition extensions into enabled (prefixed with `+`)
and others, and then individually handle the resulting ranges.
LLVM has pretty thorough support for `int128`, and it has started seeing
some use. Even thouth we already have support for the
`SPV_ALTERA_arbitrary_precision_integers` extension, the BE was oddly
capping integer width to 64-bits. This patch adds partial support for
lowering 128-bit integers to `OpTypeInt 128`. Some work remains to be
done around legalisation support and validating constant uses (e.g.
cases that get lowered to `OpSpecConstantOp`).
This adds support for the `SPV_NV_shader_atomic_fp16_vector` extension,
and then uses it to enable lowering of atomic add, sub, min and max on 2
and 4 component vectors of FP16, which are rather common options in ML
workloads. Even though `bfloat16` also works in practice, we do not
enable it since it's not specified in the extension (which might need
updating / promoting to KHR at least). A `TODO` is also inserted in
`SPIRVModuleAnalysis.cpp' regarding the need to upgrade its ample usage
of `report_fatal_error`; I have a WiP patch for that, but it still needs
a bit of baking. Finally, a paired patch will be necessary in the
Translator, as it's not aware of the extension either - I'll update this
review to reference the PR once I create it.
AMD uses the translator to recover LLVM-IR from SPIRV.
Currently, the translator doesn't implement the
`SPV_KHR_float_controls2` extension (I'm working on it).
If this extension is used by the SPIRV module, we cannot translate it
back to LLVM-IR.
I'm working on the extension, but in the meantime, lets just disable it
when the target triple's vendor is `amd`.
This enables support for atomic RMW ops (add, sub, min and max to be
precise) with `bfloat16` operands, via the [SPV_INTEL_16bit_atomics
extension](https://github.com/intel/llvm/pull/20009). It's logically a
successor to #166031 (I should've used a stack), but I'm putting it up
for early review.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
Enable the `SPV_INTEL_bfloat16_arithmetic` extension, which allows arithmetic, relational and `OpExtInst` instructions to take `bfloat16` arguments. This patch only adds support to arithmetic and relational ops. The extension itself is rather fresh, but `bfloat16` is ubiquitous at this point and not supporting these ops is limiting.
This adds BE support for the
[`SPV_INTEL_kernel_attributes`](https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_kernel_attributes.html)
extension. The extension is necessary to encode the rather useful
`max_work_group_size` kernel attribute, via `OpExecutionMode
MaxWorkgroupSizeINTEL`, which is the only Execution Mode added by the
extension that this patch adds full processing for. Future patches will
add the other Execution Modes and Capabilities. The test is adapted from
the equivalent Translator test; it depends on #165815.
This PR introduces the support for the SPIR-V extension
`SPV_INTEL_predicated_io`. This extension adds predicated load and store
instructions. Predicated load performs load from memory if predicate is
true; otherwise, it uses default_value as a result. Predicated store
performs store of value to memory if predicate is true; otherwise, it
does nothing.
Reference Specification:
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_predicated_io.asciidoc
This PR introduces the support for the SPIR-V extension
`SPV_KHR_bfloat16`. This extension extends the `OpTypeFloat` instruction
to enable the use of bfloat16 types with cooperative matrices and dot
products.
TODO:
Per the `SPV_KHR_bfloat16` extension, there are a limited number of
instructions that can use the bfloat16 type. For example, arithmetic
instructions like `FAdd` or `FMul` can't operate on `bfloat16` values.
Therefore, a future patch should be added to either emit an error or
fall back to FP32 for arithmetic in cases where bfloat16 must not be
used.
Reference Specification:
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_KHR_bfloat16.asciidoc
The default behaviour in DXC is to allow all extesions the compiler
knows about. We did the same in clang: all extensions that clang knows
about. However, this causes the shader to use different extensions
because the two compilers have different sets of extensions.
To avoid using a new extension when moving from DXC to Clang, we add the
special DXC suboptions to `-fspv-extension`. If `-fspv-extension=DXC` is
used, then the available extensions will be those available in DXC.
---------
Co-authored-by: Chris B <beanz@abolishcrlf.org>
Not all SPIR-V extensions are allows in every environment. When we use
the `-spirv-ext=all` option, the backend currently believes that all
extensions can be used.
This commit filters out the extensions on the command line to remove
those that are not known to be allowed for the current environment.
Alternatives considered: I considered modifying the
SPIRVExtensionsParser::parse to use a different list of extensions for
"all" depending on the target triple. However that does not work because
the target triple is not available, and cannot be made available in a
reasonable way.
Fixes#147717
---------
Co-authored-by: Victor Lomuller <victor@codeplay.com>
In DXC, there is an option to enable all KHR extension. I would like to
extend the existing `-spirv-ext` backend commandline option to have the
same capability. It is like the special case for `all` execept it only
adds the `SPV_KHR_*` extensions.
Part of https://github.com/llvm/llvm-project/issues/137650.
Add handling for FPFastMathMode in SPIR-V shaders. This is a first pass
that
simply does a direct translation when the proper extension is available.
This will unblock work for HLSL. However, it is not a full solution.
The default math mode for spir-v is determined by the API. When
targeting Vulkan many of the fast math options are assumed. We should do
something particular when targeting Vulkan.
We will also need to handle the hlsl "precise" keyword correctly when
FPFastMathMode is not available.
Unblockes https://github.com/llvm/llvm-project/issues/140739, but we are
keeing it open to track the remaining issues mentioned above.
This relands #141031
This change ensures generated SPIR-V is valid and passes machine
verification:
```
*** Bad machine code: inconsistent constant size ***
- function: foo
- basic block: %bb.1 entry (0x9ec9298)
- instruction: %12:iid(s8) = G_CONSTANT i4 1
```
That is done by promoting `G_CONSTANT` instructions with small integer
types (e.g., `i4`) to `i8` if no extensions for "special" integer types
are enabled.
Adds additional subgroup block prefetch, load,
load transposed, load transformed and store
instructions to read two-dimensional blocks of
data from a two-dimensional region of memory, or
to write two-dimensional blocks of data to a
two-dimensional region of memory.
Spec:
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/INTEL/SPV_INTEL_2d_block_io.asciidoc
---------
Co-authored-by: Dmitry Sidorov <dmitry.sidorov@intel.com>
Adds support for the SPV_INTEL_ternary_bitwise_function extension,
adding;
* the OpBitwiseFunctionINTEL SPIR-V instruction, a ternary bitwise
function where the operation performed is determined by a look-up table
index,
* and the corresponding TernaryBitwiseFunctionINTEL capability.
See
https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_ternary_bitwise_function.html.
Signed-off-by: Larsen, Steffen <steffen.larsen@intel.com>
- Use static instead of anonymous namespace for file local functions.
- Enclose file-local classes in anonymous namespace.
- Eliminate `llvm::` qualifier when file has `using namespace llvm`.
- Eliminate namespace surrounding entire code in
SPIRVConvergenceRegionAnalysis.cpp file.
- Eliminate call to `initializeSPIRVStructurizerPass` from the pass
constructor (https://github.com/llvm/llvm-project/issues/111767)
Spec can be found here https://github.com/intel/llvm/pull/15225
TODO for future patches:
- During spec review need to decide whether only FunctionCall or Atomic
instructions can be decorated and if not - move the code around adding
handling for other instructions;
- Handle optional string metadata;
- Handle LLVM atomic instructions;
- Handle SPIR-V friendly atomic calls returning via sret argument.
Signed-off-by: Sidorov, Dmitry <dmitry.sidorov@intel.com>
This PR fixes https://github.com/llvm/llvm-project/issues/124703:
* added a new API call `SPIRVTranslate` that is to replace entirely old
`SPIRVTranslateModule` after existing clients switch into the new
function;
* the new `SPIRVTranslate` doesn't require option parsing, replacing the
`Opts` argument with explicit `CodeGenOptLevel` and `Triple` arguments;
* the old `SPIRVTranslateModule` call is a wrapper for `SPIRVTranslate`,
it doesn't require option parsing either and doesn't hold any logic
inside except for converting string options into `CodeGenOptLevel` and
`Triple` arguments;
* usage of the extensions list is reworked to avoid writes to the global
cl::opt variable `lib/Target/SPIRV/SPIRVSubtarget.cpp::Extensions` --
instead a new class member in SPIRVSubtarget.cpp is implemented that
allows to replace supported extensions after SPIRVSubtarget.cpp is
created;
* both API calls don't require option parsing and don't write to global
cl::opt variables.
Other related/required changes:
* SPIRV::Capability::Shader is marked as an capability of lesser
priority for OpenCL environment (to remediate absence of the
"avoid-spirv-capabilities" command line option in API calls);
* unit tests are updated and extended to cover testing of a newer API
call;
* old API call is marked with TODO to remove it after existing clients
switch into the new function.
This PR adds the following features:
* saturation and float rounding mode decorations,
* arithmetic constrained floating-point intrinsics (strict_fadd,
strict_fsub, strict_fmul, strict_fdiv, strict_frem, strict_fma and
strict_fldexp),
* and SPV_INTEL_float_controls2 extension,
* using recent improvements of emit-intrinsics step, this PR also
simplifies pre- and post-legalizer steps and improves instruction
selection.
The spec is available here:
https://github.com/intel/llvm/pull/12497
The PR doesn't add OpCooperativeMatrixApplyFunctionINTEL instruction as
it's still experimental and not properly tested E2E.
The PR also fixes few bugs in the related code:
1. CooperativeMatrixMulAddKHR optional operand must be literal, not a
constant;
2. Fixed available capabilities table creation for a case, when a single
extension adds few capabilities, that occupy not contiguous op codes.
---------
Signed-off-by: Sidorov, Dmitry <dmitry.sidorov@intel.com>
This PR fixes:
* emission of OpNames (added newly inserted internal intrinsics and
basic blocks)
* emission of function attributes (SRet is added)
* implementation of SPV_INTEL_optnone so that it emits OptNoneINTEL
Function Control flag, and add implementation of the SPV_EXT_optnone
SPIR-V extension.
