This PR is to add vector reduction instructions according to
https://llvm.org/docs/GlobalISel/GenericOpcode.html#vector-reduction-operations
and widen in such a way a range of successful supported conversions,
covering new cases of vector reduction instructions which IRTranslator
is unable to resolve.
By legalizing vector reduction instructions we introduce a new
instruction patterns that should be addressed, including patterns that
are delegated to pre-legalize step. To address this problem, a new pass
is added that is to bring newly generated instructions after
legalization to an aspect required by instruction selection.
Expected overheads for existing cases is minimal, because a new pass is
working only with newly introduced instructions, otherwise it's just a
additional code traverse without any actions.
This pull request aims to remove any dependency on OpenCL/SPIR-V type
information in LLVM IR metadata. While, using metadata might simplify
and prettify the resulting SPIR-V output (and restore some of the
information missed in the transformation to opaque pointers), the
overall methodology for resolving kernel parameter types is highly
inefficient.
The high-level strategy is to assign kernel parameter types in this order:
1. Resolving the types using builtin function calls as mangled names
must contain type information or by looking up builtin definition in
SPIRVBuiltins.td. Then:
- Assigning the type temporarily using an intrinsic and later setting
the right SPIR-V type in SPIRVGlobalRegistry after IRTranslation
- Inserting a bitcast
2. Defaulting to LLVM IR types (in case of pointers the generic i8*
type or types from byval/byref attributes)
In case of type incompatibility (e.g. parameter defined initially as
sampler_t and later used as image_t) the error will be found early on
before IRTranslation (in the SPIRVEmitIntrinsics pass).
This PR adds SPIR-V extension SPV_INTEL_variable_length_array that
allows to allocate local arrays whose number of elements is unknown at
compile time:
* add a new SPIR-V internal intrinsic:int_spv_alloca_array
* legalize G_STACKSAVE and G_STACKRESTORE
* implement allocation of arrays (previously getArraySize() of
AllocaInst was not used)
* add tests
This PR is to add support for the 'freeze' instruction:
https://llvm.org/docs/LangRef.html#freeze-instruction
There is no way to implement `freeze` correctly without support on
SPIR-V standard side, but we may at least address a simple (static) case
when undef/poison value presence is obvious. The main benefit of even
incomplete `freeze` support is preventing of translation from crashing
due to lack of support on legalization and instruction selection steps.
This PR is to prevent creation of jump tables from switch. The reason is
that SPIR-V doesn't know how to lower jump tables, and a sequence of
commands that IRTranslator generates for switch via jump tables breaks
SPIR-V Backend code generation with complains to G_BRJT. The next
example is the shortest code to break SPIR-V Backend code generation in
this way:
```
target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64"
target triple = "spir64-unknown-unknown"
define spir_func void @foo(i32 noundef %val) {
entry:
switch i32 %val, label %sw.epilog [
i32 0, label %sw.bb
i32 1, label %sw.bb2
i32 2, label %sw.bb3
i32 3, label %sw.bb4
]
sw.bb:
br label %sw.epilog
sw.bb2:
br label %sw.epilog
sw.bb3:
br label %sw.epilog
sw.bb4:
br label %sw.epilog
sw.epilog:
ret void
}
```
To resolve the issue we set a high lower limit for number of blocks in a
jump table via getMinimumJumpTableEntries() and prevent undesirable (or
rather unsupported at the moment) path of code generation.
This pull request fixes an issue with missing vector element count
immediate in OpExtInst calls and adds a case for generating bitcasts
before GEPs for kernel arguments of non-matching pointer type. The new
LITs are based on basic/vload_local and basic/vload_global OpenCL CTS
tests. The tests after this change pass SPIR-V validation.
This PR is to add support for the SPIR-V extension
SPV_KHR_uniform_group_instructions that adds new instructions to SPIR-V
to support additional group operations within uniform control flow.
SPIRV-V Backend generates unnecessary OpExecutionMode records, putting
into the id's which are not the Entry Point operands of an OpEntryPoint
(ref: https://github.com/llvm/llvm-project/issues/81753). This PR is to
fix the issue.
This PR adds support for atomic instruction on floating-point numbers:
* SPV_EXT_shader_atomic_float_add
* SPV_EXT_shader_atomic_float_min_max
* SPV_EXT_shader_atomic_float16_add
and fixes asm printer output for half floating-type.
This PR adds support for the SPV_KHR_linkonce_odr extension and modifies
existing negative test with a positive check for the extension and
proper linkage type in case when the extension is enabled.
SPV_KHR_linkonce_odr adds a "LinkOnceODR" linkage type, allowing proper
translation of, for example, C++ templates classes merging during
linking from different modules and supporting any other cases when a
global variable/function must be merged with equivalent global
variable(s)/function(s) from other modules during the linking process.
By SPIR-V specification: "If an instruction, enumerant, or other feature
specifies multiple enabling capabilities, only one such
capability needs to be declared to use the feature."
However, one capability may be preferred over another. One important
case is Shader capability that may not be supported by a backend, but
always is inserted if "OpDecorate SpecId" is found, because Enabling
Capabilities for the latter is the list of Shader and Kernel, where
Shader is coming first and thus always selected as the first available
option.
In this PR we address the problem by keeping current behaviour of
selecting the first option among enabling capabilities as is, but giving
a user a way to filter capabilities during the selection process via a
newly introduced "--avoid-spirv-capabilities" command line option. This
option is to avoid selection of certain capabilities if there are other
available enabling capabilities.
This PR is changing also existing pruneCapabilities() function. It
doesn't remove capability from module requirement anymore, but only adds
implicitly required capabilities recursively, so its name is changed
accordingly. This change fixes the present bug in collecting required by
a module capabilities. Before the change, introduced by this PR,
pruneCapabilities() function has been removing, for example, Kernel
capability from required by a module, because Kernel is initially
required and the second time it was needed pruneCapabilities() removed
it by mistake.
The goal of this PR is to tolerate differences between description of
formal arguments by function metadata (represented by "kernel_arg_type")
and LLVM actual parameter types. A compiler may use "kernel_arg_type" of
function metadata fields to encode detailed type information, whereas
LLVM IR may utilize for an actual parameter a more general type, in
particular, opaque pointer type. This PR proposes to resolve this by a
fallback to LLVM actual parameter types during the lowering of formal
function arguments in cases when the type can't be created by string
content of "kernel_arg_type", i.e., when "kernel_arg_type" contains a
type unknown for the SPIR-V Backend.
An example of the issue manifestation is
https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/main/test/transcoding/KernelArgTypeInOpString.ll,
where a compiler generates for the following kernel function detailed
`kernel_arg_type` info in a form of `!{!"image_kernel_data*", !"myInt",
!"struct struct_name*"}`, and in LLVM IR same arguments are referred to
as `@foo(ptr addrspace(1) %in, i32 %out, ptr addrspace(1) %outData)`.
Both definitions are correct, and the resulting LLVM IR is correct, but
lowering stage of SPIR-V Backend fails to generate SPIR-V type.
```
typedef int myInt;
typedef struct {
int width;
int height;
} image_kernel_data;
struct struct_name {
int i;
int y;
};
void kernel foo(__global image_kernel_data* in,
__global struct struct_name *outData,
myInt out) {}
```
```
define spir_kernel void @foo(ptr addrspace(1) %in, i32 %out, ptr addrspace(1) %outData) ... !kernel_arg_type !7 ... {
entry:
ret void
}
...
!7 = !{!"image_kernel_data*", !"myInt", !"struct struct_name*"}
```
The PR changes a contract of `SPIRVType *getArgSPIRVType(...)` in a way
that it may return `nullptr` to signal that the metadata string content
is not recognized, so corresponding comments are added and a couple of
checks for `nullptr` are inserted where appropriate.
This PR fixes https://github.com/llvm/llvm-project/issues/79057 and
improves code generation for opaque pointers by replacing the culprit
SPIRVGlobalRegistry::getOpTypePointer() call with a more appropriate
SPIRVGlobalRegistry::getOrCreateSPIRVPointerType() call. The latter
function works together with the `DuplicatesTracker`
(`SPIRVGeneralDuplicatesTracker DT;` from `class SPIRVGlobalRegistry`)
to trace existence of previous definitions of opaque pointers. This
allows to produce just one `OpTypePointer` command for all identical
opaque pointers definitions and to return the very same type record for
subsequent `SPIRVGlobalRegistry::createSPIRVType()` invocations.
This PR alone improves code generation by producing a single needed
definition per all opaque pointers to i8 of the same address space
instead of multiple identical definitions produced before the patch.
From the root cause analysis of
https://github.com/llvm/llvm-project/issues/79057 we see also that this
PR resolves the problem of inconsistency between keeping multiple
instruction for identical opaque pointer types and just a single record
for all such instructions in the `DuplicatesTracker`, and so it also
resolves the issue with crashes on creation of a struct with opaque
pointer fields due to the fact that now such struct fields refer to the
same operand `<id>` having a required record in the data structure used
for dependencies analysis (see
https://github.com/llvm/llvm-project/issues/79057).
The goal of this PR is to fix the issue when use of token type in LLVM
intrinsic causes undefined behavior of SPIR-V Backend code generator
when assertions are disabled:
https://github.com/llvm/llvm-project/issues/78434
Among possible fix options, discussed in the
https://github.com/llvm/llvm-project/issues/78434 issue description, the
option to generate a meaningful error before execution arrives at the
`llvm_unreachable` call looks like a better solution for now, because
SPIR-V doesn't support token type anyway without additional extensions.
The PR is to generate a user-friendly error message and exit without
generating a stack dump when such a usage of token type was detected
that would lead to undefined behavior of SPIR-V Backend code generator.
The goal of this PR is to fix the issue of global unnamed variables
causing SPIR-V Backend code generation to crash:
https://github.com/llvm/llvm-project/issues/78278
The reason for the crash is that GlobalValue's getGlobalIdentifier()
would fail for unnamed global variable when trying to access the first
character of the name (see lib/IR/Globals.cpp:150). This leads to assert
in Debug and undefined behaviour in Release builds.
The proposed fix generates a name of an unnamed global variable as
__unnamed_<unsigned number>, in a style of similar existing LLVM
implementation (see lib/IR/Mangler.cpp:131). A new class member variable
is added into `SPIRVInstructionSelector` class to keep track of the
number we give to anonymous global values to generate the same name
every time when this is needed.
The patch adds a new LIT test with the smallest implementation of
reproducer ll code.
Handle a special case when StoreInst's value operand is a kernel
argument of a pointer type. Since these arguments could have either a
basic element type (e.g. float*) or OpenCL builtin type (sampler_t),
bitcast the StoreInst's value operand to default pointer element type
(i8).
This pull request addresses the issue
https://github.com/llvm/llvm-project/issues/72864
Prior to this change spv_ptrcast (and OpBitcast) was never emitted for
GEP resulting pointers. While such SPIR-V was (mostly) accepted by the
NEO GPU driver, the generated SPIR-V was incorrect.
The newly added test (pointers/getelementptr-bitcast-load.ll) verifies
that a correct bitcast is added for more complex cases and passes
spirv-val. The test is based on an OpenCL CTS test (basic/prefetch).
The structurizer will require the frontend to emit convergence
intrinsics. Once uses to restructurize the control-flow, those
intrinsics shall be removed, as they cannot be converted to
SPIR-V.
This commit adds a new pass to the SPIR-V backend which strips those
intrinsics.
Those 2 new steps are not limited to Vulkan as OpenCL could
also benefit from not crashing if a convertent operation is in
the IR (even though the frontend doesn't generate such intrinsics).
Signed-off-by: Nathan Gauër <brioche@google.com>
Add Float16 to Vulkan's available capabilities, and guard Float16Buffer
(Kernel-only capability) against being added outside OpenCL
environments.
Add tests to verify half and half vector types, and validate with
spirv-val.
Fixes#66398
This is the first of the 7 steps outlined in #75801. This PR explicitely
calls the SimplifyLoops pass. Directly following this pass should follow
the 6 others required to structurize the IR.
Running this pass could generate empty basic-blocks, which are implicit
fallthrough to the successor BB.
There was a specific condition in the SPIR-V ISel which handled implicit
fallthrough, but it couldn't work on empty basic-blocks. This commits
removes the old logic, and adds this new logic, which checks all
basic-blocks for implicit fallthroughs, including empty ones.
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
This patch introduces a new spv_ptrcast intrinsic for tracking expected
pointer types. The change fixes multiple OpenCL CTS regressions due the
switch to opaque pointers (e.g. basic/hiloeo).
Add spirv-dis (disassembler) and spirv-val (validator) from SPIRV-Tools
as external dependencies for testing the SPIR-V backend. These tools are
test dependencies only.
SPIR-V backend tests now have a dependency on the spirv-dis and
spirv-val targets when the `LLVM_INCLUDE_SPIRV_TOOLS_TESTS` cmake
variable is set, which allows additional test files with the `REQUIRES:
spirv-tools` constraint to run, along with additional `RUN: %if
spirv-tools ...` lines in existing tests. All other SPIR-V backend tests
will run normally when `LLVM_INCLUDE_SPIRV_TOOLS_TESTS` is not set.
Several tests are included to show these tools' use, however more tests
will be migrated and added later.
* OpVariable_order.ll shows how spirv-val can catch bugs in the backend.
* basic_int_types_spirvdis.ll shows how tests can be much shorter and
more readable by FileChecking the spirv-dis output.
* basic_int_types.ll shows how an additional RUN line can add validation
to existing tests.
RFC:
https://discourse.llvm.org/t/rfc-add-a-test-dependency-on-spirv-tools/75135
These tests are currently failing and their fix is being tracked in
Issue #60133. Marking them as XFAIL for now will get the test suite to a
passing state so we can work on adding a GitHub action to automatically
run these tests on a PR bot to help keep the tree green.
Also removed the no-longer supported -opaque-pointers=0 flag from the
couple tests where it was remaining.
Since efe0e10718 changes in tests are required. Need to add extension to
Extensions list
and command line to enable use of the extension for test runs.
There is no log10 instruction in the GLSL Extended Instruction Set so to
implement the HLSL log10 intrinsic when targeting Vulkan this change
adds the logic to derive the result using the following formula:
```
log10(x) = log2(x) * (1 / log2(10))
= log2(x) * 0.30103
```
This commit adds 2 new instructions in the selector:
- OpAccessChain
- OpInBoundsAccessChain.
The choice between the two relies on the `inbounds` marker.
Those instruction are not used for OpenCL, to maintain the same
behavior as previously. They are only added when building for logical
SPIR-V, as it doesn't support the pointer equivalent.
Because logical SPIR-V doesn't support pointer cast either, the
assign_ptr_type intrinsic need to be generated so OpAccessChain gets
lowered with the correct pointer type, instead of i8*.
Fixes#66107
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
Adds new extension SPV_KHR_expect_assume, new capability
ExpectAssumeKHR as well as the new instructions:
* OpExpectKHR
* OpAssumeTrueKHR
These are lowered from respectively llvm.expect.<ty> and llvm.assume
intrinsics.
Previously https://reviews.llvm.org/D157696
Print OpConstant floats as formatted decimal floating points, with
special case exceptions to print infinity and NaN as hexfloats.
This change follows from the fixes in
https://github.com/llvm/llvm-project/pull/66686 to correct how
constant values are printed generally.
Differential Revision: https://reviews.llvm.org/D159376
Previously, the SPIR-V instruction printer was always printing the first
operand of an `OpConstant`'s literal value as one of the fixed operands.
This is incorrect for 64-bit values, where the first operand is actually
the value's lower-order word and should be combined with the following
higher-order word before printing.
This change fixes that issue by waiting to print the last fixed operand
of `OpConstant` instructions until the variadic operands are ready to be
printed, then using `NumFixedOps - 1` as the starting operand index for
the literal value operands.
Depends on D156049
Add Int16, Int64 and Float64 capabilities as always available for Vulkan
(since 1.0), and add tests covering most of the basic types from
clang/test/CodeGenHLSL/basic_types.hlsl except for half floats.
Depends on D156049
Lookup extended instruction numbers in the given instruction set so that
correct names are now emitted for GLSL.std.450 instructions as well as
OpenCL.std.
Add a single test to verify correct abs intrinsic names are emitted when
targetting logical SPIR-V.
Depends on D156424
Differential Revision: https://reviews.llvm.org/D159227
This commits adds the minimal required bits to build a logical SPIR-V
compute shader using LLC.
- Skip OpenCL-only capabilities & extensions for Logical SPIR-V.
- Generate required metadata for entrypoints from HLSL frontend.
- Fix execution mode to GLCompute in logical.
The main issue is the lack of "vulkan" bit in the triple.
This might need to be added as a vendor?
Because as-is, SPIRV32/64 assumes OpenCL, and then, SPIRV assumes
Vulkan. This is ok-ish today, but not correct.
Differential Revision: https://reviews.llvm.org/D156424
Adds support for SPV_INTEL_optnone.
Currently still in draft form but I wanted to open this revision
to ask some questions.
Differential Revision: https://reviews.llvm.org/D156297
This is a follow-up to b71edfaa4ec3c998aadb35255ce2f60bba2940b0
since I forgot the lit.local.cfg files in that one.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: barannikov88, kwk
Differential Revision: https://reviews.llvm.org/D150762
IRTranslator lowers switches to [G_SUB] + G_ICMP + G_BRCOND + G_BR
sequences. Since values and destination MBBs are included in the
spv_switch intrinsics, the sequences are not needed for ISel.
Before this commit, the information decoded by these sequences were
added to spv_switch intrinsics in SPIRVPreLegalizer and the sequences
were kept until SPIRVModuleAnalysis where they were marked skipped for
emission.
After this commit, the [G_SUB] + G_ICMP + G_BRCOND + G_BR sequences
and MBBs containing only these MIs are erased in SPIRVPreLegalizer.
Differential Revision: https://reviews.llvm.org/D146923
After this patch all arbitrary size integers (smaller than 64 bits) in
LLVM IR will be promoted to regular size type in SPIR-V (OpTypeInt
8/16/32/64).
Differential Revision: https://reviews.llvm.org/D145137
This patch adds support for TargetExtType/target(...) representing
SPIR-V builtin types. After D135202, target(...) is the preferred way
for representing SPIR-V builtin types in LLVM IR and the only working
in the opaque pointer mode.
In order to maintain compatibility with LLVM IR generated by older
versions of Clang and LLVM/SPIR-V Translator, pointers-to-opaque-structs
denoting SPIR-V/OpenCL builtin types will be translated to equivalent
SPIR-V target extension types. This translation is only available in the
typed pointer mode (-opaque-pointers=0).
The relevant LIT tests with SPIR-V builtins were converted to use the
new target(...) notation.
Differential Revision: https://reviews.llvm.org/D144494
This change adds a new spv_undef intrinsic which is emitted in place of
aggregate undef operands and later selected to single OpUndef SPIR-V
instruction. The behavior matches that of Khronos SPIR-V Translator and
should support nested aggregates.
Differential Revision: https://reviews.llvm.org/D143107
