This commit adds initial support to lower the intrinsinc
`@llvm.structured.gep` into proper SPIR-V.
For now, the backend continues to support both GEP formats. We might
want to revisit this at some point for the logical part.
This reverts commit 1daef5927ff0e43b92d615380a8d60f213cc5e52.
From the ASAN buildbot:
```bash
FAIL: LLVM :: CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll (46596 of 94488)
******************** TEST 'LLVM :: CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll' FAILED ********************
Exit Code: 2
Command Output (stdout):
--
# RUN: at line 5
/home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/llc -verify-machineinstrs -O0 -mtriple=spirv32-unknown-unknown /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/test/CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll -o - --spirv-ext=+SPV_INTEL_function_pointers | /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/test/CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll
# executed command: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/llc -verify-machineinstrs -O0 -mtriple=spirv32-unknown-unknown /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/test/CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll -o - --spirv-ext=+SPV_INTEL_function_pointers
# note: command had no output on stdout or stderr
# error: command failed with exit status: 1
# executed command: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/test/CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll
# .---command stderr------------
# | FileCheck error: '<stdin>' is empty.
# | FileCheck command line: /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm_build_asan/bin/FileCheck /home/b/sanitizer-aarch64-linux-bootstrap-asan/build/llvm-project/llvm/test/CodeGen/SPIRV/extensions/SPV_INTEL_function_pointers/fun-ptr-addrcast.ll
# `-----------------------------
# error: command failed with exit status: 2
```
I did not confirm this commit is introducing the issue. But since its
likely related I'm reverting it until I confirm everything is ok or fix
it.
In the SPIRV backend, the `SPIRVEmitIntrinscs::processGlobalValue`
function adds intrinsic calls for every global variable of the module,
on every function.
These intrinsics are used to keep track of global variables, their types
and initializers.
In SPIRV everything is an instruction (even globals/constants). We
currently represent these global entities as individual instructions on
every function. Later, the `SPIRVModuleAnalysis` collects these entities
and maps function _local_ registers to _global_ registers. The
`SPIRVAsmPrinter` is in charge of mapping back the _local_ registers to
the appropriate _global_ register.
These instructions associated with global entities on functions that do
not reference them leads to a bloated intermediate representation and
high memory consumption (as it happened in
https://github.com/llvm/llvm-project/issues/170339).
Consider this example:
```cpp
int A[1024] = { 0, 1, 2, ..., 1023 };
int get_a(int i) {
return A[i];
}
void say_hi() {
puts("hi!\n");
}
```
Although, `say_hi` does not reference `A`; we would generate machine
instructions for the declaration of `A`, for the constant array that
initializes it, and for the 1024 constant elements of `A`.
This patch doesn't fix the underlying issue, but it mitigates it by only
emitting global-variable SPIRV intrinsics on the functions that use it.
If the global is not referenced by any function, we just pick the first
function definition.
With this patch, the example in
https://github.com/llvm/llvm-project/issues/170339 drops from ~33Gb of
maximum resident set size to less than ~590Mb. And compile time goes
from 2:09min to ~5secs.
The changes in the tests are due to changes in the order in which the
instructions appear.
In `fun-with-aggregate-arg-in-const-init.ll` 2 duplicated
`OpConstantComposite` are emitted anymore.
Replace uses of getAllocatedType() in PromoteMemoryToRegister.cpp with
type tracking from actual loads and stores. This makes the promotion
logic more semantic - it now checks that all loads/stores use a
consistent type rather than requiring them to match the alloca's
declared type.
Changes:
- isAllocaPromotable() now tracks the first load/store type seen and
ensures all subsequent accesses use the same type
- AllocaInfo gains a ValueType field populated during AnalyzeAlloca()
- PromoteMem2Reg tracks AllocaValueTypes alongside other per-alloca info
- PHI nodes and UndefValues are created using the tracked type
This is semantically more permissive - an alloca declared as i64 but
only accessed as i32 is now promotable. This is correct because the
alloca is just a blob of memory; what matters for Mem2Reg is consistent
access patterns.
Test changes:
- asan-stack-safety.ll: Changed loads/stores to volatile to prevent
promotion while preserving ASAN stack safety analysis behavior
- SPIRV pointer tests (array-skips-gep.ll, load-struct.ll,
store-struct.ll, store-to-array-first-element.ll): Added escape calls to
prevent alloca promotion, as these tests verify SPIRV backend handling
of Function-storage-class pointers
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The IR can store to the first element of an array the same way it stores
to the first element of a struct by specifying the base pointer. This
commit fixes the pointercast legalization pass to support this.
Refines OpName emission to only target Global Variables, Functions,
Function Parameters, Local Variables (allocas/phis), and Basic Blocks.
This reduces binary size and clutter by avoiding OpName for every
intermediate instruction (arithmetic, casts, etc.), while preserving
readability for interfaces and program structure.
Also updates the test suite to align with this change:
- Removes OpName checks for intermediate instructions.
- Adds side-effects (e.g., volatile stores) to tests where instructions
were previously kept alive solely by their OpName usage.
- Updates checks to use generic ID matching where specific names are no
longer available.
- Adds debug-info/opname-filtering.ll to verify the new policy.
fixes#170241
PR #169090 updated vector swizzle elements individually for HLSL This is
because rawbuffer writes via rawBufferStore can cause datarace in
concurrent writing the way vectors are written all at once. This means
we needed individual writes per element. So that means we need to be
able to GEP into an element of a vector.
The SPIRV backend did not support this pattern. SPIRV assumes Composite
types (ie Vectors, Structs, and Arrays) only for Ptr legalization in
it's store transformations via transformStore.
Fixing things at the point of ptr legalziation for the store would be
too late because we would have lost critical ptr type information still
available in LLVM IR. Instead what we needed to do is teach the
walkLogicalAccessChain used by buildLogicalAccessChainFromGEP to
converts a byte-offset GEP on an i8-pointer into a logical SPIR-V
composite access chain.
In short:
The fundamental issue is that walkLogicalAccessChain currently refuses
to handle vector-based logical GEPs, but Clang’s new HLSL path produces
exactly that pattern:
- you have a logical i8* GEP that represents indexing into the elements
of a <4 x i32>.
We need to teach walkLogicalAccessChain to treat a FixedVectorType
similarly to an array of its element type. So in walkLogicalAccessChain
replace the “give up on vector” part with logic that:
- Interprets the byte Offset as indexing into the vector elements.
- Uses the vector’s element type and number of elements.
This patch does two things:
1. it extends the aggregate arg / ret replacement transform to work on
indirect calls / pointers to function. It is somewhat spread out as
retrieving the original function type is needed in a few places. In
general, we should rethink / rework the entire infrastructure around
aggregate arg/ret handling, using an opaque target specific type rather
than i32;
2. it enables global variables of pointer to function type, and, more
specifically, global variables of a aggregate type (arrays / structures)
with pointer to function elements.
This also exposes some issues in how we handle pointers to function and
lowering indirect function calls, primarily around not using the program
address space. These will be handled in a subsequent patch as they'll
require somewhat more intrusive surgery, possibly involving modifying
the data layout.
At the moment AMDGCN flavoured SPIRV uses the SPIRV ABI with some tweaks
revolving around passing aggregates as direct. This is problematic in
multiple ways:
- it leads to divergence from code compiled for a concrete target, which
makes it difficult to debug;
- it incurs a run time cost, when dealing with larger aggregates;
- it incurs a compile time cost, when dealing with larger aggregates.
This patch switches over AMDGCN flavoured SPIRV to implement the AMDGPU
ABI (except for dealing with variadic functions, which will be added in
the future). One additional complication (and the primary motivation
behind the current less than ideal state of affairs) stems from `byref`,
which AMDGPU uses, not being expressible in SPIR-V. We deal with this by
CodeGen-ing for `byref`, lowering it to the `FuncParamAttr ByVal` in
SPIR-V, and restoring it when doing reverse translation from AMDGCN
flavoured SPIR-V.
The instruction selection pass for SPIR-V now performs dead code
elimination (DCE).
This change removes unused instructions, leading to more optimized
SPIR-V output.
As a consequence of this, several tests were updated to ensure their
continued
correctness and to prevent previously tested code from being optimized
away.
Specifically:
- Many tests now store computed values into global variables to ensure
they are
not eliminated by DCE, allowing their code generation to be verified.
- The test `keep-tracked-const.ll` was removed because it no longer
tested
its original intent. The check statements in this test were for
constants
generated when expanding a G_TRUNC instruction, which is now removed by
DCE
instead of being expanded.
- A new test, `remove-dead-type-intrinsics.ll`, was added to confirm
that dead
struct types are correctly removed by the compiler.
These updates improve the SPIR-V backends optimization capabilities and
maintain the robustness of the test suite.
---------
Co-authored-by: Nathan Gauër <github@keenuts.net>
This commit adds support for legalizing pointer casts between array and
vector types within the SPIRV backend.
This is necessary to handle cases where a vector is loaded from or
stored to an array, which can occur with HLSL matrix types.
The following changes are included:
- Added to load a vector from an array.
- Added to store a vector to an array.
- Added the test case to verify the functionality.
Due to limitations in GISel / IRTranslator, the SPIR-V BE replaces aggregate function args with `i32` placeholders, which are subsequently used to retrieve the original type after IR translation, from metadata. Due to what appears to be an oversight, the current implementation only handles a single mutation, as it does not traverse the metadata, but rather only takes the first operand. This patch addresses that limitation by correctly iterating the metadata.
The previous check for vector bitcasts in `loadVectorFromVector` only
compared the number of elements, which is insufficient when the element
types differ. This can lead to incorrect assumptions about the validity
of the cast.
This commit replaces the element count check with a comparison of the
total size of the vectors in bits. This ensures that the bitcast is
only performed between vectors of the same size, preventing potential
miscompilations.
Part of https://github.com/llvm/llvm-project/issues/153091
Generate `OpBitCast` instruction for pointer cast operation if the
element type is different.
The HLSL for the unit test is
```hlsl
StructuredBuffer<uint2> In : register(t0);
RWStructuredBuffer<double2> Out : register(u2);
[numthreads(1,1,1)]
void main() {
Out[0] = asdouble(In[0], In[1]);
}
```
Resolves https://github.com/llvm/llvm-project/issues/153513
Removes uniformity bit from resource initialization intrinsics `llvm.{dx|spv}.resource.handlefrombinding` and `llvm.{dx|spv}.resource.handlefromimplicitbinding`. The flag currently always set to `false`. It should be derived from resource analysis and not provided by codegen.
Closes#135452
Prior to this patch, when `NumElems` was 0, `OpTypeRuntimeArray` was
directly generated, but it requires `Shader` capability, so it can only
be generated if `Shader` env is being used. We have observed a pattern
of using unbound arrays that translate into `[0 x ...]` types in OpenCL,
which implies `Kernel` capability, so `OpTypeRuntimeArray` should not be
used. To prevent this scenario, this patch simplifies GEP instructions
where type is a 0-length array and the first index is also 0. In such
scenario, we effectively drop the 0-length array and the first index.
Additionally, the newly added test prior to this patch was generating a
module with both `Shader` and `Kernel` capabilities at the same time,
but they're incompatible. This patch also fixes that.
Finally, prior to this patch, the newly added test was adding `Shader`
capability to the module even with the command line flag
`--avoid-spirv-capabilities=Shader`. This patch also has a fix for that.
Pointers and GEP are untyped. SPIR-V required structured OpAccessChain.
This means the backend will have to determine a good way to retrieve the
structured access from an untyped GEP. This is not a trivial problem,
and needs to be addressed to have a robust compiler.
The issue is other workstreams relies on the access chain deduction to
work. So we have 2 options:
- pause all dependent work until we have a good chain deduction.
- submit this limited fix to we can work on both this and other features
in parallel.
Choice we want to make is #2: submitting this **knowing this is not a
good** fix. It only increase the number of patterns we can work with,
thus allowing others to continue working on other parts of the backend.
This patch as-is has many limitations:
- If cannot robustly determine the depth of the structured access from a
GEP. Fixing this would require looking ahead at the full GEP chain.
- It cannot always figure out the correct access indices, especially
with dynamic indices. This will require frontend collaboration.
Because we know this is a temporary hack, this patch only impacts the
logical SPIR-V target. Physical SPIR-V, which can rely on pointer cast
remains on the old method.
Related to #145002
This is a quick fix to make progress to the backend until we get a
proper type scavenging system.
The previous code was only checking the type if the resource was used
once. Slightly changed the code to look to all usages, and get the first
type.
This will certainly break in other cases, but it allows us to move
forward for now until we rewrite the type scavenging to handle untyped
GEP/ptradd correctly.
Related to #145002
The SPIR-V backend does not have access to the original name of a
resource in the source, so it tries to create a name. This leads to some
problems with reflection.
That is why start to pass the name of the resource from Clang to the
SPIR-V backend.
Fixes#138533
Current implementation outputs opcode is an immediate but spirv-tools
requires that the name of the operation without "Op" is needed for the
instruction OpSpecConstantOp
that is if the opcode is OpBitcast the instruction must be
`%1 = OpSpecConstantOp %6 Bitcast %17`
instead of
`%1 = OpBitcast %6 124 %17`
[refer this commit for more
info](0f166be68d)
---------
Co-authored-by: Dmitry Sidorov <dmitry.sidorov@intel.com>
Co-authored-by: Ebin-McW <ebin.jose@multicorewareinc.com>
1. There are failed LIT tests at the moment due to type inference
errors.
```
Failed Tests (3):
LLVM :: CodeGen/SPIRV/pointers/ptr-eq-types.ll
LLVM :: CodeGen/SPIRV/validate/sycl-hier-par-basic.ll
LLVM :: CodeGen/SPIRV/validate/sycl-tangle-group-algorithms.ll
```
This PR improves type inference to fix the errors.
2. The following tests start passing:
```
Unexpectedly Passed Tests (2):
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast-2.ll
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast.ll
```
This PR removes XFAILS in those two test cases.
This commit enables a pass in the backend which propagates the addrspace
of the pointers down to the last use, making sure the addrspace remains
consistent, and thus stripping any addrspacecast. This is required to
lower LLVM-IR to logical SPIR-V, which does not support generic
pointers.
This is now required as HLSL emits several address spaces, and thus
addrspacecasts in some cases:
Example 1: resource access
```llvm
%handle = tail call target("spirv.VulkanBuffer", ...)
%rptr = @llvm.spv.resource.getpointer(%handle, ...);
%cptr = addrspacecast ptr addrspace(11) %rptr to ptr
%fptr = load i32, ptr %cptr
```
Example 2: object methods
```llvm
define void @objectMethod(ptr %this) {
}
define void @foo(ptr addrspace(11) %object) {
call void @objectMethod(ptr addrspacecast(addrspace(11) %object to ptr));
}
```
Adds support the the spv.gep intrinsic to the spv ptrcast legalization
step. Those intrinsics are generated by the backend thus not directly
visible in the tests.
This is a pre-requisite to implement addrspacecast legalization for
logical SPIR-V.
This PR is to thoroughly rework duplicate tracker implementation and
tracking of IR entities and types. These are legacy parts of the project
resulting in an extremely bloated intermediate representation and
computational delays due to inefficient data flow and structure choices.
Main results of the rework:
1) Improved compile-time performance. The reference binary LLVM IR used
to measure speed gains in
https://github.com/llvm/llvm-project/pull/120415 shows ~x5 speed up also
after this PR. The timing before this PR is ~42s and after this PR it's
~7.5s. In total this PR and the previous overhaul of the module analysis
in https://github.com/llvm/llvm-project/pull/120415 results in ~x25
speed improvement.
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj
real 0m7.545s
user 0m6.685s
sys 0m0.859s
```
2) Less bloated intermediate representation of internal translation
steps. Elimination of `spv_track_constant` intrinsic usage for scalar
constants, rework of `spv_assign_name`, removal of the gMIR `GET_XXX`
pseudo code and a smaller number of generated `ASSIGN_TYPE` pseudo codes
substantially decrease volume of data generated during translation.
3) Simpler code and easier maintenance. The duplicate tracker
implementation is simplified, as well as other features.
4) Numerous fixes of issues and logical flaws in different passes. The
main achievement is rework of the duplicate tracker itself that had
never guaranteed a correct caching of LLVM IR entities, rarely and
randomly returning stale/incorrect records (like, remove an instruction
from gMIR but still refer to it). Other fixes comprise consistent
generation of OpConstantNull, assigning types to newly created
registers, creation of integer/bool types, and other minor fixes.
5) Numerous fixes of LIT tests: mainly CHECK-DAG to properly reflect
SPIR-V spec guarantees, `{{$}}` at the end of constants to avoid
matching of substrings, and XFAILS for `SPV_INTEL_long_composites` test
cases, because the feature is not completed in full yet and doesn't
generate a requested by the extension sequence of instructions.
6) New test cases are added.
OpenCL is allowed to cast pointers, meaning they can resolve some type
mismatches this way. In logical SPIR-V, those are restricted. This new
pass legalizes such pointer cast when targeting logical SPIR-V.
For now, this pass supports 3 cases we witnessed:
- loading a vec3 from a vec4*.
- loading a scalar from a vec*.
- loading the 1st element of an array.
---------
Co-authored-by: Steven Perron <stevenperron@google.com>
The following reproducer demonstrates the issue with invalid definition
of GEP results during type inference
```
define spir_kernel void @foo(i1 %fl, i64 %idx, ptr addrspace(1) %dest, ptr addrspace(3) %src) {
%p1 = getelementptr inbounds i8, ptr addrspace(1) %dest, i64 %idx
%res = tail call spir_func target("spirv.Event") @_Z22__spirv_GroupAsyncCopyjPU3AS1iPU3AS3Kimm9ocl_event(i32 2, ptr addrspace(1) %p1, ptr addrspace(3) %src, i64 128, i64 1, target("spirv.Event") zeroinitializer)
ret void
}
declare dso_local spir_func target("spirv.Event") @_Z22__spirv_GroupAsyncCopyjPU3AS1iPU3AS3Kimm9ocl_event(i32, ptr addrspace(1), ptr addrspace(3), i64, i64, target("spirv.Event"))
```
Here `OpGroupAsyncCopy` expects i32* arguments and type inference fails
to set a correct type of the GEP result `%p1`, because it is an argument
of `OpGroupAsyncCopy`.
This PR fixes the issue by preventing type change of GEP results in type
inference.
This PR is to address legacy issues with module analysis that currently
uses a complicated and not so efficient approach to trace dependencies
between SPIR-V id's via a duplicate tracker data structures and an
explicitly built dependency graph. Even a quick performance check
without any specialized benchmarks points to this part of the
implementation as a biggest bottleneck.
This PR specifically:
* eliminates a need to build a dependency graph as a data structure,
* updates the test suite (mainly, by fixing incorrect CHECK's referring
to a hardcoded order of definitions, contradicting the spec requirement
to allow certain definitions to go "in any order", see
https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_logical_layout_of_a_module),
* improves function pointers implementation so that it now passes
EXPENSIVE_CHECKS (thus removing 3 XFAIL's in the test suite).
As a quick sanity check of whether goals of the PR are achieved, we can
measure time of translation for any big LLVM IR. While testing the PR in
the local development environment, improvements of the x5 order have
been observed.
For example, the SYCL test case "group barrier" that is a ~1Mb binary IR
input shows the following values of the naive performance metric that we
can nevertheless apply here to roughly estimate effects of the PR.
before the PR:
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj
real 3m33.241s
user 3m14.688s
sys 0m18.530s
```
after the PR
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj
real 0m42.031s
user 0m38.834s
sys 0m3.193s
```
Next work should probably address Duplicate Tracker further, as it needs
analysis now from the perspective of what parts of it are not necessary
now, after changing the approach to implementation of the module
analysis step.
This PR fixes:
* emission of OpNames (added newly inserted internal intrinsics and
basic blocks)
* emission of function attributes (SRet is added)
* implementation of SPV_INTEL_optnone so that it emits OptNoneINTEL
Function Control flag, and add implementation of the SPV_EXT_optnone
SPIR-V extension.
This PR resolved the following issues:
(1) There are rare but possible cases when there are bitcasts between
pointers intertwined in a sophisticated way with loads, stores, function
calls and other instructions that are part of type deduction. In this
case we must account for inserted bitcasts between pointers rather than
just ignore them.
(2) Null pointers have the same constant representation but different
types. Type info from Intrinsic::spv_track_constant() refers to the
opaque (untyped) pointer, so that each MF/v-reg pair would fall into the
same Const record in Duplicate Tracker and would be represented by a
single OpConstantNull instruction, unless we use precise pointee type
info. We must be able to distinguish one constant (null) pointer from
another to avoid generating invalid code with inconsistent types of
operands.
Re-land of #116636
Adds a new address spaces: hlsl_private. Variables with such address
space will be emitted with a Private storage class.
This is useful for variables global to a SPIR-V module, since up to now,
they were still emitted with a Function storage class, which is wrong.
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
Adds a new address spaces: `hlsl_private`. Variables with such address
space will be emitted with a `Private` storage class.
This is useful for variables global to a SPIR-V module, since up to now,
they were still emitted with a `Function` storage class, which is wrong.
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
Goals of the PR are:
* to ensure that correct types are applied to virtual registers which
were used as return values in call lowering. A reproducer is attached as
a new test case, before the PR it fails because spirv-val considers
output invalid due to wrong result/operand types in OpPhi's;
* improve type inference by speeding up postprocessing of types: by
limiting iterations by checking what remains to process, and processing
each instruction just once for any number of operands with uncomplete
types;
* improve type inference by more accurate work with uncomplete types
(pass uncomplete property to dependent operands, ensure consistency of
uncomplete-types data structure);
* change processing order and add traversing of PHI nodes when type
inference apply instructions results to specify/update/cast operands
type (fixes an issue with OpPhi's result type mismatch with operand
types).
This PR is to solve several intertwined issues with type inference while
adding support for builtins for OpIAddCarry and OpISubBorrow:
* OpIAddCarry and OpISubBorrow generation in a way of supporting SPIR-V
friendly builtins `__spirv_...` -- introduces a new element to account
for, namely, `ptr sret (%struct) %0` argument that is a place to put a
result of the instruction;
* fix early definition of SPIR-V types during call lowering -- namely,
the goal of the PR is to ensure that correct types are applied to
virtual registers which were used as arguments in call lowering and so
caused early definition of SPIR-V types; reproducers are attached as a
new test cases;
* improve parsing of builtin names (e.g., understand a name of a kind
`"anon<int, int> __spirv_IAddCarry<int, int>(int, int)"` that was
incorrectly parsed as `anon` before the PR);
* improve type inference and fix access to erased from parent after
visit instructions -- before the PR visiting of instructions in
emitintrinsics pass replaced old alloca's, bitcast's, etc. instructions
with a newly generated internal SPIR-V intrinsics and after erasing old
instructions there were still references to them in a postprocessing
working list, while records for newly deduced pointee types were lost;
this PR fixes the issue by adding as consistent wrt. internal data
structures action `SPIRVEmitIntrinsics::replaceAllUsesWith()` that fixes
above mentioned problems;
* LLVM IR add/sub instructions result in logical SPIR-V instructions
when applied to bool type;
* fix validation of pointer types for frexp and lgamma_r,
* fix hardcoded reference to AS0 as a Function storage class in
lib/Target/SPIRV/SPIRVBuiltins.cpp -- now it's
`storageClassToAddressSpace(SPIRV::StorageClass::Function)`,
* re-use the same OpTypeStruct for two identical references to struct's
in arithmetic with overflow instructions.
OpenCL Extended Instruction Set Specification defines relations between
return/operand types and pointee type of pointer arguments in case of
remquo, fract, frexp, lgamma_r, modf, sincos and prefetch instructions
(https://registry.khronos.org/SPIR-V/specs/unified1/OpenCL.ExtendedInstructionSet.100.html).
This PR ensures correct pointee types of those OpenCL Extended
Instructions' pointer arguments.
The goal of the PR is to ensure that if module contains functions with
mutated signature (due to preprocessing of aggregate types), functions
still are going through re-creating of function type to preserve pointee
type information for arguments.
This fixes a bug when a module with (1) a function having aggregate
arguments and/or return, and (2) at least two functions with signatures
different only wrt. pointee types is translated so that one of two
similar functions gets an incorrect OpFunctionParameter type that is
different from the corresponding OpTypeFunction definition.
A reproducer is attached as a new test case.
This PR implements instruction selection for G_BITCAST on an earlier
stage to avoid MachineVerifier complains on subtle semantics difference
between G_BITCAST and OpBitcast.
We do instruction selections for OpBitcast after IR Translation instead
of calling MIB.buildBitcast() generating the general op code G_BITCAST,
because when MachineVerifier validates G_BITCAST we see a check of a
kind: 'if Source Type is equal to Destination Type then report error
"bitcast must change the type"'. This doesn't take into account the
notion of a typed pointer that is important for SPIR-V where a user may
and should use bitcast between pointers with different pointee types
(https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#OpBitcast).
It's important for correct lowering in SPIR-V, because interpretation of
the data type is not left to instructions that utilize the pointer, but
encoded by the pointer declaration, and the SPIRV target can and must
handle the declaration and use of pointers that specify the type of data
they point to.
It's not feasible to improve validation of G_BITCAST using just
information provided by low level types of source and destination.
Therefore we don't produce G_BITCAST as the general op code with
semantics different from OpBitcast, but rather lower to OpBitcast
immediately.
See discussion in https://github.com/llvm/llvm-project/pull/110270 for
even more context.
This PR reworks implementation of OpSpecConstantOp with ptr-cast
operation (PtrCastToGeneric, GenericCastToPtr). Previous implementation
didn't take into account a lot of use cases, including multiple
inclusion of pointers, reference to a pointer from OpName, etc. A
reproducer is attached as a new test case.
This PR also fixes wrong type inference for IR patterns which generate
new virtual registers without SPIRV type. Previous implementation
assumed always that result has the same address space as a source that
is not the fact, and, for example, led to impossibility to emit a
ptr-cast operation in the reproducer, because wrong type inference
rendered source and destination with the same address space, eliminating
translation of G_ADDRSPACE_CAST.
This PR improves type inference and fixes inconsistency between
previously deduced element type of a pointer and function's return type.
It fixes https://github.com/llvm/llvm-project/issues/109401 by ensuring
that OpPhi is consistent with respect to operand types.
This PR continues https://github.com/llvm/llvm-project/pull/101732
changes in virtual register processing aimed to improve correctness of
emitted MIR between passes from the perspective of MachineVerifier.
Namely, the following changes are introduced:
* register classes (lib/Target/SPIRV/SPIRVRegisterInfo.td) and
instruction patterns (lib/Target/SPIRV/SPIRVInstrInfo.td) are corrected
and simplified (by removing unnecessary sophisticated options) -- e.g.,
this PR gets rid of duplicating 32/64 bits patterns, removes ANYID
register class and simplifies definition of the rest of register
classes,
* hardcoded LLT scalar types in passes before instruction selection are
corrected -- the goal is to have correct bit width before instruction
selection, and use 64 bits registers for pattern matching in the
instruction selection pass; 32-bit registers remain where they are
described in such terms by SPIR-V specification (like, for example,
creation of virtual registers for scope/mem semantics operands),
* rework virtual register type/class assignment for calls/builtins
lowering,
* a series of minor changes to fix validity of emitted code between
passes:
- ensure that that bitcast changes the type,
- fix the pattern for instruction selection for OpExtInst,
- simplify inline asm operands usage,
- account for arbitrary integer sizes / update legalizer rules;
* add '-verify-machineinstrs' to existed test cases.
See also https://github.com/llvm/llvm-project/issues/88129 that this PR
may resolve.
This PR fixes a great number of issues reported by MachineVerifier and,
as a result, reduces a number of failed test cases for the mode with
expensive checks set on from ~200 to ~57.
This PR contains changes in virtual register processing aimed to improve
correctness of emitted MIR between passes from the perspective of
MachineVerifier. This potentially helps to detect previously missed
flaws in code emission and harden the test suite. As a measure of
correctness and usefulness of this PR we may use a mode with expensive
checks set on, and MachineVerifier reports problems in the test suite.
In order to satisfy Machine Verifier requirements to MIR correctness not
only a rework of usage of virtual registers' types and classes is
required, but also corrections into pre-legalizer and instruction
selection logics. Namely, the following changes are introduced:
* scalar virtual registers have proper bit width,
* detect register class by SPIR-V type,
* add a superclass for id virtual register classes,
* fix Tablegen rules used for instruction selection,
* fixes of minor existed issues (missed flag for proper representation
of a null constant for OpenCL vs. HLSL, wrong usage of integer virtual
registers as a synonym of any non-type virtual register).
This PR improves type inference of operand presented by opaque pointers
and aggregate types:
* tries to restore original function return type for aggregate types so
that it's possible to deduce a correct type during emit-intrinsics step
(see llvm/test/CodeGen/SPIRV/SpecConstants/restore-spec-type.ll for the
reproducer of the previously existed issue when spirv-val found a
mismatch between object and ptr types in OpStore due to the incorrect
aggregate types tracing),
* explores untyped pointer operands of store to deduce correct pointee
types,
* creates an extension type to track pointee types from emit-intrinsics
step and further instead of direct and naive usage of TypePointerType
that led previously to crashes due to ban of creation of Value of
TypePointerType type,
* tracks instructions with uncomplete type information and tries to
improve their type info after pass calculated types for all machine
functions (it doesn't traverse a code but rather checks only those
instructions which were tracked as uncompleted),
* address more cases of removing unnecessary bitcasts (see, for example,
changes in test/CodeGen/SPIRV/transcoding/OpGenericCastToPtr.ll where
`CHECK-SPIRV-NEXT` in LIT checks show absence of unneeded bitcasts and
unmangled/mangled versions have proper typing now with equivalent type
info),
* address more cases of well known types or relations between types
within instructions (see, for example, atomic*.ll test cases and
Event-related test cases for improved SPIR-V code generated by the
Backend),
* fix the issue of removing unneeded ptrcast instructions in
pre-legalizer pass that led to creation of new assign-type instructions
with the same argument as source in ptrcast and caused errors in type
inference (the reproducer `complex.ll` test case is added to the PR).
This PR fixes the issue
https://github.com/llvm/llvm-project/issues/96614 by improve pattern
matching and tracking of constant integers. The attached test is
successful if it doesn't crash and generate valid SPIR-V code for both
32 and 64 bits targets.
