59 Commits

Author SHA1 Message Date
Steven Perron
834a3cca31
[SPIRV] Handle ptrcast between array and vector types (#166418)
This commit adds support for legalizing pointer casts between array and
vector types within the SPIRV backend.

This is necessary to handle cases where a vector is loaded from or
stored to an array, which can occur with HLSL matrix types.

The following changes are included:
- Added  to load a vector from an array.
- Added  to store a vector to an array.
- Added the  test case to verify the functionality.
2025-11-12 10:27:31 -05:00
Alex Voicu
513334faec
[NFC][SPIRV] Fix function type recovery (#165934)
Due to limitations in GISel / IRTranslator, the SPIR-V BE replaces aggregate function args with `i32` placeholders, which are subsequently used to retrieve the original type after IR translation, from metadata. Due to what appears to be an oversight, the current implementation only handles a single mutation, as it does not traverse the metadata, but rather only takes the first operand. This patch addresses that limitation by correctly iterating the metadata.
2025-11-03 19:30:04 +02:00
Steven Perron
9f72fab490
[SPIRV] Fix vector bitcast check in LegalizePointerCast (#164997)
The previous check for vector bitcasts in `loadVectorFromVector` only
compared the number of elements, which is insufficient when the element
types differ. This can lead to incorrect assumptions about the validity
of the cast.

This commit replaces the element count check with a comparison of the
total size of the vectors in bits. This ensures that the bitcast is
only performed between vectors of the same size, preventing potential
miscompilations.

Part of https://github.com/llvm/llvm-project/issues/153091
2025-10-31 10:12:26 -04:00
Lucie Choi
b0ad9c293a
[SPIR-V] Fix asdouble issue in SPIRV codegen to correctly generate OpBitCast instruction. (#161891)
Generate `OpBitCast` instruction for pointer cast operation if the
element type is different.

The HLSL for the unit test is 
```hlsl
StructuredBuffer<uint2> In : register(t0);

RWStructuredBuffer<double2> Out : register(u2);


[numthreads(1,1,1)]
void main() {
  Out[0] = asdouble(In[0], In[1]);
}
```

Resolves https://github.com/llvm/llvm-project/issues/153513
2025-10-03 16:02:06 -04:00
Helena Kotas
ddf8cd3c6c
[HLSL][DirectX] Remove uniformity bit from resource initialization intrinsics (#155332)
Removes uniformity bit from resource initialization intrinsics `llvm.{dx|spv}.resource.handlefrombinding` and `llvm.{dx|spv}.resource.handlefromimplicitbinding`. The flag currently always set to `false`. It should be derived from resource analysis and not provided by codegen.

Closes #135452
2025-08-28 09:12:36 -07:00
Marcos Maronas
cda4820270
[SPIRV] Do not use OpTypeRuntimeArray in Kernel env. (#149522)
Prior to this patch, when `NumElems` was 0, `OpTypeRuntimeArray` was
directly generated, but it requires `Shader` capability, so it can only
be generated if `Shader` env is being used. We have observed a pattern
of using unbound arrays that translate into `[0 x ...]` types in OpenCL,
which implies `Kernel` capability, so `OpTypeRuntimeArray` should not be
used. To prevent this scenario, this patch simplifies GEP instructions
where type is a 0-length array and the first index is also 0. In such
scenario, we effectively drop the 0-length array and the first index.

Additionally, the newly added test prior to this patch was generating a
module with both `Shader` and `Kernel` capabilities at the same time,
but they're incompatible. This patch also fixes that.

Finally, prior to this patch, the newly added test was adding `Shader`
capability to the module even with the command line flag
`--avoid-spirv-capabilities=Shader`. This patch also has a fix for that.
2025-08-05 15:10:15 +02:00
Nathan Gauër
7c1b948e30
[SPIR-V] Fix some GEP legalization (#150943)
Pointers and GEP are untyped. SPIR-V required structured OpAccessChain.
This means the backend will have to determine a good way to retrieve the
structured access from an untyped GEP. This is not a trivial problem,
and needs to be addressed to have a robust compiler.

The issue is other workstreams relies on the access chain deduction to
work. So we have 2 options:
 - pause all dependent work until we have a good chain deduction.
- submit this limited fix to we can work on both this and other features
in parallel.

Choice we want to make is #2: submitting this **knowing this is not a
good** fix. It only increase the number of patterns we can work with,
thus allowing others to continue working on other parts of the backend.

This patch as-is has many limitations:
- If cannot robustly determine the depth of the structured access from a
GEP. Fixing this would require looking ahead at the full GEP chain.
- It cannot always figure out the correct access indices, especially
with dynamic indices. This will require frontend collaboration.

Because we know this is a temporary hack, this patch only impacts the
logical SPIR-V target. Physical SPIR-V, which can rely on pointer cast
remains on the old method.

Related to #145002
2025-08-01 11:10:35 +02:00
Nathan Gauër
f45e6a2834
[SPIR-V] Fix emit intrinsic for resource type (#150224)
This is a quick fix to make progress to the backend until we get a
proper type scavenging system.
The previous code was only checking the type if the resource was used
once. Slightly changed the code to look to all usages, and get the first
type.

This will certainly break in other cases, but it allows us to move
forward for now until we rewrite the type scavenging to handle untyped
GEP/ptradd correctly.

Related to #145002
2025-07-24 16:45:33 +02:00
Steven Perron
bd33eef7f1
[HLSL][SPIRV] Use resource names (#143412)
The SPIR-V backend does not have access to the original name of a
resource in the source, so it tries to create a name. This leads to some
problems with reflection.
    
That is why start to pass the name of the resource from Clang to the
SPIR-V backend.
    
Fixes #138533
2025-06-13 12:21:38 -04:00
VISHAKH PRAKASH
fad1972d74
[SPIRV] FIX print the symbolic operand for opcode for the operation OpSpecConstantOp (#135756)
Current implementation outputs opcode is an immediate but spirv-tools
requires that the name of the operation without "Op" is needed for the
instruction OpSpecConstantOp
that is if the opcode is OpBitcast the instruction must be
`%1 = OpSpecConstantOp %6 Bitcast %17` 
instead of 
`%1 = OpBitcast %6 124 %17`

[refer this commit for more
info](0f166be68d)

---------

Co-authored-by: Dmitry Sidorov <dmitry.sidorov@intel.com>
Co-authored-by: Ebin-McW <ebin.jose@multicorewareinc.com>
2025-06-11 20:13:01 +02:00
Vyacheslav Levytskyy
8bd35ca412
[SPIR-V] Fix LIT tests, improve ICmpInst's type inference (#139726)
1. There are failed LIT tests at the moment due to type inference
errors.
```
Failed Tests (3):
  LLVM :: CodeGen/SPIRV/pointers/ptr-eq-types.ll
  LLVM :: CodeGen/SPIRV/validate/sycl-hier-par-basic.ll
  LLVM :: CodeGen/SPIRV/validate/sycl-tangle-group-algorithms.ll
```
This PR improves type inference to fix the errors.

2. The following tests start passing:
```
Unexpectedly Passed Tests (2):
  LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast-2.ll
  LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast.ll
```
This PR removes XFAILS in those two test cases.
2025-05-15 10:46:54 +02:00
Nathan Gauër
7d98b66e3d
[SPIR-V] Add InferAddrSpaces pass to the backend (#137766)
This commit enables a pass in the backend which propagates the addrspace
of the pointers down to the last use, making sure the addrspace remains
consistent, and thus stripping any addrspacecast. This is required to
lower LLVM-IR to logical SPIR-V, which does not support generic
pointers.

This is now required as HLSL emits several address spaces, and thus
addrspacecasts in some cases:

Example 1: resource access

```llvm
%handle = tail call target("spirv.VulkanBuffer", ...)
%rptr = @llvm.spv.resource.getpointer(%handle, ...);
%cptr = addrspacecast ptr addrspace(11) %rptr to ptr
%fptr = load i32, ptr %cptr
```

Example 2: object methods

```llvm
define void @objectMethod(ptr %this) {
}

define void @foo(ptr addrspace(11) %object) {
  call void @objectMethod(ptr addrspacecast(addrspace(11) %object to ptr));
}
```
2025-05-07 16:53:25 +02:00
Victor Lomuller
082598a64d
[SPIRV] Add intrinsic for OpGenericCastToPtrExplicit (#137626)
The patch adds an intrinsic to encode OpGenericCastToPtrExplicit and the
associated lowering logic.
2025-04-29 11:58:25 +02:00
Nathan Gauër
c1940cd0de
[SPIR-V] Add store legalization for ptrcast (#135369)
This commits adds handling for spv.ptrcast result being used in a store
instruction, modifying the store to operate on the source type.
2025-04-22 11:58:55 +02:00
Nathan Gauër
d8b0e616b7
[SPIR-V] Fix OpVectorShuffle operands on load (#135954)
The generated OpVectorShuffle was wrong, as the indices we pass are not
to select the vector to sample from, but the position in the vector.
2025-04-22 11:07:02 +02:00
Nathan Gauër
739062d2c3
[SPIR-V] Add spv.gep support for ptrcast legal (#134388)
Adds support the the spv.gep intrinsic to the spv ptrcast legalization
step. Those intrinsics are generated by the backend thus not directly
visible in the tests.
This is a pre-requisite to implement addrspacecast legalization for
logical SPIR-V.
2025-04-08 12:55:37 +02:00
Vyacheslav Levytskyy
54cc4141e4
[SPIR-V] Rework duplicate tracker and tracking of IR entities and types to improve compile-time performance (#130605)
This PR is to thoroughly rework duplicate tracker implementation and
tracking of IR entities and types. These are legacy parts of the project
resulting in an extremely bloated intermediate representation and
computational delays due to inefficient data flow and structure choices.

Main results of the rework:

1) Improved compile-time performance. The reference binary LLVM IR used
to measure speed gains in
https://github.com/llvm/llvm-project/pull/120415 shows ~x5 speed up also
after this PR. The timing before this PR is ~42s and after this PR it's
~7.5s. In total this PR and the previous overhaul of the module analysis
in https://github.com/llvm/llvm-project/pull/120415 results in ~x25
speed improvement.
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj

real    0m7.545s
user    0m6.685s
sys     0m0.859s
```

2) Less bloated intermediate representation of internal translation
steps. Elimination of `spv_track_constant` intrinsic usage for scalar
constants, rework of `spv_assign_name`, removal of the gMIR `GET_XXX`
pseudo code and a smaller number of generated `ASSIGN_TYPE` pseudo codes
substantially decrease volume of data generated during translation.

3) Simpler code and easier maintenance. The duplicate tracker
implementation is simplified, as well as other features.

4) Numerous fixes of issues and logical flaws in different passes. The
main achievement is rework of the duplicate tracker itself that had
never guaranteed a correct caching of LLVM IR entities, rarely and
randomly returning stale/incorrect records (like, remove an instruction
from gMIR but still refer to it). Other fixes comprise consistent
generation of OpConstantNull, assigning types to newly created
registers, creation of integer/bool types, and other minor fixes.

5) Numerous fixes of LIT tests: mainly CHECK-DAG to properly reflect
SPIR-V spec guarantees, `{{$}}` at the end of constants to avoid
matching of substrings, and XFAILS for `SPV_INTEL_long_composites` test
cases, because the feature is not completed in full yet and doesn't
generate a requested by the extension sequence of instructions.

6) New test cases are added.
2025-03-26 17:58:10 +01:00
Nathan Gauër
0aba8334f9
[SPIR-V] Handle struct member loading from ptrcast (#130089)
Adds support for loading the first element of an aggregate from a GEP
instruction & load to the member type.
2025-03-19 15:11:21 +01:00
Nathan Gauër
7c8b1275bc
[SPIR-V] Add pass to remove spv_ptrcast intrinsics (#128896)
OpenCL is allowed to cast pointers, meaning they can resolve some type
mismatches this way. In logical SPIR-V, those are restricted. This new
pass legalizes such pointer cast when targeting logical SPIR-V.

For now, this pass supports 3 cases we witnessed:
 - loading a vec3 from a vec4*.
 - loading a scalar from a vec*.
 - loading the 1st element of an array.

---------

Co-authored-by: Steven Perron <stevenperron@google.com>
2025-03-04 10:30:46 +01:00
Vyacheslav Levytskyy
494f67282f
[SPIR-V] Prevent type change of GEP results in type inference (#129250)
The following reproducer demonstrates the issue with invalid definition
of GEP results during type inference

```
define spir_kernel void @foo(i1 %fl, i64 %idx, ptr addrspace(1) %dest, ptr addrspace(3) %src) {
  %p1 = getelementptr inbounds i8, ptr addrspace(1) %dest, i64 %idx
  %res = tail call spir_func target("spirv.Event") @_Z22__spirv_GroupAsyncCopyjPU3AS1iPU3AS3Kimm9ocl_event(i32 2, ptr addrspace(1) %p1, ptr addrspace(3) %src, i64 128, i64 1, target("spirv.Event") zeroinitializer)
  ret void
}

declare dso_local spir_func target("spirv.Event") @_Z22__spirv_GroupAsyncCopyjPU3AS1iPU3AS3Kimm9ocl_event(i32, ptr addrspace(1), ptr addrspace(3), i64, i64, target("spirv.Event"))
```

Here `OpGroupAsyncCopy` expects i32* arguments and type inference fails
to set a correct type of the GEP result `%p1`, because it is an argument
of `OpGroupAsyncCopy`.

This PR fixes the issue by preventing type change of GEP results in type
inference.
2025-02-28 20:55:14 +01:00
Vyacheslav Levytskyy
b74d3e179d
[SPIR-V] Specify target environment in tests referring to the BuiltIn WorkgroupSize variable (#122755)
https://github.com/KhronosGroup/SPIRV-Tools/pull/5407 introduces a check
for WorkgroupSize variable to be a 3-component 32-bit int vector, and
indeed, we see this requirement in
https://registry.khronos.org/vulkan/specs/latest/man/html/WorkgroupSize.html#VUID-WorkgroupSize-WorkgroupSize-04427

However, OpenCL imposes different requirements, documented here:
https://registry.khronos.org/OpenCL/specs/3.0-unified/html/OpenCL_Env.html#_built_in_variables
OpenCL environment requires WorkgroupSize variable to have components of
size_t size that will be 32 or 64 depending on a target. This is the way
how the SPIR-V Backend implements this, by querying pointer size of the
current platform/target.

To allow spirv-val to account target environments difference, this PR
adds `--target-env <env>` to test cases referring to the BuiltIn
WorkgroupSize variable.
2025-01-14 17:44:07 +01:00
Vyacheslav Levytskyy
83c1d00311
[SPIR-V] Overhaul module analysis to improve translation speed and simplify the underlying logics (#120415)
This PR is to address legacy issues with module analysis that currently
uses a complicated and not so efficient approach to trace dependencies
between SPIR-V id's via a duplicate tracker data structures and an
explicitly built dependency graph. Even a quick performance check
without any specialized benchmarks points to this part of the
implementation as a biggest bottleneck.

This PR specifically:
* eliminates a need to build a dependency graph as a data structure,
* updates the test suite (mainly, by fixing incorrect CHECK's referring
to a hardcoded order of definitions, contradicting the spec requirement
to allow certain definitions to go "in any order", see
https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_logical_layout_of_a_module),
* improves function pointers implementation so that it now passes
EXPENSIVE_CHECKS (thus removing 3 XFAIL's in the test suite).

As a quick sanity check of whether goals of the PR are achieved, we can
measure time of translation for any big LLVM IR. While testing the PR in
the local development environment, improvements of the x5 order have
been observed.

For example, the SYCL test case "group barrier" that is a ~1Mb binary IR
input shows the following values of the naive performance metric that we
can nevertheless apply here to roughly estimate effects of the PR.

before the PR:
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj

real    3m33.241s
user    3m14.688s
sys     0m18.530s
```

after the PR

```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj

real    0m42.031s
user    0m38.834s
sys     0m3.193s
```

Next work should probably address Duplicate Tracker further, as it needs
analysis now from the perspective of what parts of it are not necessary
now, after changing the approach to implementation of the module
analysis step.
2025-01-07 10:42:23 +01:00
Vyacheslav Levytskyy
874b4fb6ad
[SPIR-V] Fix emission of debug and annotation instructions and add SPV_EXT_optnone SPIR-V extension (#118402)
This PR fixes:
* emission of OpNames (added newly inserted internal intrinsics and
basic blocks)
* emission of function attributes (SRet is added)
* implementation of SPV_INTEL_optnone so that it emits OptNoneINTEL
Function Control flag, and add implementation of the SPV_EXT_optnone
SPIR-V extension.
2024-12-03 16:18:06 +01:00
Vyacheslav Levytskyy
db4cbe5069
[SPIR-V] Fix generation of invalid SPIR-V in cases of of bitcasts between pointers and multiple null pointers used in the input LLVM IR (#118298)
This PR resolved the following issues:
(1) There are rare but possible cases when there are bitcasts between
pointers intertwined in a sophisticated way with loads, stores, function
calls and other instructions that are part of type deduction. In this
case we must account for inserted bitcasts between pointers rather than
just ignore them.
(2) Null pointers have the same constant representation but different
types. Type info from Intrinsic::spv_track_constant() refers to the
opaque (untyped) pointer, so that each MF/v-reg pair would fall into the
same Const record in Duplicate Tracker and would be represented by a
single OpConstantNull instruction, unless we use precise pointee type
info. We must be able to distinguish one constant (null) pointer from
another to avoid generating invalid code with inconsistent types of
operands.
2024-12-03 16:08:25 +01:00
Nathan Gauër
5f99eb9b13
[SPIR-V] Fixup storage class for global private (#118318)
Re-land of #116636
Adds a new address spaces: hlsl_private. Variables with such address
space will be emitted with a Private storage class.
This is useful for variables global to a SPIR-V module, since up to now,
they were still emitted with a Function storage class, which is wrong.

---------

Signed-off-by: Nathan Gauër <brioche@google.com>
2024-12-03 13:42:02 +01:00
Nathan Gauër
f8b4182f07
Revert "[SPIR-V] Fixup storage class for global private (#116636)" (#118312)
This reverts commit aa7fe1c10e5d6d0d3aacdb345fed995de413e142.
2024-12-02 17:32:54 +01:00
Nathan Gauër
aa7fe1c10e
[SPIR-V] Fixup storage class for global private (#116636)
Adds a new address spaces: `hlsl_private`. Variables with such address
space will be emitted with a `Private` storage class.
This is useful for variables global to a SPIR-V module, since up to now,
they were still emitted with a `Function` storage class, which is wrong.

---------

Signed-off-by: Nathan Gauër <brioche@google.com>
2024-12-02 16:17:44 +01:00
Vyacheslav Levytskyy
b5132b7d04
[SPIR-V] Improve type inference: fix types of return values in call lowering (#116609)
Goals of the PR are:
* to ensure that correct types are applied to virtual registers which
were used as return values in call lowering. A reproducer is attached as
a new test case, before the PR it fails because spirv-val considers
output invalid due to wrong result/operand types in OpPhi's;
* improve type inference by speeding up postprocessing of types: by
limiting iterations by checking what remains to process, and processing
each instruction just once for any number of operands with uncomplete
types;
* improve type inference by more accurate work with uncomplete types
(pass uncomplete property to dependent operands, ensure consistency of
uncomplete-types data structure);
* change processing order and add traversing of PHI nodes when type
inference apply instructions results to specify/update/cast operands
type (fixes an issue with OpPhi's result type mismatch with operand
types).
2024-11-29 20:44:25 +01:00
Vyacheslav Levytskyy
8ac46d6b4f
[SPIR-V] Implement builtins for OpIAddCarry/OpISubBorrow and improve/fix type inference (#115192)
This PR is to solve several intertwined issues with type inference while
adding support for builtins for OpIAddCarry and OpISubBorrow:
* OpIAddCarry and OpISubBorrow generation in a way of supporting SPIR-V
friendly builtins `__spirv_...` -- introduces a new element to account
for, namely, `ptr sret (%struct) %0` argument that is a place to put a
result of the instruction;
* fix early definition of SPIR-V types during call lowering -- namely,
the goal of the PR is to ensure that correct types are applied to
virtual registers which were used as arguments in call lowering and so
caused early definition of SPIR-V types; reproducers are attached as a
new test cases;
* improve parsing of builtin names (e.g., understand a name of a kind
`"anon<int, int> __spirv_IAddCarry<int, int>(int, int)"` that was
incorrectly parsed as `anon` before the PR);
* improve type inference and fix access to erased from parent after
visit instructions -- before the PR visiting of instructions in
emitintrinsics pass replaced old alloca's, bitcast's, etc. instructions
with a newly generated internal SPIR-V intrinsics and after erasing old
instructions there were still references to them in a postprocessing
working list, while records for newly deduced pointee types were lost;
this PR fixes the issue by adding as consistent wrt. internal data
structures action `SPIRVEmitIntrinsics::replaceAllUsesWith()` that fixes
above mentioned problems;
* LLVM IR add/sub instructions result in logical SPIR-V instructions
when applied to bool type;
* fix validation of pointer types for frexp and lgamma_r,
* fix hardcoded reference to AS0 as a Function storage class in
lib/Target/SPIRV/SPIRVBuiltins.cpp -- now it's
`storageClassToAddressSpace(SPIRV::StorageClass::Function)`,
* re-use the same OpTypeStruct for two identical references to struct's
in arithmetic with overflow instructions.
2024-11-14 15:30:05 +01:00
Vyacheslav Levytskyy
5a062191f7
[SPIR-V] Ensure correct pointee types of some OpenCL Extended Instructions' pointer arguments (#114846)
OpenCL Extended Instruction Set Specification defines relations between
return/operand types and pointee type of pointer arguments in case of
remquo, fract, frexp, lgamma_r, modf, sincos and prefetch instructions
(https://registry.khronos.org/SPIR-V/specs/unified1/OpenCL.ExtendedInstructionSet.100.html).
This PR ensures correct pointee types of those OpenCL Extended
Instructions' pointer arguments.
2024-11-06 12:44:53 +01:00
Vyacheslav Levytskyy
ebfafa2511
[SPIR-V] Fix OpFunctionParameter vs. OpTypeFunction types for pointer arguments when there are functions with aggregate arguments (#115044)
The goal of the PR is to ensure that if module contains functions with
mutated signature (due to preprocessing of aggregate types), functions
still are going through re-creating of function type to preserve pointee
type information for arguments.

This fixes a bug when a module with (1) a function having aggregate
arguments and/or return, and (2) at least two functions with signatures
different only wrt. pointee types is translated so that one of two
similar functions gets an incorrect OpFunctionParameter type that is
different from the corresponding OpTypeFunction definition.

A reproducer is attached as a new test case.
2024-11-06 11:17:45 +01:00
Vyacheslav Levytskyy
c616f24bcb
[SPIR-V] Do instruction selection for G_BITCAST on an earlier stage (#114216)
This PR implements instruction selection for G_BITCAST on an earlier
stage to avoid MachineVerifier complains on subtle semantics difference
between G_BITCAST and OpBitcast.

We do instruction selections for OpBitcast after IR Translation instead
of calling MIB.buildBitcast() generating the general op code G_BITCAST,
because when MachineVerifier validates G_BITCAST we see a check of a
kind: 'if Source Type is equal to Destination Type then report error
"bitcast must change the type"'. This doesn't take into account the
notion of a typed pointer that is important for SPIR-V where a user may
and should use bitcast between pointers with different pointee types
(https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#OpBitcast).

It's important for correct lowering in SPIR-V, because interpretation of
the data type is not left to instructions that utilize the pointer, but
encoded by the pointer declaration, and the SPIRV target can and must
handle the declaration and use of pointers that specify the type of data
they point to.

It's not feasible to improve validation of G_BITCAST using just
information provided by low level types of source and destination.
Therefore we don't produce G_BITCAST as the general op code with
semantics different from OpBitcast, but rather lower to OpBitcast
immediately.

See discussion in https://github.com/llvm/llvm-project/pull/110270 for
even more context.
2024-10-30 20:49:21 +01:00
Vyacheslav Levytskyy
3e79c7fec0
[SPIR-V] Implement OpSpecConstantOp with ptr-cast operation (#109979)
This PR reworks implementation of OpSpecConstantOp with ptr-cast
operation (PtrCastToGeneric, GenericCastToPtr). Previous implementation
didn't take into account a lot of use cases, including multiple
inclusion of pointers, reference to a pointer from OpName, etc. A
reproducer is attached as a new test case.

This PR also fixes wrong type inference for IR patterns which generate
new virtual registers without SPIRV type. Previous implementation
assumed always that result has the same address space as a source that
is not the fact, and, for example, led to impossibility to emit a
ptr-cast operation in the reproducer, because wrong type inference
rendered source and destination with the same address space, eliminating
translation of G_ADDRSPACE_CAST.
2024-10-01 10:47:15 +02:00
Vyacheslav Levytskyy
8bc8b84225
[SPIR-V] Fix inconsistency between previously deduced element type of a pointer and function's return type (#109660)
This PR improves type inference and fixes inconsistency between
previously deduced element type of a pointer and function's return type.
It fixes https://github.com/llvm/llvm-project/issues/109401 by ensuring
that OpPhi is consistent with respect to operand types.
2024-10-01 10:46:56 +02:00
Vyacheslav Levytskyy
67d3ef74b3
[SPIR-V] Rework usage of virtual registers' types and classes (#104104)
This PR continues https://github.com/llvm/llvm-project/pull/101732
changes in virtual register processing aimed to improve correctness of
emitted MIR between passes from the perspective of MachineVerifier.
Namely, the following changes are introduced:
* register classes (lib/Target/SPIRV/SPIRVRegisterInfo.td) and
instruction patterns (lib/Target/SPIRV/SPIRVInstrInfo.td) are corrected
and simplified (by removing unnecessary sophisticated options) -- e.g.,
this PR gets rid of duplicating 32/64 bits patterns, removes ANYID
register class and simplifies definition of the rest of register
classes,
* hardcoded LLT scalar types in passes before instruction selection are
corrected -- the goal is to have correct bit width before instruction
selection, and use 64 bits registers for pattern matching in the
instruction selection pass; 32-bit registers remain where they are
described in such terms by SPIR-V specification (like, for example,
creation of virtual registers for scope/mem semantics operands),
* rework virtual register type/class assignment for calls/builtins
lowering,
* a series of minor changes to fix validity of emitted code between
passes:
  - ensure that that bitcast changes the type,
  - fix the pattern for instruction selection for OpExtInst,
  - simplify inline asm operands usage,
  - account for arbitrary integer sizes / update legalizer rules;
* add '-verify-machineinstrs' to existed test cases.

See also https://github.com/llvm/llvm-project/issues/88129 that this PR
may resolve.

This PR fixes a great number of issues reported by MachineVerifier and,
as a result, reduces a number of failed test cases for the mode with
expensive checks set on from ~200 to ~57.
2024-08-22 09:40:27 +02:00
Vyacheslav Levytskyy
f9c98068c8
[SPIR-V] Rework usage of virtual registers' types and classes (#101732)
This PR contains changes in virtual register processing aimed to improve
correctness of emitted MIR between passes from the perspective of
MachineVerifier. This potentially helps to detect previously missed
flaws in code emission and harden the test suite. As a measure of
correctness and usefulness of this PR we may use a mode with expensive
checks set on, and MachineVerifier reports problems in the test suite.

In order to satisfy Machine Verifier requirements to MIR correctness not
only a rework of usage of virtual registers' types and classes is
required, but also corrections into pre-legalizer and instruction
selection logics. Namely, the following changes are introduced:
* scalar virtual registers have proper bit width,
* detect register class by SPIR-V type,
* add a superclass for id virtual register classes,
* fix Tablegen rules used for instruction selection,
* fixes of minor existed issues (missed flag for proper representation
of a null constant for OpenCL vs. HLSL, wrong usage of integer virtual
registers as a synonym of any non-type virtual register).
2024-08-12 15:49:43 +02:00
Vyacheslav Levytskyy
dbd00a5968
[SPIRV] Improve type inference of operand presented by opaque pointers and aggregate types (#98035)
This PR improves type inference of operand presented by opaque pointers
and aggregate types:
* tries to restore original function return type for aggregate types so
that it's possible to deduce a correct type during emit-intrinsics step
(see llvm/test/CodeGen/SPIRV/SpecConstants/restore-spec-type.ll for the
reproducer of the previously existed issue when spirv-val found a
mismatch between object and ptr types in OpStore due to the incorrect
aggregate types tracing),
* explores untyped pointer operands of store to deduce correct pointee
types,
* creates an extension type to track pointee types from emit-intrinsics
step and further instead of direct and naive usage of TypePointerType
that led previously to crashes due to ban of creation of Value of
TypePointerType type,
* tracks instructions with uncomplete type information and tries to
improve their type info after pass calculated types for all machine
functions (it doesn't traverse a code but rather checks only those
instructions which were tracked as uncompleted),
* address more cases of removing unnecessary bitcasts (see, for example,
changes in test/CodeGen/SPIRV/transcoding/OpGenericCastToPtr.ll where
`CHECK-SPIRV-NEXT` in LIT checks show absence of unneeded bitcasts and
unmangled/mangled versions have proper typing now with equivalent type
info),
* address more cases of well known types or relations between types
within instructions (see, for example, atomic*.ll test cases and
Event-related test cases for improved SPIR-V code generated by the
Backend),
* fix the issue of removing unneeded ptrcast instructions in
pre-legalizer pass that led to creation of new assign-type instructions
with the same argument as source in ptrcast and caused errors in type
inference (the reproducer `complex.ll` test case is added to the PR).
2024-07-11 07:16:29 +02:00
Vyacheslav Levytskyy
0d9172ecac
[SPIR-V] Improve pattern matching and tracking of constant integers (#96615)
This PR fixes the issue
https://github.com/llvm/llvm-project/issues/96614 by improve pattern
matching and tracking of constant integers. The attached test is
successful if it doesn't crash and generate valid SPIR-V code for both
32 and 64 bits targets.
2024-06-26 19:38:51 +02:00
Vyacheslav Levytskyy
b0efde6db0
[SPIR-V]: Improve pattern matching to recognize a composite constant to be a constant (#96286)
This PR is to fix https://github.com/llvm/llvm-project/issues/96285 by:
* improve pattern matching to recognize an aggregate constant to be a
constant
* do not emit Bitcast for an aggregate type
2024-06-24 17:19:08 +02:00
Vyacheslav Levytskyy
9a737109a0
[SPIR-V] Improve type inference, addrspacecast and dependencies between SPIR-V entities and required capability/extensions (#94626)
This PR continues https://github.com/llvm/llvm-project/pull/94467 and
contains fixes in emission of type intrinsics, constant recording and
corresponding test cases:
* type-deduce-global-dup.ll -- fix of integer constant emission on
32-bit platforms and correct type deduction for globals
* type-deduce-simple-for.ll -- fix of GEP translation (there was an
issue previously that led to incorrect translation/broken logic of
for-range implementation)

This PR also:
* fixes a cast between identical storage classes and updates the test
case to include validation run by spirv-val,
* ensures that Bitcast for pointers satisfies the requirement that the
address spaces must match and adds the corresponding test case,
* improve encode in Tablegen and decode in code of dependencies between
SPIR-V entities and required capability/extensions,
* prevent emission of identical OpTypePointer instructions.
2024-06-07 21:12:33 +02:00
Vyacheslav Levytskyy
486ea1ec6e
[SPIR-V] Fix pre-legalizer pass in SPIR-V Backend to support more gMIR opcode inserted by IRTranslator (#89890)
Translating global values, IRTranslator pass can sometimes generates
code patterns that require additional efforts during pre-legalization.
This PR addresses this problem to support G_PTRTOINT instruction used in
initialization of GV.
2024-04-24 11:28:29 +02:00
Michal Paszkowski
c071c1d3ab
[SPIR-V] New validation tests for pointer and primitive types (#89632)
This patch adds new tests mostly checking SPIR-V validation of pointer
and primitive types.
2024-04-24 01:26:42 -07:00
Vyacheslav Levytskyy
47e996d89d
[SPIR-V] Fix OpVariable instructions place in a function (#87554)
This PR:
* fixes OpVariable instructions place in a function (see
https://github.com/llvm/llvm-project/issues/66261),
* improves type inference,
* helps avoiding unneeded bitcasts when validating function call's

This allows to improve existing and add new test cases with more strict
checks. OpVariable fix refers to "All OpVariable instructions in a
function must be the first instructions in the first block" requirement
from SPIR-V spec.
2024-04-04 10:50:35 +02:00
Vyacheslav Levytskyy
6cce67a8f9
[SPIR-V] Fix validity of atomic instructions (#87051)
This PR fixes validity of atomic instructions and improves type
inference. More tests are able now to be accepted by `spirv-val`.
2024-04-02 10:59:18 +02:00
Vyacheslav Levytskyy
b7ac8fddb5
[SPIR-V] Improve type inference: deduce types of composite data structures (#86782)
This PR improves type inference in general and deduces types of
composite data structures in particular. Also added a way to insert a
bitcast to make a fun call valid in case of arguments types mismatch due
to opaque pointers type inference.

The attached test `pointers/nested-struct-opaque-pointers.ll`
demonstrates new capabilities: the SPIRV code emitted for this test is
now (1) valid in a sense of data field types and (2) accepted by
`spirv-val`.

More strict LIT checks, support of more composite data structures and
improvement of fun calls from the perspective of type correctness are
main todo's at the moment.
2024-03-28 08:08:06 +01:00
Vyacheslav Levytskyy
b0d03ccc08
[SPIR-V] Fix illegal OpConstantComposite instruction with non-const constituents in SPIR-V Backend (#86352)
This PR fixes illegal use of OpConstantComposite with non-constant
constituents. The test attached to the PR is able now to satisfy
`spirv-val` check. Before the fix SPIR-V Backend produced for the
attached test case a pattern like
```
%a = OpVariable %_ptr_CrossWorkgroup_uint CrossWorkgroup %uint_123
%11 = OpConstantComposite %_struct_6 %a %a
```
so that `spirv-val` complained with
```
error: line 25: OpConstantComposite Constituent <id> '10[%a]' is not a constant or undef.
  %11 = OpConstantComposite %_struct_6 %a %a
```
2024-03-25 10:14:46 +01:00
Vyacheslav Levytskyy
1d250d9099
[SPIR-V] Improve type inference in SPIR-V Backend for opaque pointers (#86283)
This PR improves type inference in SPIR-V Backend for opaque pointers,
accounting or a case when there is a chain of function calls that allows
to deduce formal parameter types from actual arguments. The attached
test demonstrates the case.
2024-03-25 10:14:08 +01:00
Vyacheslav Levytskyy
afec257d36
[SPIRV] Add type inference of function parameters by call instances (#85077)
This PR adds type inference of function parameters by call instances.
Two use cases that demonstrate the problem are added.
2024-03-14 10:50:11 +01:00
Vyacheslav Levytskyy
0a443f13b4
[SPIR-V] Add implementation of G_SPLAT_VECTOR opcode and fix invalid types processing (#84766)
This PR:
* adds support for G_SPLAT_VECTOR generic opcode that may be legally
generated instead of G_BUILD_VECTOR by previous passes of the translator
(see https://github.com/llvm/llvm-project/pull/80378 for the source of
breaking changes);
* improves deduction of types for opaque pointers.

This PR also fixes the following issues:
* if a function has ptr argument(s), two functions that have different
SPIR-V type definitions may get identical LLVM function types and break
agreements of global register and duplicate checker;
* checks for pointer types do not account for TypedPointerType.

Update of tests:
* A test case is added to cover the issue with function ptr parameters.
* The first case, that is support for G_SPLAT_VECTOR generic opcode, is
covered by existing test cases.
* Multiple additional checks by `spirv-val` is added to cover more
possibilities of generation of invalid code.
2024-03-13 08:32:01 +01:00
Vyacheslav Levytskyy
fb1be9b33c
[SPIR-V] Insert a bitcast before load/store instruction to keep SPIR-V code valid (#84069)
This PR introduces a step after instruction selection where instructions
can be traversed from the perspective of their validity from the
specification point of view. The PR adds also a way to correct
load/store when there is a type mismatch contradicting the specification
-- an additional bitcast is inserted to keep types consistent.
Correspondent test cases are added and existing test cases are
corrected.

This PR helps to successfully validate with the `spirv-val` tool
(https://github.com/KhronosGroup/SPIRV-Tools) some output that
previously led to validation errors and crashes of back translation from
SPIRV to LLVM IR from the side of SPIRV Translator project
(https://github.com/KhronosGroup/SPIRV-LLVM-Translator).

The added step of bringing instructions to required by the specification
type correspondence can be (should be and will be) extended beyond
load/store instructions to ensure validity rules of other SPIRV
instructions related to type inference.
2024-03-08 08:31:56 +01:00