Fix code quality issues reported by static analysis tool, such as:
- Rule of Three/Five.
- Dereference after null check.
- Unchecked return value.
- Variable copied when it could be moved.
In Vulkan, the signedness of the accesses to images has to match the
signedness of the backing image.
See
https://docs.vulkan.org/spec/latest/chapters/textures.html#textures-input,
where it says the behaviour is undefined if
> the signedness of any read or sample operation does not match the
signedness of the image’s format.
Users who define say an `RWBuffer<int>` will create a Vulkan image with
a signed integer format. So the HLSL that is generated must match that
expecation.
The solution we use is to generate a `spirv.SignedImage` target type for
signed integer instead of `spirv.Image`. The two types are otherwise the
same.
The backend will add the `signExtend` image operand to access to the
image to ensure the image is access as a signed image.
Fixes#144580
Adds additional subgroup block prefetch, load,
load transposed, load transformed and store
instructions to read two-dimensional blocks of
data from a two-dimensional region of memory, or
to write two-dimensional blocks of data to a
two-dimensional region of memory.
Spec:
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/INTEL/SPV_INTEL_2d_block_io.asciidoc
---------
Co-authored-by: Dmitry Sidorov <dmitry.sidorov@intel.com>
- Use a bool in `generateGetQueryInst` and rename the variable to better
convey its purpose.
- Replace mentions of 0 by `DefaultValue` in comments.
- Fix typos.
The __spirv_GenericCastToPtrExplicit_To* builtins and its equivalent
OpenCL builtins (to_global, to_local and to_private) were mapped to
OpGenericCastToPtr instead of OpGenericCastToPtrExplicit.
The patch now uses OpGenericCastToPtrExplicit for these builtins.
Adds support for the SPV_INTEL_ternary_bitwise_function extension,
adding;
* the OpBitwiseFunctionINTEL SPIR-V instruction, a ternary bitwise
function where the operation performed is determined by a look-up table
index,
* and the corresponding TernaryBitwiseFunctionINTEL capability.
See
https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_ternary_bitwise_function.html.
Signed-off-by: Larsen, Steffen <steffen.larsen@intel.com>
Adds code to expand the `llvm.spv.resource.handlefrombinding` and
`llvm.spv.resource.getpointer` when the resource type is
`spirv.VulkanBuffer`.
It gets expanded as a storage buffer or uniform buffer denpending on the
storage class used.
This is implementing part of
https://github.com/llvm/wg-hlsl/blob/main/proposals/0018-spirv-resource-representation.md.
- Use static instead of anonymous namespace for file local functions.
- Enclose file-local classes in anonymous namespace.
- Eliminate `llvm::` qualifier when file has `using namespace llvm`.
- Eliminate namespace surrounding entire code in
SPIRVConvergenceRegionAnalysis.cpp file.
- Eliminate call to `initializeSPIRVStructurizerPass` from the pass
constructor (https://github.com/llvm/llvm-project/issues/111767)
This PR is to thoroughly rework duplicate tracker implementation and
tracking of IR entities and types. These are legacy parts of the project
resulting in an extremely bloated intermediate representation and
computational delays due to inefficient data flow and structure choices.
Main results of the rework:
1) Improved compile-time performance. The reference binary LLVM IR used
to measure speed gains in
https://github.com/llvm/llvm-project/pull/120415 shows ~x5 speed up also
after this PR. The timing before this PR is ~42s and after this PR it's
~7.5s. In total this PR and the previous overhaul of the module analysis
in https://github.com/llvm/llvm-project/pull/120415 results in ~x25
speed improvement.
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj
real 0m7.545s
user 0m6.685s
sys 0m0.859s
```
2) Less bloated intermediate representation of internal translation
steps. Elimination of `spv_track_constant` intrinsic usage for scalar
constants, rework of `spv_assign_name`, removal of the gMIR `GET_XXX`
pseudo code and a smaller number of generated `ASSIGN_TYPE` pseudo codes
substantially decrease volume of data generated during translation.
3) Simpler code and easier maintenance. The duplicate tracker
implementation is simplified, as well as other features.
4) Numerous fixes of issues and logical flaws in different passes. The
main achievement is rework of the duplicate tracker itself that had
never guaranteed a correct caching of LLVM IR entities, rarely and
randomly returning stale/incorrect records (like, remove an instruction
from gMIR but still refer to it). Other fixes comprise consistent
generation of OpConstantNull, assigning types to newly created
registers, creation of integer/bool types, and other minor fixes.
5) Numerous fixes of LIT tests: mainly CHECK-DAG to properly reflect
SPIR-V spec guarantees, `{{$}}` at the end of constants to avoid
matching of substrings, and XFAILS for `SPV_INTEL_long_composites` test
cases, because the feature is not completed in full yet and doesn't
generate a requested by the extension sequence of instructions.
6) New test cases are added.
The SPIR-V Backend uses the same set of utility functions, mostly though
not entirely from SPIRVGlobalRegistry, to generate gMIR and SPIR-V
opcodes, depending on the current stage of translation. This is
controlled by an explicit EmitIR flag rather than the current
translation pass, and there are legacy pieces of code where the EmitIR
flag is declared so that it has a default true value, allowing using
utility functions without explicitly declaring their intent to work
either in gMIR or in SPIR-V part of the lowering process.
While it may be ok to leave this default EmitIR flag as is in generation
of scalar integer/float types, as we don't expect to see any dependent
opcodes derived from such OpTypeXXX instructions, using of EmitIR by
default in aggregation types is a source of hidden logical flaws and
actual issues.
This PR provides a partial fix to the problem by removing default status
of EmitIR, requiring a user call site to explicitly announce its intent
to generate gMIR or SPIR-V code, fixes several cases of misuse of
EmitIR, and, the most important, fixes a nasty logical error that breaks
passing of actually asked EmitIR value by the default value in the
middle of the chain of calls, in the `findSPIRVType` call. The latter
error was a source of issues in the post-instruction selection pass that
has been getting gMIR code where SPIR-V was explicitly requested due to
overloaded with default parameters internal API in SPIRVGlobalRegistry
(most notably, `findSPIRVType`).
This PR adds support for `cl_khr_extended_bit_ops` in SPIRV Backend.
Note that `cl_khr_extended_bit_ops` only supports types in
```
char, charn, uchar, ucharn, short, shortn, ushort, ushortn, int, intn, uint, uintn, long, longn, ulong, and ulongn
```
where `n is 2, 3, 4, 8, or 16`.
Subsequent PRs will introduce support for non-standard bit width
required by `SPV_KHR_bit_instructions`.
This continues the work on dot product instructions already started in
3cdac06.
This change adds support for all OpenCL integer dot product builtins
under `cl_khr_integer_dot_product` extension, namely:
```
* dot
* dot_acc_sat
* dot_4x8packed_(uu/ss/su/us)_(u)int
* dot_acc_sat_4x8packed_(uu/ss/su/us)_(u)int
```
Fixed a type mismatch issue in the comparison of std::memory_order with
integers.
This fixes an issue reported by clang-debian-cpp20 buildbot for
https://github.com/llvm/llvm-project/pull/123654
Adding SPIRV to LLVM_ALL_TARGETS
(https://github.com/llvm/llvm-project/pull/119653) revealed a series of
minor compilation problems and sanitizer complaints. This PR is to
address the problem.
This PR fixes https://github.com/llvm/llvm-project/issues/120078 and
improves/simplifies parsing of demangled function name that aims to
detect a tip for floating point decorations. The latter improvement
fixes also a complaint from `LLVM_USE_SANITIZER=Address`.
This PR adds the following features:
* saturation and float rounding mode decorations,
* arithmetic constrained floating-point intrinsics (strict_fadd,
strict_fsub, strict_fmul, strict_fdiv, strict_frem, strict_fma and
strict_fldexp),
* and SPV_INTEL_float_controls2 extension,
* using recent improvements of emit-intrinsics step, this PR also
simplifies pre- and post-legalizer steps and improves instruction
selection.
This PR improves general validity of emitted code between passes due to
generation of `TargetOpcode::PHI` instead of `SPIRV::OpPhi` after
Instruction Selection, fixing generation of OpTypePointer instructions
and using of proper virtual register classes.
Using `TargetOpcode::PHI` instead of `SPIRV::OpPhi` after Instruction
Selection has a benefit to support existing optimization passes
immediately, as an alternative path to disable those passes that use
`MI.isPHI()`. This PR makes it possible thus to revert
https://github.com/llvm/llvm-project/pull/116060 actions and get back to
use the `MachineSink` pass.
This PR is a solution of the problem discussed in details in
https://github.com/llvm/llvm-project/pull/110507. It accepts an advice
from code reviewers of the PR #110507 to postpone generation of OpPhi
rather than to patch CodeGen. This solution allows to unblock
improvements wrt. expensive checks and makes it unrelated to the general
points of the discussion about OpPhi vs. G_PHI/PHI.
This PR contains numerous small patches of emitted code validity that
allows to substantially pass rate with expensive checks. Namely, the
test suite with expensive checks set ON now has only 12 fails out of 569
total test cases.
FYI @bogner
This PR is to fix the following issues:
* the SPIR-V Backend didn't generate Alignment decoration for alloca
instructions,
* we need to use types from demangled function declarations to specify
types for opaque pointers.
The spec is available here:
https://github.com/intel/llvm/pull/12497
The PR doesn't add OpCooperativeMatrixApplyFunctionINTEL instruction as
it's still experimental and not properly tested E2E.
The PR also fixes few bugs in the related code:
1. CooperativeMatrixMulAddKHR optional operand must be literal, not a
constant;
2. Fixed available capabilities table creation for a case, when a single
extension adds few capabilities, that occupy not contiguous op codes.
---------
Signed-off-by: Sidorov, Dmitry <dmitry.sidorov@intel.com>
Goals of the PR are:
* to ensure that correct types are applied to virtual registers which
were used as return values in call lowering. A reproducer is attached as
a new test case, before the PR it fails because spirv-val considers
output invalid due to wrong result/operand types in OpPhi's;
* improve type inference by speeding up postprocessing of types: by
limiting iterations by checking what remains to process, and processing
each instruction just once for any number of operands with uncomplete
types;
* improve type inference by more accurate work with uncomplete types
(pass uncomplete property to dependent operands, ensure consistency of
uncomplete-types data structure);
* change processing order and add traversing of PHI nodes when type
inference apply instructions results to specify/update/cast operands
type (fixes an issue with OpPhi's result type mismatch with operand
types).
This PR is to solve several intertwined issues with type inference while
adding support for builtins for OpIAddCarry and OpISubBorrow:
* OpIAddCarry and OpISubBorrow generation in a way of supporting SPIR-V
friendly builtins `__spirv_...` -- introduces a new element to account
for, namely, `ptr sret (%struct) %0` argument that is a place to put a
result of the instruction;
* fix early definition of SPIR-V types during call lowering -- namely,
the goal of the PR is to ensure that correct types are applied to
virtual registers which were used as arguments in call lowering and so
caused early definition of SPIR-V types; reproducers are attached as a
new test cases;
* improve parsing of builtin names (e.g., understand a name of a kind
`"anon<int, int> __spirv_IAddCarry<int, int>(int, int)"` that was
incorrectly parsed as `anon` before the PR);
* improve type inference and fix access to erased from parent after
visit instructions -- before the PR visiting of instructions in
emitintrinsics pass replaced old alloca's, bitcast's, etc. instructions
with a newly generated internal SPIR-V intrinsics and after erasing old
instructions there were still references to them in a postprocessing
working list, while records for newly deduced pointee types were lost;
this PR fixes the issue by adding as consistent wrt. internal data
structures action `SPIRVEmitIntrinsics::replaceAllUsesWith()` that fixes
above mentioned problems;
* LLVM IR add/sub instructions result in logical SPIR-V instructions
when applied to bool type;
* fix validation of pointer types for frexp and lgamma_r,
* fix hardcoded reference to AS0 as a Function storage class in
lib/Target/SPIRV/SPIRVBuiltins.cpp -- now it's
`storageClassToAddressSpace(SPIRV::StorageClass::Function)`,
* re-use the same OpTypeStruct for two identical references to struct's
in arithmetic with overflow instructions.
The SPIRV backend has a special type named `spirv.Image`. This type is
meant to correspond to the OpTypeImage instruction in SPIR-V, but there
is one difference. The access qualifier operand in OpTypeImage is
optional. On top of that, the access qualifiers are only valid for
kernels, and not for shaders.
We want to reuse this type when generating shader from HLSL, but we
can't use the access qualifier. This commit make the access qualifer
optional in the target extension type.
The same is done for `spirv.SampledImage`.
Contributes to #81036
This PR continues https://github.com/llvm/llvm-project/pull/101732
changes in virtual register processing aimed to improve correctness of
emitted MIR between passes from the perspective of MachineVerifier.
Namely, the following changes are introduced:
* register classes (lib/Target/SPIRV/SPIRVRegisterInfo.td) and
instruction patterns (lib/Target/SPIRV/SPIRVInstrInfo.td) are corrected
and simplified (by removing unnecessary sophisticated options) -- e.g.,
this PR gets rid of duplicating 32/64 bits patterns, removes ANYID
register class and simplifies definition of the rest of register
classes,
* hardcoded LLT scalar types in passes before instruction selection are
corrected -- the goal is to have correct bit width before instruction
selection, and use 64 bits registers for pattern matching in the
instruction selection pass; 32-bit registers remain where they are
described in such terms by SPIR-V specification (like, for example,
creation of virtual registers for scope/mem semantics operands),
* rework virtual register type/class assignment for calls/builtins
lowering,
* a series of minor changes to fix validity of emitted code between
passes:
- ensure that that bitcast changes the type,
- fix the pattern for instruction selection for OpExtInst,
- simplify inline asm operands usage,
- account for arbitrary integer sizes / update legalizer rules;
* add '-verify-machineinstrs' to existed test cases.
See also https://github.com/llvm/llvm-project/issues/88129 that this PR
may resolve.
This PR fixes a great number of issues reported by MachineVerifier and,
as a result, reduces a number of failed test cases for the mode with
expensive checks set on from ~200 to ~57.
This PR addresses a TODO in
lib/Target/SPIRV/SPIRVInstructionSelector.cpp by adding implementation
of the non-const G_BUILD_VECTOR, and fix emission of the
OpGroupBroadcast instruction for the case when the `..._group_broadcast`
builtin has more than one `local_id` argument and `OpGroupBroadcast`
requires a newly constructed vector with 2 or 3 components instead of
originally passed series of `local_id` arguments.
This PR may resolve https://github.com/llvm/llvm-project/issues/97310 if
the reason for the reported fail is an incorrectly generated
OpGroupBroadcast instruction that was definitely a case.
Existing test is hardened and a new test is added to cover this special
case of the OpGroupBroadcast instruction emission.
This PR contains changes in virtual register processing aimed to improve
correctness of emitted MIR between passes from the perspective of
MachineVerifier. This potentially helps to detect previously missed
flaws in code emission and harden the test suite. As a measure of
correctness and usefulness of this PR we may use a mode with expensive
checks set on, and MachineVerifier reports problems in the test suite.
In order to satisfy Machine Verifier requirements to MIR correctness not
only a rework of usage of virtual registers' types and classes is
required, but also corrections into pre-legalizer and instruction
selection logics. Namely, the following changes are introduced:
* scalar virtual registers have proper bit width,
* detect register class by SPIR-V type,
* add a superclass for id virtual register classes,
* fix Tablegen rules used for instruction selection,
* fixes of minor existed issues (missed flag for proper representation
of a null constant for OpenCL vs. HLSL, wrong usage of integer virtual
registers as a synonym of any non-type virtual register).
This PR improves type inference of operand presented by opaque pointers
and aggregate types:
* tries to restore original function return type for aggregate types so
that it's possible to deduce a correct type during emit-intrinsics step
(see llvm/test/CodeGen/SPIRV/SpecConstants/restore-spec-type.ll for the
reproducer of the previously existed issue when spirv-val found a
mismatch between object and ptr types in OpStore due to the incorrect
aggregate types tracing),
* explores untyped pointer operands of store to deduce correct pointee
types,
* creates an extension type to track pointee types from emit-intrinsics
step and further instead of direct and naive usage of TypePointerType
that led previously to crashes due to ban of creation of Value of
TypePointerType type,
* tracks instructions with uncomplete type information and tries to
improve their type info after pass calculated types for all machine
functions (it doesn't traverse a code but rather checks only those
instructions which were tracked as uncompleted),
* address more cases of removing unnecessary bitcasts (see, for example,
changes in test/CodeGen/SPIRV/transcoding/OpGenericCastToPtr.ll where
`CHECK-SPIRV-NEXT` in LIT checks show absence of unneeded bitcasts and
unmangled/mangled versions have proper typing now with equivalent type
info),
* address more cases of well known types or relations between types
within instructions (see, for example, atomic*.ll test cases and
Event-related test cases for improved SPIR-V code generated by the
Backend),
* fix the issue of removing unneeded ptrcast instructions in
pre-legalizer pass that led to creation of new assign-type instructions
with the same argument as source in ptrcast and caused errors in type
inference (the reproducer `complex.ll` test case is added to the PR).
This PR:
* adds missing __spirv_ wrappers to Non-Uniform, Atomic, and Convert
Instructions,
* fixes emission of Group builtins,
* adds relevant checks to test cases to cover newly added __spirv_
wrappers.
This PR improves type inference for a known instruction's builtin:
OpGroupAsyncCopy:
* deduce a type of one source/destination pointer when it's possible to
deduce a type of another argument, and
* validate src and dest types and tries to unfold a parameter if it's a
structure wrapper around a scalar/vector type.
This PR:
* supports cl_ext_float_atomics by mapping atomic_fetch_add and
atomic_fetch_sub applied to float arguments to the corresponding
instructions from SPV_EXT_shader_atomic_float*_add, and
* fix errors in definition of atomic_fetch_*_explicit builtins by fixing
a valid number of arguments.
This PR fixes https://github.com/llvm/llvm-project/issues/96513.
The way of creation of array type constant was incorrect: instead of
creating [1, 1, 1] or [1, 1, 1, 1, 1, ....] constants, the same [1]
constant was always created, substituting original composite constants.
This in its turn led to a situation when only one of constants might
exist in the code without emitting invalid code, the second constant
would be eventually rewritten to the first constant, because a key to
address both was an array of a single element (like [1]).
This PR fixes the issue and purges from the code unneeded copy/pasted
clone of the function that creates an array constant.
This PR adds builtin functions to insert instructions from 'Group and
Subgroup' section of the SPIR-V Specification. Corresponding tests are
updated, `spirv-val` run is added where it was missed.
This PR implements insertion of OpGenericCastToPtr using builtin
functions (both opencl `to_global|local|private` and `__spirv_`
wrappers), and improves type inference.
This PR is to add validation to the test case with
get_image_array_size/get_image_dim calls
(transcoding/check_ro_qualifier.ll). This test case didn't pass
validation because of invalid emission of OpCompositeExtract instruction
(Result Type must be the same type as Composite.).
In order to fix the problem this PR improves type inference in general
and partially addresses issues:
* https://github.com/llvm/llvm-project/issues/91998
* https://github.com/llvm/llvm-project/issues/91997
A reproducer from the description of the latter issue is added as a new
test case as a part of this PR.
