This PR introduces changes into processing of internal/service data in
SPIRV Backend so that Duplicates Tracker accounts for possible changes
in Constant usage after optimization, namely this PR fixes the case when
a Constant register stored in Duplicates Tracker after all passes is
represented by a non-constant expression. In this case we may be sure
that it neither is able to create a duplicate nor is in need of a
special placement as a Constant instruction.
This PR doesn't introduce a new feature, and in this case we rely on
existing set of test cases in the SPIRV Backend test suite to ensure
that this PR doesn't break existing assumptions without introducing new
test cases. There is a reproducer of the issue available as part of SYCL
CTS test suite, however it's a binary of several MB's size. Given the
subtlety of the issue, reduction of the reproducer to a reasonable site
for inclusion into the SPIRV Backend test suite doesn't seem realistic.
The SPIRV backend has a special type named `spirv.Image`. This type is
meant to correspond to the OpTypeImage instruction in SPIR-V, but there
is one difference. The access qualifier operand in OpTypeImage is
optional. On top of that, the access qualifiers are only valid for
kernels, and not for shaders.
We want to reuse this type when generating shader from HLSL, but we
can't use the access qualifier. This commit make the access qualifer
optional in the target extension type.
The same is done for `spirv.SampledImage`.
Contributes to #81036
The commit introduces support for fundamental DI instruction. Metadata
handlers required for this instruction is stored inside debug records
(https://llvm.org/docs/SourceLevelDebugging.html) parts of the module
which rises the necessity of it's traversal.
This PR is to ensure that OpExtInst instructions generated by
NonSemantic_Shader_DebugInfo_100 are not mixed up with other OpExtInst
instructions.
Original implementation
(https://github.com/llvm/llvm-project/pull/97558) has introduced an
issue by moving OpExtInst instruction with the 3rd operand equal to
DebugSource (value 35) or DebugCompilationUnit (value 1) even if
OpExtInst is not generated by NonSemantic_Shader_DebugInfo_100
implementation code.
The reproducer is attached as a new test case. The code of the test case
reproduces the issue, because "lgamma" has the same code (35) inside
OpenCL_std as DebugSource inside NonSemantic_Shader_DebugInfo_100.
This commit introduces emission of DebugSource, DebugCompileUnit from
NonSemantic.Shader.DebugInfo.100 and required OpString with filename.
NonSemantic.Shader.DebugInfo.100 is divided, following DWARF into two
main concepts – emitting DIE and Line.
In DWARF .debug_abbriev and .debug_info sections are responsible for
emitting tree with information (DEIs) about e.g. types, compilation
unit. Corresponding to that in NonSemantic.Shader.DebugInfo.100 have
instructions like DebugSource, DebugCompileUnit etc. which preforms same
role in SPIR-V file. The difference is in fact that in SPIR-V there are
no sections but logical layout which forces order of the instruction
emission.
The NonSemantic.Shader.DebugInfo.100 requires for this type of global
information to be emitted after OpTypeXXX and OpConstantXXX
instructions.
One of the goals was to minimize changes and interaction with
SPIRVModuleAnalysis as possible which current commit achieves by
emitting it’s instructions directly into MachineFunction.
The possibility of duplicates are mitigated by guard inside pass which
emits the global information only once in one function.
By that method duplicates don’t have chance to be emitted.
From that point, adding new debug global instructions should be
straightforward.
This PR continues https://github.com/llvm/llvm-project/pull/94467 and
contains fixes in emission of type intrinsics, constant recording and
corresponding test cases:
* type-deduce-global-dup.ll -- fix of integer constant emission on
32-bit platforms and correct type deduction for globals
* type-deduce-simple-for.ll -- fix of GEP translation (there was an
issue previously that led to incorrect translation/broken logic of
for-range implementation)
This PR also:
* fixes a cast between identical storage classes and updates the test
case to include validation run by spirv-val,
* ensures that Bitcast for pointers satisfies the requirement that the
address spaces must match and adds the corresponding test case,
* improve encode in Tablegen and decode in code of dependencies between
SPIR-V entities and required capability/extensions,
* prevent emission of identical OpTypePointer instructions.
This PR introduces support of llvm.ptr.annotation to SPIR-V Backend, and
implement several extensions which make use of spirv.Decorations and
llvm.ptr.annotation to annotate global variables and pointers:
- SPV_INTEL_cache_controls
- SPV_INTEL_global_variable_host_access
- SPV_INTEL_global_variable_fpga_decorations
This PR introduces support for inline assembly calls for SPIR-V Backend
in general, and support for SPV_INTEL_inline_assembly [1] extension in
particular. The former part of the PR is agnostic towards
vendor-specific requirements and resolves the task of supporting
successful transformation of inline assembly as long as it's possible
without specific SPIR-V instruction codes.
As a part of the PR there appears an opportunity to bring coherent
inline assembly information up to latest passes of the transformation
process (emitting final SPIR-V instructions), so that PR makes it easy
to add any another required flavor of inline assembly, other then
supported by the vendor specific SPV_INTEL_inline_assembly extension,
if/when needed.
At the moment, however, SPV_INTEL_inline_assembly is the only
implemented way to bring LLVM IR inline assembly calls up to valid
SPIR-V instructions and also the default one. This means that inline
assembly calls will generate an error message of such extension is not
used to prevent LLVM-generated error messages at the final stages of
translation. When the SPV_INTEL_inline_assembly extension is mentioned
among supported, translation of inline assembly is intercepted by this
extension implementation on a pre-legalizer step, and this is a place
where support for a new inline assembly extension may be added if
needed.
This PR also extends support for register classes, improves type
inference during pre-legalizer pass, and fixes a minor bug with
asm-printing of string literals.
[1]
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_inline_assembly.asciidoc
Recognize `cl_khr_kernel_clock` builtins and translate them to
`OpReadClockKHR` instructions. The `Scope` operand is deduced from the
builtin function name.
spirv-val does not pass yet due to OpReadClockKHR only supporting the
valid scopes for Vulkan (Device and Subgroup, but not Workgroup), so
leave validation disabled with a TODO.
This patch:
- Adds SPIR-V backend's registered generator magic number to the emitted
binary. The magic number consists of the generator ID (43) and LLVM
major version.
- Adds SPIR-V version to the binary.
- Allows reading the expected (maximum supported) SPIR-V version from
the target triple.
- Uses VersionTuple for representing versions throughout the backend's
codebase.
- Registers v1.6 for spirv32 and spirv64 triple.
See more: https://github.com/KhronosGroup/SPIRV-Headers/commit/7d500c
If there is no information about OpenCL version we are forced to
generate OpenCL 1.0 by default for the OpenCL environment to avoid
puzzling run-times with Unknown/0.0 version output. For a reference,
LLVM-SPIRV Translator avoids potential issues with run-times in a
similar manner.
Add support to generate valid SPIR-V for the WaveGetLaneIndex() HLSL
builtin.
To implement this, I had to fix a few small issues in the backend, like
the i8* pointer type being emitted, even if we have the type information
elsewhere.
Signed-off-by: Nathan Gauër <brioche@google.com>
This PR:
* adds Lifetime intrinsics/instructions
* fixes how the binary header is emitted (correct version and better
approximation of Bound)
* add validation into more test cases
This PR is to add explicit support for SPV_KHR_float_controls
(https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_KHR_float_controls.asciidoc).
This extension is included into SPIR-V after version 1.4, but in case of
lower versions it is to be included explicitly and OpExtension must be
present in the module with `OpExtension "SPV_KHR_float_controls"`.
This PR fixes this issue and fixes the test case
test/CodeGen/SPIRV/exec_mode_float_control_khr.ll to account for a
version lower than 1.4.
This PR adds SPIR-V extension SPV_INTEL_variable_length_array that
allows to allocate local arrays whose number of elements is unknown at
compile time:
* add a new SPIR-V internal intrinsic:int_spv_alloca_array
* legalize G_STACKSAVE and G_STACKRESTORE
* implement allocation of arrays (previously getArraySize() of
AllocaInst was not used)
* add tests
This PR is to add support for the SPIR-V extension
SPV_KHR_uniform_group_instructions that adds new instructions to SPIR-V
to support additional group operations within uniform control flow.
This PR adds support for atomic instruction on floating-point numbers:
* SPV_EXT_shader_atomic_float_add
* SPV_EXT_shader_atomic_float_min_max
* SPV_EXT_shader_atomic_float16_add
and fixes asm printer output for half floating-type.
This PR adds support for the SPV_KHR_linkonce_odr extension and modifies
existing negative test with a positive check for the extension and
proper linkage type in case when the extension is enabled.
SPV_KHR_linkonce_odr adds a "LinkOnceODR" linkage type, allowing proper
translation of, for example, C++ templates classes merging during
linking from different modules and supporting any other cases when a
global variable/function must be merged with equivalent global
variable(s)/function(s) from other modules during the linking process.
By SPIR-V specification: "If an instruction, enumerant, or other feature
specifies multiple enabling capabilities, only one such
capability needs to be declared to use the feature."
However, one capability may be preferred over another. One important
case is Shader capability that may not be supported by a backend, but
always is inserted if "OpDecorate SpecId" is found, because Enabling
Capabilities for the latter is the list of Shader and Kernel, where
Shader is coming first and thus always selected as the first available
option.
In this PR we address the problem by keeping current behaviour of
selecting the first option among enabling capabilities as is, but giving
a user a way to filter capabilities during the selection process via a
newly introduced "--avoid-spirv-capabilities" command line option. This
option is to avoid selection of certain capabilities if there are other
available enabling capabilities.
This PR is changing also existing pruneCapabilities() function. It
doesn't remove capability from module requirement anymore, but only adds
implicitly required capabilities recursively, so its name is changed
accordingly. This change fixes the present bug in collecting required by
a module capabilities. Before the change, introduced by this PR,
pruneCapabilities() function has been removing, for example, Kernel
capability from required by a module, because Kernel is initially
required and the second time it was needed pruneCapabilities() removed
it by mistake.
The goal of this PR is to fix an issue when Module Analysis stage is not
able to complete processing of a really big LLVM source:
https://github.com/llvm/llvm-project/issues/76048.
There is an example of a bulky LLVM source:
https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/main/test/SpecConstants/long-spec-const-composite.ll
Processing of this file with
`llc -mtriple=spirv64-unknown-unknown -O0 long-spec-const-composite.ll
-o long-spec-const-composite.spvt`
to produce SPIR-V output using LLVM SPIR-V backend takes too long, and
I've never been able to see it actually completes. After the patch from
this PR applied elapsed time for me is ~30 sec.
The fix changes underlying data structure to be `std::set` to trace
instructions with identical operands instead of the existing approach of
the `findSameInstrInMS()` function.
Add Float16 to Vulkan's available capabilities, and guard Float16Buffer
(Kernel-only capability) against being added outside OpenCL
environments.
Add tests to verify half and half vector types, and validate with
spirv-val.
Fixes#66398
We were removing bit_instructions cap from All caps but this was a
mistake.
Test SPV_KHR_bit_instructions was failing. Remove function
removeCapabilityIf. It was not being done correctly and is now
unnecessary.
Adds new extension SPV_KHR_expect_assume, new capability
ExpectAssumeKHR as well as the new instructions:
* OpExpectKHR
* OpAssumeTrueKHR
These are lowered from respectively llvm.expect.<ty> and llvm.assume
intrinsics.
Previously https://reviews.llvm.org/D157696
Add Int16, Int64 and Float64 capabilities as always available for Vulkan
(since 1.0), and add tests covering most of the basic types from
clang/test/CodeGenHLSL/basic_types.hlsl except for half floats.
Depends on D156049
This commits adds the minimal required bits to build a logical SPIR-V
compute shader using LLC.
- Skip OpenCL-only capabilities & extensions for Logical SPIR-V.
- Generate required metadata for entrypoints from HLSL frontend.
- Fix execution mode to GLCompute in logical.
The main issue is the lack of "vulkan" bit in the triple.
This might need to be added as a vendor?
Because as-is, SPIRV32/64 assumes OpenCL, and then, SPIRV assumes
Vulkan. This is ok-ish today, but not correct.
Differential Revision: https://reviews.llvm.org/D156424
Adds support for SPV_INTEL_optnone.
Currently still in draft form but I wanted to open this revision
to ask some questions.
Differential Revision: https://reviews.llvm.org/D156297
IRTranslator lowers switches to [G_SUB] + G_ICMP + G_BRCOND + G_BR
sequences. Since values and destination MBBs are included in the
spv_switch intrinsics, the sequences are not needed for ISel.
Before this commit, the information decoded by these sequences were
added to spv_switch intrinsics in SPIRVPreLegalizer and the sequences
were kept until SPIRVModuleAnalysis where they were marked skipped for
emission.
After this commit, the [G_SUB] + G_ICMP + G_BRCOND + G_BR sequences
and MBBs containing only these MIs are erased in SPIRVPreLegalizer.
Differential Revision: https://reviews.llvm.org/D146923
SPIRVModuleAnalysis collects module and external function registers
(usually result of OpFunction) for use when emitting OpFunctionCall.
This patch makes the mapping between the functions and registers using
pointers (instead of name strings) to ensure anonymous functions and
calls can be resolved properly.
Differential Revision: https://reviews.llvm.org/D140548
The patch fixes lowering of anonymous functions, removes file/linkage
info for builtin call demangling, and adds relevant test demonstrating
a fixed problem.
Differential Revision: https://reviews.llvm.org/D135390
The patch adds the regularization pass that prepare LLVM IR for
the IR translation. It also contains following changes:
- reduce indentation, make getNonParametrizedType, getSamplerType,
getPipeType, getImageType, getSampledImageType static in SPIRVBuiltins,
- rename mayBeOclOrSpirvBuiltin to getOclOrSpirvBuiltinDemangledName,
- move isOpenCLBuiltinType, isSPIRVBuiltinType, isSpecialType from
SPIRVGlobalRegistry.cpp to SPIRVUtils.cpp, renaming isSpecialType to
isSpecialOpaqueType,
- implment getTgtMemIntrinsic() in SPIRVISelLowering,
- add hasSideEffects = 0 in Pseudo (SPIRVInstrFormats.td),
- add legalization rule for G_MEMSET, correct G_BRCOND rule,
- add capability processing for OpBuildNDRange in SPIRVModuleAnalysis,
- don't correct types of registers holding constants and used in
G_ADDRSPACE_CAST (SPIRVPreLegalizer.cpp),
- lower memset/bswap intrinsics to functions in SPIRVPrepareFunctions,
- change TargetLoweringObjectFileELF to SPIRVTargetObjectFile
in SPIRVTargetMachine.cpp,
- correct comments.
5 LIT tests are added to show the improvement.
Differential Revision: https://reviews.llvm.org/D133253
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
The patch adds support for OpenCL and SPIR-V built-in functions.
Their detection and properties are implemented using TableGen.
Five tests are added to demonstrate the improvement.
Differential Revision: https://reviews.llvm.org/D132024
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
This patch supports SPIR-V capabilities and extensions. In addition,
it inserts decorations related to MIFlags and improves support of switches.
Five tests are included to demonstrate the improvement.
Differential Revision: https://reviews.llvm.org/D131221
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
The patch replaces SPIRVBaseInfo.* previously created using macros by
the tablegen approach. There are many small changes in other files due to
differences in namespaces. Also, functions in SPIRVUtils are moved to
the llvm namespace.
Differential Revision: https://reviews.llvm.org/D130518
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
The patch adds SPIRVPrepareFunctions pass, which modifies function
signatures containing aggregate arguments and/or return values before
IR translation. Information about the original signatures is stored in
metadata. It is used during call lowering to restore correct SPIR-V types
of function arguments and return values. This pass also substitutes some
llvm intrinsic calls to function calls, generating the necessary functions
in the module, as the SPIRV translator does.
The patch also includes changes in other modules, fixing errors and
enabling many SPIR-V features that were omitted earlier. And 15 LIT tests
are also added to demonstrate the new functionality.
Differential Revision: https://reviews.llvm.org/D129730
Co-authored-by: Aleksandr Bezzubikov <zuban32s@gmail.com>
Co-authored-by: Michal Paszkowski <michal.paszkowski@outlook.com>
Co-authored-by: Andrey Tretyakov <andrey1.tretyakov@intel.com>
Co-authored-by: Konrad Trifunovic <konrad.trifunovic@intel.com>
SPIR-V module typically contains some global entities that were not
global before made it to SPIR-V, e.g. types and constants are not usually
declared globally in LLVM. By design SPIR-V requires such stuff to be declared
once and in the module's global section. Since MIR is not able to represent
such things properly they were generated per-function, and then at the very end
of the backend's pipeline hoisted into some 'meta' function minding possible
duplicates.
New SPIRVDuplicatesTracker keeps mapping of the original LLVM entities such
as types, constant, global variables, etc to their MIR counterparts -
(MachineFunction, Register). Later SPIRVModuleAnalysis (apart from other
thing it's responsible for) performs topological sorting of the
tracker's entries to ensure proper ordering before the hoisting,
and actually performs the hoisting in a duplicates-free manner
by the tracker's nature.
Differential Revision: https://reviews.llvm.org/D128471
