[This
case](bc35715696/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (L2815-L2818))
is not covered by any existing test (checked via code coverage and
inserting an `abort`). New test proposed that covers this line, as
demonstrated by test failure when an `abort` is present in that line.
[This
body](73768f88fb/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (L857-L860))
is not covered by any existing test (checked via code coverage and
inserting an abort). New test proposed that covers this line, as
demonstrated by test failure when an abort is present in that line.
74e4694 moved ObjCARCContract from running before the codegen pipeline
into addISelPrepare(), which runs after PreISelIntrinsicLowering.
This broke ObjCARCContract's retainRV-to-claimRV optimization because
ObjCARCContract identifies ARC calls via intrinsics, not their lowered
counterparts.
This patch restores the pre-74e4694 ordering by moving ObjCARCContract
to addISelPasses.
The IntrinsicInst.cpp change looks extraneous but is required here:
ObjCARCContract may now rewrite the bundle operand from retainRV to
claimRV. When PreISelIntrinsicLowering then encounters this new
intrinsic use, lowerObjCCall asserts mayLowerToFunctionCall.
Assisted-by: claude
rdar://137997453
- Add CPP_for_OpenCL source language operand
- Handle opencl.cxx.version metadata
Align handling with SPIR-V translator logic and tests presented there
- The SPIR-V spec requires that OpConstantComposite must not reference
spec constant operands. When a composite contains non-constant
constitued, OpSpecConstantComposite should be emitted instead of
OpConstantComposite
- Avoid creating function pointer types when the
SPV_INTEL_function_pointers extension is unavailable, falling back to i8
as the pointee type
- Re-enable spirv-val validation in tests that previously failed due to
this issue, and un-XFAIL the block_w_struct_return and global_block
transcoding tests
related to #60133fixes#186756
The OpBitReverse is available when Shader or SPV_KHR_bit_instructions
extension is enabled. For targets without these capabilities, introduce
software emulation of G_BITREVERSE based on the parallel bit reversal
algorithm:
https://graphics.stanford.edu/~seander/bithacks.html#ReverseParallel
The emulation supports 8/16/32/64-bit scalars and vectors using bitwise
operations (shifts, AND, OR). A helper lambda avoids undefined behavior
when computing masks for 64-bit types.
Tests added for both emulation and native paths across all supported
types.
Assisted-by: Claude Code
Adds the `GroupMemoryBarrier()` HLSL function to SPIRV and DirectX with
additional tests for the different backends.
When this moves in, will create another PR with this as a template for
the other Barriers:
- `AllMemoryBarrier()` #99076
- `AllMemoryBarrierWithGroupSync()` #99090
- `DeviceMemoryBarrier()` #99105
- `DeviceMemoryBarrierWithGroupSync()` #99106
`Barrier()` does not have support for SPIRV, so I will exclude that from
the next PR.
- [x] Implement GroupMemoryBarrier clang builtin,
- [x] Link GroupMemoryBarrier clang builtin with hlsl_intrinsics.h
- [x] Add sema checks for GroupMemoryBarrier to
CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
- [x] Add codegen for GroupMemoryBarrier to EmitHLSLBuiltinExpr in
CGBuiltin.cpp
- [x] Add codegen tests to
clang/test/CodeGenHLSL/builtins/GroupMemoryBarrier.hlsl
- [x] Add sema tests to
clang/test/SemaHLSL/BuiltIns/GroupMemoryBarrier-errors.hlsl
- [x] Create the int_dx_GroupMemoryBarrier intrinsic in
IntrinsicsDirectX.td
- [x] Create the DXILOpMapping of int_dx_GroupMemoryBarrier to 80 in
DXIL.td
- [x] Create the GroupMemoryBarrier.ll and GroupMemoryBarrier_errors.ll
tests in llvm/test/CodeGen/DirectX/
- [x] Create the int_spv_GroupMemoryBarrier intrinsic in
IntrinsicsSPIRV.td
- [x] In SPIRVInstructionSelector.cpp create the GroupMemoryBarrier
lowering and map it to int_spv_GroupMemoryBarrier in
SPIRVInstructionSelector::selectIntrinsic.
- [x] Create SPIR-V backend test case in
llvm/test/CodeGen/SPIRV/hlsl-intrinsics/GroupMemoryBarrier.ll
<!-- branch-stack-start -->
<!-- branch-stack-end -->
This patch fixes the logic when lowering firsbithigh and low when
targeting spirv. It adds an aditional check to make sure the second half
is also -1, if it is it returns -1 otherwise returns the resulting sum.
Fix: #143003 and #143171
---------
Co-authored-by: Finn Plummer <mail@inbelic.dev>
This PR adds QuadReadAcrossY intrinsic support in HLSL with codegen for
both DirectX and SPIRV backends. Resolves
https://github.com/llvm/llvm-project/issues/99176.
- [x] Implement `QuadReadAcrossY` clang builtin,
- [x] Link `QuadReadAcrossY` clang builtin with `hlsl_intrinsics.h`
- [x] Add sema checks for `QuadReadAcrossY` to
`CheckHLSLBuiltinFunctionCall` in `SemaChecking.cpp`
- [x] Add codegen for `QuadReadAcrossY` to `EmitHLSLBuiltinExpr` in
`CGBuiltin.cpp`
- [x] Add codegen tests to
`clang/test/CodeGenHLSL/builtins/QuadReadAcrossY.hlsl`
- [x] Add sema tests to
`clang/test/SemaHLSL/BuiltIns/QuadReadAcrossY-errors.hlsl`
- [x] Create the `int_dx_QuadReadAcrossY` intrinsic in
`IntrinsicsDirectX.td`
- [x] Create the `DXILOpMapping` of `int_dx_QuadReadAcrossY` to `123` in
`DXIL.td`
- [x] Create the `QuadReadAcrossY.ll` and `QuadReadAcrossY_errors.ll`
tests in `llvm/test/CodeGen/DirectX/`
- [x] Create the `int_spv_QuadReadAcrossY` intrinsic in
`IntrinsicsSPIRV.td`
- [x] In SPIRVInstructionSelector.cpp create the `QuadReadAcrossY`
lowering and map it to `int_spv_QuadReadAcrossY` in
`SPIRVInstructionSelector::selectIntrinsic`.
- [x] Create SPIR-V backend test case in
`llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadAcrossY.ll`
This adds two related changes to HLSL debug info support in the SPIR-V
backend. It's a first small step towards the plan I described in
https://discourse.llvm.org/t/hlsl-spirv-nsdi-debug-info-support-for-clang-dxc/90149.
## Tag HLSL shaders with `DW_LANG_HLSL` in the front-end
`GetSourceLanguage()` in `clang/lib/CodeGen/CGDebugInfo.cpp` checked
`LO.CPlusPlus` before `LO.HLSL`. Since HLSL is compiled as C++, the HLSL
check was never reached. Shaders compiled with `-g` were tagged with
`DW_LANG_C_plus_plus_14` instead of `DW_LANG_HLSL`. The NSDI pass
already had the correct mapping for `DW_LANG_HLSL` but it was never
triggered.
This fixes#136929 and #136995.
## Make `SPIRVEmitNonSemanticDI` activate automatically when `-g` is
used
`SPIRVPassConfig::addPreEmitPass()` only scheduled
`SPIRVEmitNonSemanticDI` when `--spv-emit-nonsemantic-debug-info` was
set or the target vendor was AMD. Passing `-g` to clang had no effect on
the SPIR-V backend pass.
The pass is now added unconditionally and self-activates by checking for
`llvm.dbg.cu` in the module. When no debug metadata is present it exits
early with no effect. This avoids the need to inspect module metadata at
pass-configuration time, which is not reliably available.
`--spv-emit-nonsemantic-debug-info` is now a deprecated synonym for
`-g`.
The alternative to the unconditional pass approach is to check at
pass-configuration time whether the module was compiled with debug info
(e.g. via `TargetOptions::DebugInfoForProfiling` or a similar flag
forwarded from the driver). I went with the unconditional approach
because it is simpler and the pass is cheap to enter and exit when no
`llvm.dbg.cu` is present.
I'm not sure whether adding a pass unconditionally is acceptable. Does
this sound reasonable, or would it be better to implement the
flag-forwarding approach?
Changes to tests:
- `clang/test/CodeGenHLSL/` (new): verifies that `-g` on an HLSL SPIR-V
target produces `DebugCompilationUnit` with language code 5
(`DW_LANG_HLSL`).
-
`llvm/test/CodeGen/SPIRV/debug-info/hlsl-debug-info-auto-activation.ll`
(new): verifies that a module with `llvm.dbg.cu` and `DW_LANG_HLSL`
produces `DebugCompilationUnit` without
`--spv-emit-nonsemantic-debug-info`.
- Existing `debug-compilation-unit.ll`, `debug-type-basic.ll`,
`debug-type-pointer.ll`: updated to verify NSDI is emitted whenever
debug metadata is present.
- `llc-pipeline.ll`: updated to reflect that `SPIRVEmitNonSemanticDI` is
now always in the pipeline.
---------
Co-authored-by: Eric Christopher <echristo@gmail.com>
Registered SPIRVEmitIntrinsics with the new pass manager as
`spirv-emit-intrinsics`. The motivation for this is to allow it to be run from
`opt` as a standalone pass for testing purposes. A simple pipeline test
is also included to check that the registration works.
This PR adds a new SPIRV pass that generates a kernel named
"spirv$device$init" that iterates the pointers in the table pointed by
__init_array_start and __init_array_end and executes them. It also
generates symbols for each constructor with the form
__init_array_object_NAME_PRIORITY.
These symbols will be used by the Level Zero plugin in the liboffload
runtime (with the support introduced by #187510) to generate the
aforementioned table as spirv-link cannot create the table itself.
It also does the same thing for destructors, with the kernel name being
"spirv$device$fini", the table pointers __fini_array_start and
__fini_array_end, and the generated symbols prefix __fini_array_object.
The code was mostly generated by Claude 4.5 and has been reviewed by me
to the best of my ability.
- Fix buildNDRange according to OpenCL and SPIRV specs.
- Fix tablegen SPIRV builtins for ndrange_* functions: despite of OpenCL
spec, the real call has additional first argument - structure return,
changed min and max num arguments accordingly.
- Update test, add checks, combined with BuildNDRange_2
This adds a SPIR-V intrinsic for associating a name (textual identifier)
to a specialisation constant. The name is encoded in metadata, and is
intended to be used within LLVM / by the SPIR-V BE (e.g. #134016 would
be a direct user), as it is never emitted into the SPIR-V object.
Non-boolean and composite specialisation constants will be handled in
the future, via dedicated intrinsics, if there is interest.
https://github.com/KhronosGroup/SPIRV-Tools/pull/6232 added support for
`SPV_INTEL_function_pointers` on `spirv-val`.
This PR updates some relevant tests to run `spirv-val` and document why
some others are failing.
SPIR-V backend previously only supported function annotations in
llvm.global.annotations and crashed with a fatal error when encountering
global variable entries
Add missing capabilities to finalize SPV_INTEL_16bit_atomics extension:
- AtomicInt16CompareExchangeINTEL (6260): for i16
load/store/exchange/cmpxchg
- Int16AtomicsINTEL (6261): for i16 arithmetic atomics (add, sub, min,
max, etc.)
- AtomicBFloat16LoadStoreINTEL (6262): for bfloat16 load/store/exchange
This completes the implementation started in 6ef3218.
Specification:
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_16bit_atomics.asciidoc
This PR adds QuadReadAcrossX intrinsic support in HLSL with codegen for
both DirectX and SPIRV backends. Resolves
https://github.com/llvm/llvm-project/issues/99175.
- [x] Implement QuadReadAcrossX clang builtin
- [x] Link QuadReadAcrossX clang builtin with hlsl_intrinsics.h
- [x] Add sema checks for QuadReadAcrossX to
CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
- [x] Add codegen for QuadReadAcrossX to EmitHLSLBuiltinExpr in
CGBuiltin.cpp
- [x] Add codegen tests to
clang/test/CodeGenHLSL/builtins/QuadReadAcrossX.hlsl
- [x] Add sema tests to
clang/test/SemaHLSL/BuiltIns/QuadReadAcrossX-errors.hlsl
- [x] Create the int_dx_QuadReadAcrossX intrinsic in
IntrinsicsDirectX.td
- [x] Create the DXILOpMapping of int_dx_QuadReadAcrossX to 123 in
DXIL.td
- [x] Create the QuadReadAcrossX.ll and QuadReadAcrossX_errors.ll tests
in llvm/test/CodeGen/DirectX/
- [x] Create the int_spv_QuadReadAcrossX intrinsic in IntrinsicsSPIRV.td
- [x] In SPIRVInstructionSelector.cpp create the QuadReadAcrossX
lowering and map it to int_spv_QuadReadAcrossX in
SPIRVInstructionSelector::selectIntrinsic.
- [x] Create SPIR-V backend test case in
llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadAcrossX.ll
This PR improves the SPIR-V instruction printer output for integer
constants using `SPV_ALTERA_arbitrary_precision_integers` extension.
Previously, when `OpConstantI` was encoded with multiple 32-bit words
(for integer widths > 64), the inst printer printed the raw per-word
immediates. This was hard to read and did not reflect the actual value
at the declared integer bitwidth.
Now, with the change in this patch, the instruction printer reconstructs
the multi-word literal into a single `APInt`, truncates it to the
bitwidth declared by the corresponding `OpTypeInt`, and prints the
resulting value as one readable integer (including correct negative
values where applicable).
---------
Co-authored-by: yixing.zhang <yixingzh@smtp.igk.intel.com>
SPIRV backend uses 0 to represent unbounded arrays. This patch makes
unbounded resources be represented with 0 when binding them, as well as
makes sure the backend uses OpTypeRuntimeArray to represent such cases.
Fix: https://github.com/llvm/llvm-project/issues/183367
- fixes#179879
- Change is three fold:
1. Look for the Matrix Memory layout.
2. refactor out the common pieces of loadVectorFromArray into a helper
that can be shared with the matrix case.
3. The matrix case needs special indexing so we can do vector geps
instead of scalar geps that would require a 2d loop.
The `int_spv_gep` intrinsic was defined with `llvm_anyptr_ty` which
forced it to return a scalar pointer. Change the return type to
`llvm_any_ty` to allow the intrinsic to match the actual result type of
the original GEP, whether scalar or vector
BranchProbabilityInfo will compute it's own dominator tree and
post-dominator tree if none is specified; avoid this by using the
analysis manager/pass manager to get the analysis, which will reuse the
previously computed DomTree.
[This
line](https://github.com/ambergorzynski/llvm-project/blob/main/llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp#L2362)
in the SPIRV backend is uncovered by the existing test suite (checked
using coverage, and by asserting that no tests in the existing test
suite fails if we insert an `abort()` at this line).
We propose a test that covers this line. We demonstrate the test by
inserting an `abort()` at that line in commit
[#e37e88e](e37e88e250).
Running all tests shows that only our proposed test fails in the
presence of the `abort`. We've removed the `abort` ahead of merging.
This is the only test that fails in the presence of the abort (which
includes our new test): `LLVM.CodeGen/SPIRV/instructions/icmp.ll`
In order to make codegen compatible with DXC and pass spirv validation,
this patch updates the codegen for half types following the same codegen
as DXC.
fix: https://github.com/llvm/llvm-project/issues/183611
Adds the intrinsics resource_load_level intrinic for DXIL and SPIR-V. It
will be used to load a value from an specific location in the image at
the given mip level. It will be used to implement the Texture Load and
mips[][] methods.
Assisted-by: Gemini
They should be translated to SPIR-V and have Import linkage (unless they
are Interface variables).
Also add a test for protected visibility, just to make sure, that we are
aligned it its translation.
Adds the WaveActiveBitOr intrinsic from issue #99167. This intrinsic
required a bit more work than the last intrinsics that I have done.
There are some peculiarities, which I verified with dxcompiler:
- WaveActiveBitOr only works on uint and uint64_t, no other types are
allowed
- There is no 16 bit version of WaveActiveBitOr
Followed the checklist:
- [x] Implement WaveActiveBitOr clang builtin,
- [x] Link WaveActiveBitOr clang builtin with hlsl_intrinsics.h
- [x] Add sema checks for WaveActiveBitOr to
CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
- [x] Add codegen for WaveActiveBitOr to EmitHLSLBuiltinExpr in
CGBuiltin.cpp
- [x] Add codegen tests to
clang/test/CodeGenHLSL/builtins/WaveActiveBitOr.hlsl
- [x] Add sema tests to
clang/test/SemaHLSL/BuiltIns/WaveActiveBitOr-errors.hlsl
- [x] Create the int_dx_WaveActiveBitOr intrinsic in
IntrinsicsDirectX.td
- [x] Create the DXILOpMapping of int_dx_WaveActiveBitOr to 120 in
DXIL.td
- [x] Create the WaveActiveBitOr.ll and WaveActiveBitOr_errors.ll tests
in llvm/test/CodeGen/DirectX/
- [x] Create the int_spv_WaveActiveBitOr intrinsic in IntrinsicsSPIRV.td
- [x] In SPIRVInstructionSelector.cpp create the WaveActiveBitOr
lowering and map it to int_spv_WaveActiveBitOr in
SPIRVInstructionSelector::selectIntrinsic.
- [x] Create SPIR-V backend test case in
llvm/test/CodeGen/SPIRV/hlsl-intrinsics/WaveActiveBitOr.ll
offload test: https://github.com/llvm/offload-test-suite/pull/487
This fixes an assertion I was hitting in the fragment density map sample
in Vulkan Samples. In starfield.frag.hlsl, we have
float starCol = pow((rnd - threshhold) / (1.0 - threshhold), 16.0);
The optimizer recognizes 16.0 as a whole number and converts the call to
`llvm.powi`. The backend goes on to fail with:
fatal error: error in backend: cannot select: %46:fid(s64) = nnan ninf
nsz arcp afn reassoc G_FPOWI %44:fid, %45:iid(s64) (in function:
_Z9starFieldDv3_f)
On Vulkan, there is no integer-exponent for pow. This patch lowers it by
converting the exponent to float and calling GLSL.std.450's Pow.
---------
Co-authored-by: Steven Perron <stevenperron@google.com>
Without the overload constrain lowering would go with a default path
which would later result in a crash in case if for example AMDGPU asm is
inlined (for example v_ would be Unknown).
Overload sets ConstraintType to be always RegClass for SPIR-V.
From issue #99165, adds the implementation of WaveActiveProduct. This
time with the new types for SPIRVTypeInst
- [x] Implement WaveActiveProduct clang builtin,
- [x] Link WaveActiveProduct clang builtin with hlsl_intrinsics.h
- [x] Add sema checks for WaveActiveProduct to
CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
- [x] Add codegen for WaveActiveProduct to EmitHLSLBuiltinExpr in
CGBuiltin.cpp
- [x] Add codegen tests to
clang/test/CodeGenHLSL/builtins/WaveActiveProduct.hlsl
- [x] Add sema tests to
clang/test/SemaHLSL/BuiltIns/WaveActiveProduct-errors.hlsl
- [x] Create the int_dx_WaveActiveProduct intrinsic in
IntrinsicsDirectX.td
- [x] Create the DXILOpMapping of int_dx_WaveActiveProduct to 119 in
DXIL.td
- [x] Create the WaveActiveProduct.ll and WaveActiveProduct_errors.ll
tests in llvm/test/CodeGen/DirectX/
- [x] Create the int_spv_WaveActiveProduct intrinsic in
IntrinsicsSPIRV.td
- [x] In SPIRVInstructionSelector.cpp create the WaveActiveProduct
lowering and map it to int_spv_WaveActiveProduct in
SPIRVInstructionSelector::selectIntrinsic.
- [x] Create SPIR-V backend test case in
llvm/test/CodeGen/SPIRV/hlsl-intrinsics/WaveActiveProduct.ll
