Fixes#184922
- [x] Implement `transpose` clang builtin in `Builtins.td`
- [x] Link `transpose` clang builtin with `hlsl_alias_intrinsics.h`
- [x] Add sema checks for `transpose` to `CheckHLSLBuiltinFunctionCall`
in `SemaHLSL.cpp`
- [x] Add codegen for `transpose` to `EmitHLSLBuiltinExpr` in
`CGHLSLBuiltins.cpp`
- `transpose` lowers to the `llvm.matrix.transpose` intrinsic
- [x] Add codegen tests to
`clang/test/CodeGenHLSL/builtins/transpose.hlsl`
- [x] Add sema tests to
`clang/test/SemaHLSL/BuiltIns/transpose-errors.hlsl`
- [x] Implement lowering of the `llvm.matrix.transpose` intrinsic in the
DXIL backend in `DXILIntrinsicExpansion.cpp`
- The intrinsic lowers to a shufflevector like in DXC
https://hlsl.godbolt.org/z/Gj959q6sq
- [x] Add DXIL lowering tests to
`llvm/test/CodeGen/DirectX/matrix-transpose.ll`
Assisted-by: claude-opus-4.6
Fixes#99138
- Defines a `__builtin_hlsl_mul` clang builtin in `Builtins.td`.
- Links the `__builtin_hlsl_mul` clang builtin with
`hlsl_alias_intrinsics.h` under the name `mul` for matrix cases
- Implement scalar and vector elementwise multiplication cases of the
`mul` function in `hlsl_intrinsics.h` and `hlsl_intrinsic_helpers.h`
- Adds sema for `__builtin_hlsl_mul` to `CheckBuiltinFunctionCall` in
`SemaHLSL.cpp`
- Adds codegen for `__builtin_hlsl_mul` to `EmitHLSLBuiltinExpr` in
`CGHLSLBuiltins.cpp`
- Vector-vector cases lower to `dot` (except double vectors, which
expands to scalar multiply-adds).
- Matrix-matrix, matrix-vector, and vector-matrix multiplication lower
to the `llvm.matrix.multiply` intrinsic
- Adds codegen tests to `clang/test/CodeGenHLSL/builtins/mul.hlsl`
- Adds sema tests to `clang/test/SemaHLSL/BuiltIns/mul-errors.hlsl`
- Implements lowering of the `llvm.matrix.multiply` intrinsic to DXIL in
`DXILIntrinsicExpansion.cpp`
Note: Currently the SPIRV backend does not support row-major matrix
memory layouts when lowering matrix multiply, and just assumes
column-major layout. Therefore this PR also makes the DirectX backend
only assume column-major layout. Implementing support for row-major
order shall be done in a separate PR. (Tracked by
https://github.com/llvm/llvm-project/issues/184906)
This PR locally passes the `mul` offload tests in both DirectX 12 and
Vulkan: https://github.com/llvm/offload-test-suite/pull/941
Assisted-by: claude-opus-4.6
`Constant::isZeroValue` currently behaves same as
`Constant::isNullValue` for all types except floating-point, where it
additionally returns true for negative zero (`-0.0`). However, in
practice, almost all callers operate on integer/pointer types where the
two are equivalent, and the few FP-relevant callers have no meaningful
dependence on the `-0.0` behavior.
This PR removes `isZeroValue` to eliminate the confusing API. All
callers are changed to `isNullValue` with no test failures.
`isZeroValue` will be reintroduced in a future change with clearer
semantics: when null pointers may have non-zero bit patterns,
`isZeroValue` will check for bitwise-all-zeros, while `isNullValue` will
check for the semantic null (which
may be non-zero).
#171456 set `ImplicitTrunc` to false by default. So `NegZero` value was
no longer being created as a signed integer.
This caused a similar crash during `DXILIntrinsicExpansion` as reported
here:
https://github.com/llvm/llvm-project/pull/171456#issuecomment-3718690088.
This change fixes the test case from crashing in the DirectX backend by
manually specifying it as a signed integer.
fixes#165051
This change reverts the experiment we did for #165311
While some backends seem to support llvm.assume without validation The
validator itself does not so it makes more sense to just remove it.
fixes#156068
- We needed to add a new sub arch to the target tripple so we can test
that emulation does not happen when targeting SM6.9
- The HLSL toolchain needed to be updated to handle the conversion of
strings to enums for the new sub arch.
- The emulation is done in DXILIntrinsicExpansion.cpp and needs to be
able to convert both llvm.is.fpclass and lvm.dx.isinf to the proper
emulation
- test updates in TargetParser/TripleTest.cpp, isinf.ll, is_fpclass.ll,
and DXCModeTest.cpp
For SM6.2 and earlier, Raw buffer Loads and Stores can't handle 64 bit
types. This PR expands Raw Buffer Loads and Stores for 64 bit types
double and int64_t. This Adds to the work done in #139996 and #145047 .
Raw Buffer Loads and Stores allow for 64 bit type vectors of size 3 and
4, and the code is modified to allow for that.
Closes#144747
fixes#140321
Specifically it fixes ` error: Cannot create BufferLoad operation:
Invalid overload type`
https://hlsl.godbolt.org/z/dTq4q7o58
but no new DML shaders are building. This change now exposes #144747.
The change does two things it adds i64 support for intrinsic expansion
for the `dx_resource_load_typedbuffer`, and
`dx_resource_store_typedbuffer` intrinsics.
It also lets loaded typedbuffers crash more gracefully because of ` auto
*EVI = cast<ExtractValueInst>(U);` is now a `dyn_cast` and
`llvm_unreachable`.
typedBufferLoad of double/double2 is expanded to a typedBufferLoad of a
<2 x i32>/<4 x i32> and asdouble
typedBufferStore of a double/double2 is expanded to a splitdouble and a
typedBufferStore of a <2 x i32>/<4 x i32>
Add tests showing result of intrinsic expansion for typedBufferLoad and
typedBufferStore
Add tests showing dxil op lowering can handle typedBufferLoad and
typedBufferStore where the target type doesn't match the typedBufferLoad
and typedBufferStore type
Closes#104423
Fixes#137209
This PR:
- Adds a case to `expandIntrinsic()` in `DXILIntrinsicExpansion.cpp` to
expand the `Intrinsic::is_fpclass` in the case of
`FPClassTest::fcNegZero`
- Defines the `IsNaN`, `IsFinite`, `IsNormal` DXIL ops in `DXIL.td`
- Adds a case to `lowerIntrinsics()` in `DXILOpLowering.cpp` to handle
the lowering of `Intrinsic::is_fpclass` to the DXIL ops `IsNaN`,
`IsInf`, `IsFinite`, `IsNormal` when the FPClassTest is `fcNan`,
`fcInf`, `fcFinite`, and `fcNormal` respectively
- Creates a test `llvm/test/CodeGen/DirectX/is_fpclass.ll` to exercise
the intrinsic expansion and DXIL op lowering of `Intrinsic::is_fpclass`
~~A separate PR will be made to remove the now-redundant `dx_isinf`
intrinsic to address #87777.~~
A proper implementation for the lowering of the `llvm.is.fpclass`
intrinsic to handle all possible combinations of FPClassTest can be
implemented in a separate PR. This PR's implementation focuses primarily
on addressing the current use-cases for DirectML and HLSL intrinsics.
This implements the result of the discussion at:
https://discourse.llvm.org/t/rfc-report-fatal-error-and-the-default-value-of-gencrashdialog/73587
There are two different use cases for report_fatal_error, so replace it
with two functions reportFatalInternalError() and
reportFatalUsageError(). The former indicates a bug in LLVM and
generates a crash dialog. The latter does not. The names have been
suggested by rnk and people seemed to like them.
This replaces a lot of the usages that passed an explicit value for
GenCrashDiag. I did not bulk replace remaining report_fatal_error usage
-- they probably require case by case review for which function to use.
The `dx.dot2`, `dot3`, and `dot4` intrinsics exist purely to lower
`dx.fdot`, and they map exactly to the DXIL ops of the same name. Using
vectors for their arguments adds unnecessary complexity and causes us to
have vector operations that are not trivial to lower post-scalarizer.
Similarly, the `dx.dot2add` intrinsic is overly generic for something
that only needs to lower to a single `dot2AddHalf` DXIL op. Update its
signature to match the operation it lowers to.
Fixes#134569.
fixes#135285
This change implements the `usub.sat` intrinsic to perform an unsigned
saturating subtraction on the 2 arguments.
The minimum value this operation is clamp to is 0.
## Changes
- Delete DirectX length intrinsic
- Delete HLSL length lang builtin
- Implement length algorithm entirely in the header.
## History
- In the past if an HLSL intrinsic lowered to either a spirv op code or
a DXIL opcode we represented it with intrinsics
## Why we are moving away?
- To make HLSL apis more portable the team decided that it makes sense
for some intrinsics to be defined only in the header.
- Since there tends to be more SPIRV opcodes than DXIL opcodes the plan
is to support SPIRV opcodes either with target specific builtins or via
pattern matching.
## Changes
- Delete DirectX length intrinsic
- Delete HLSL length lang builtin
- Implement length algorithm entirely in the header.
## History
- In the past if an HLSL intrinsic lowered to either a spirv op code or
a DXIL opcode we represented it with intrinsics
## Why we are moving away?
- To make HLSL apis more portable the team decided that it makes sense
for some intrinsics to be defined only in the header.
- Since there tends to be more SPIRV opcodes than DXIL opcodes the plan
is to support SPIRV opcodes either with target specific builtins or via
pattern matching.
This splits the DXILResourceAnalysis pass into TypeAnalysis and
BindingAnalysis passes. The type analysis pass is made immutable and
populated lazily so that it can be used earlier in the pipeline without
needing to carefully maintain the invariants of the binding analysis.
Fixes#118400
Fixes#88052
- Added the following intrinsics:
- `int_spv_uclamp`
- `int_spv_sclamp`
- `int_spv_fclamp`
- Updated DirectX counterparts to have the same three clamp intrinsics.
- Update the clamp.hlsl unit tests to include SPIRV
- Added the SPIRV specific tests
- add degrees builtin
- link degrees api in hlsl_intrinsics.h
- add degrees intrinsic to IntrinsicsDirectX.td
- add degrees intrinsic to IntrinsicsSPIRV.td
- add lowering from clang builtin to dx/spv intrinsics in CGBuiltin.cpp
- add semantic checks to SemaHLSL.cpp
- add expansion of directx intrinsic to llvm fmul for DirectX in
DXILIntrinsicExpansion.cpp
- add mapping to spir-v intrinsic in SPIRVInstructionSelector.cpp
- add test coverage:
- degrees.hlsl -> check hlsl lowering to dx/spv degrees intrinsics
- degrees-errors.hlsl/half-float-only-errors -> check semantic warnings
- hlsl-intrinsics/degrees.ll -> check lowering of spir-v degrees
intrinsic to SPIR-V backend
- DirectX/degrees.ll -> check expansion and scalarization of directx
degrees intrinsic to fmul
Resolves#99104
Fix an obvious typo in these tests to get them passing, and also fix the
-Wimplicit-fallthrough warning that fires when trying to build.
Reverting #110616 was tricky because of dependencies, so I'm just doing
the easy fix directly here.
This change is part of this proposal:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294
This preliminary work adds the intrinsic to llvm and expands using atan
intrinsic for DXIL backend, since DXIL has no atan2 op.
Part 1 for Implement the atan2 HLSL Function #70096.
(reland #108865 reverted in #109842 due to doc build break)
This change is part of this proposal:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294
This preliminary work adds the intrinsic to llvm and expands using atan
intrinsic for DXIL backend, since DXIL has no atan2 op.
Part 1 for Implement the atan2 HLSL Function #70096.
Use DXILResourceWrapperPass to build the resource table.
Since DXILResourceWrapperPass operates on LLVM intrinsics rather than
DXIL operations, add addPreserved for DXILResourceWrapperPass in the
passes before DXContainerGlobals
Fixes#103275
This adds the SPIRV fdot, sdot, and udot intrinsics and allows them to
be created at codegen depending on the target architecture. This
required moving some of the DXIL-specific choices to DXIL instruction
expansion out of codegen and providing it with at a more generic fdot
intrinsic as well.
Removed some stale comments that gave the obsolete impression that type
conversions should be expected to match overloads.
The SPIRV intrinsic handling involves generating multiply and add
operations for integers and the existing OpDot operation for floating
point.
New tests for generating SPIRV float and integer dot intrinsics are
added as well as expanding HLSL tests to include SPIRV generation
Used new dot product intrinsic generation to implement normalize() in SPIRV
Incidentally changed existing dot intrinsic definitions to use
DefaultAttrsIntrinsic to match the newly added inrinsics
Fixes#88056
All expansions end with replacing the previous inrinsic with the new
expansion and erasing the old one. By moving this operation to the
caller, these expansion functions can be called in more contexts and a
small amount of duplicated code is consolidated.
Pre-req for #88056
This PR adds the length intrinsic and an HLSL function that uses it.
The SPIRV implementation is left for a future PR.
This PR addresses #99134, though some SPIR-V changes still need to be
made to complete the task. Below is how this PR addresses #99134.
- "Implement `length` clang builtin" was done by defining `HLSLL ength`
in Builtins.td
- "Link `length` clang builtin with hlsl_intrinsics.h" was done by using
the alias attribute to make `length` an alias of
`__builtin_hlsl_elementwise_length` in hlsl_intrinsics.h
- "Add sema checks for `length` to `CheckHLSLBuiltinFunctionCall` in
`SemaChecking.cpp` " was done, but in this case not in SemaChecking.cpp,
rather SemaHLSL.cpp. A case was added to the builtin to check for
semantic failures, and set `TheCall` up to have the right return type.
- "Add codegen for `length` to `EmitHLSLBuiltinExpr` in `CGBuiltin.cpp`"
was done. For scalars, fabs is emitted, otherwise, length is emitted.
- "Add codegen tests to `clang/test/CodeGenHLSL/builtins/length.hlsl`
was done to test that `length` in HLSL emits the right intrinsic.
- "Add sema tests to `clang/test/SemaHLSL/BuiltIns/length-errors.hlsl`"
was done to test for diagnostics emitted in SemaHLSL.cpp
- "Create the `int_dx_length` intrinsic in `IntrinsicsDirectX.td`" was
done. Specifying return types and parameter types was difficult, but
`idot` was used for reference, and `llvm\include\llvm\IR\Intrinsics.td`
contains all the ways to express return / parameter types.
- "Create an intrinsic expansion of `int_dx_length` in
`llvm/lib/Target/DirectX/DXILIntrinsicExpansion.cpp`" was done, and was
mostly derived by looking at `TranslateLength` in `HLOperationLower.cpp`
in the DXC codebase.
- "Create the `length.ll` and `length_errors.ll` tests in
`llvm/test/CodeGen/DirectX/`" was done by taking the DXIL output of
`clang/test/CodeGenHLSL/builtins/length.hlsl` and running `opt -S
-dxil-intrinsic-expansion` and ` opt -S -dxil-op-lower` on it, checking
for how the length intrinsic was either expanded or lowered.
- "Create the `int_spv_length` intrinsic in `IntrinsicsSPIRV.td`" was
done by copying `IntrinsicsDirectX.td`.
---------
Co-authored-by: Justin Bogner <mail@justinbogner.com>
Removing the intrinsic because there is no opCodes for rcp in DXIL or
SPIR-V.
Moving means we don't have to re-implement this feature for each
backend.
fixes#87784
Co-authored-by: Farzon Lotfi <farzon@farzon.com>
This change completes #86155
- `DXIL.td` - lowering `fabs` intrinsic to the float dxil op.
- `DXILIntrinsicExpansion.cpp` - Add intrinsic expansion for the abs
case.
this implements part 1 of 2 for #83626
- `CGBuiltin.cpp` - modified to have seperate cases for signed and
unsigned integers.
- `SemaChecking.cpp` - modified to prevent the generation of a double
dot product intrinsic if the builtin were to be called directly.
- `IntrinsicsDirectX.td` creation of the signed and unsigned dot
intrinsics needed for instruction expansion.
- `DXILIntrinsicExpansion.cpp` - handle instruction expansion cases for
integer dot product.
This change implements lowering for #70076, #70100, #70072, & #70102
`CGBuiltin.cpp` - - simplify `lerp` intrinsic
`IntrinsicsDirectX.td` - simplify `lerp` intrinsic
`SemaChecking.cpp` - remove unnecessary check
`DXILIntrinsicExpansion.*` - add intrinsic to instruction expansion
cases
`DXILOpLowering.cpp` - make sure `DXILIntrinsicExpansion` happens first
`DirectX.h` - changes to support new pass
`DirectXTargetMachine.cpp` - changes to support new pass
Why `any`, and `lerp` as instruction expansion just for DXIL?
- SPIR-V there is an
[OpAny](https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#OpAny)
- SPIR-V has a GLSL lerp extension via
[Fmix](https://registry.khronos.org/SPIR-V/specs/1.0/GLSL.std.450.html#FMix)
Why `exp` instruction expansion?
- We have an `exp2` opcode and `exp` reuses that opcode. So instruction
expansion is a convenient way to do preprocessing.
- Further SPIR-V has a GLSL exp extension via
[Exp](https://registry.khronos.org/SPIR-V/specs/1.0/GLSL.std.450.html#Exp)
and
[Exp2](https://registry.khronos.org/SPIR-V/specs/1.0/GLSL.std.450.html#Exp2)
Why `rcp` as instruction expansion?
This one is a bit of the odd man out and might have to move to
`cgbuiltins` when we better understand SPIRV requirements. However I
included it because it seems like [fast math mode has an AllowRecip
flag](https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_fp_fast_math_mode)
which lets you compute the reciprocal without performing the division.
We don't have that in DXIL so thought to include it.
