Load monitor operations make more sense as atomic operations, as
non-atomic operations cannot be used for inter-thread communication w/o
additional synchronization.
The previous built-in made it work because one could just override the
CPol bits, but that bypasses the memory model and forces the user to learn
about ISA bits encoding.
Making load monitor an atomic operation has a couple of advantages.
First, the memory model foundation for it is stronger. We just lean on the
existing rules for atomic operations. Second, the CPol bits are abstracted away
from the user, which avoids leaking ISA details into the API.
This patch also adds supporting memory model and intrinsics
documentation to AMDGPUUsage.
Solves SWDEV-516398.
Exactly match the s_wait_event instruction. For some reason we already
had this instruction used through llvm.amdgcn.s.wait.event.export.ready,
but that hardcodes a specific value. This should really be a bitmask
that
can combine multiple wait types.
gfx11 -> gfx12 broke compatabilty in a weird way, by inverting the
interpretation of the bit but also shifting the used bit by 1. Simplify
the selection of the old intrinsic by just using the magic number 2,
which should satisfy both cases.
## Summary
`__builtin_amdgcn_global_load_lds` and related builtins crash when the
size argument is a template parameter. The semantic checker calls
`VerifyIntegerConstantExpression` without first checking if the
expression is value-dependent, causing an assertion failure.
**Affected builtins:**
- `__builtin_amdgcn_global_load_lds`
- `__builtin_amdgcn_raw_ptr_buffer_load_lds`
- `__builtin_amdgcn_struct_ptr_buffer_load_lds`
- `__builtin_amdgcn_load_to_lds`
**Reproducer:**
```cpp
template<int N>
__device__ void load(void* src, __shared__ void* dst) {
__builtin_amdgcn_global_load_lds(src, dst, N, 0, 0); // Crashes
}
template __device__ void load<16>(void*, __shared__ void*);
```
**Error before fix:**
```
Assertion `!isValueDependent() && "Expression evaluator can't be called on a dependent expression."' failed
```
**Fix:** Add `isValueDependent()` check before evaluating the constant
expression. Value-dependent expressions are accepted during template
definition and properly checked during template instantiation.
## Test plan
- [x] New lit test
`clang/test/SemaHIP/amdgpu-global-load-lds-template.hip`
- Tests valid template parameter sizes compile without errors
- Tests invalid sizes are caught at template instantiation
- [x] All existing SemaHIP tests pass (17/17)
- [x] Related builtin tests pass
This reverts commit 2c376ffeca490a5732e4fd6e98e5351fcf6d692a because it
breaks assembler.
```
$ llvm-mc -triple=amdgcn -mcpu=gfx1250 -show-encoding <<< "v_wmma_i32_16x16x64_iu8 v[16:23], v[0:7], v[8:15], v[16:23] matrix_b_reuse"
v_wmma_i32_16x16x64_iu8 v[16:23], v[0:7], v[8:15], v[16:23] clamp ; encoding: [0x10,0x80,0x72,0xcc,0x00,0x11,0x42,0x1c]
```
We have a fundamental issue in the clamp support in VOP3P instructions,
which will need more changes.
Fixes#166989
- Adds a clamp immediate operand to the AMDGPU WMMA iu8 intrinsic and
threads it through LLVM IR, MIR lowering, Clang builtins/tests, and MLIR
ROCDL dialect so all layers agree on the new operand
- Updates AMDGPUWmmaIntrinsicModsAB so the clamp attribute is emitted,
teaches VOP3P encoding to accept the immediate, and adjusts Clang
codegen/builtin headers plus MLIR op definitions and tests to match
- Documents what the WMMA clamp operand do
- Implement bitcode AutoUpgrade for source compatibility on WMMA IU8
Intrinsic op
Possible future enhancements:
- infer clamping as an optimization fold based on the use context
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
#140210 added `#include "clang/Frontend/FrontendDiagnostic.h"` to
clang/lib/Sema/SemaAMDGPU.cpp, but Frontend itself has a dependency on
Sema. This creates a layering issue as described in
https://llvm.org/docs/CodingStandards.html#library-layering.
Fortunately, d076608d58d1ec55016eb747a995511e3a3f72aa made this easy to
fix, as clang/Frontend/FrontendDiagnostic.h just forwards to
clang/Basic/DiagnosticFrontend.h, so it's trivial to make this depend on
basic instead of frontend.
Before, the diagnostic was emitted immediately, as soon as the error was
detected. This is problematic during the host compilation, since
the compiler performs semantic analysis of `__device__` functions with
the host's
target attributes.
A solution for this is to use `SemaRef.targetDiag` to defer the
diagnostic. The diagnostic will then be printed only if the function is
emitted.
The test included in this patch highlights a second problem: we cannot
compile a file having a call to `__builtin_amdgcn_load_to_lds` on a
`__device__` function since we typecheck the signature.
The issue is that, `__shared__ void*` on X86 doesn't translate to
`addrspace(3) void*`, so the compilation fails.
Fixes: SWDEV-555501
- Add clang built-ins + sema/codegen
- Add IR Intrinsic + verifier
- Add DAG/GlobalISel codegen for the intrinsics
- Add lowering in SIMemoryLegalizer using a MMO flag.
This is essentially just a revision of #137678 which only exposes a
builtin for the intrinsic `llvm.amdgcn.struct.ptr.buffer.load.lds`,
which expects an `__amdgpu_buffer_rsrc_t` rather than a `v4i32` as its
first argument.
The reason for excluding the other intrinsics exposed by the cited PR is
because the intrinsics taking a `v4i32` are legacy and should be
deprecated.
This PR adds a amdgns_load_to_lds intrinsic that abstracts over loads to
LDS from global (address space 1) pointers and buffer fat pointers
(address space 7), since they use the same API and "gather from a
pointer to LDS" is something of an abstract operation.
This commit adds the intrinsic and its lowerings for addrspaces 1 and 7,
and updates the MLIR wrappers to use it (loosening up the restrictions
on loads to LDS along the way to match the ground truth from target
features).
It also plumbs the intrinsic through to clang.
`Sema::getCurFunctionDecl(AllowLambda = false)` returns a nullptr when
the lambda declaration is outside a function (for example, when
assigning a lambda to a static constexpr variable).
This triggered an assertion in
`SemaAMDGPU::CheckAMDGCNBuiltinFunctionCall`.
Using `Sema::getCurFunctionDecl(AllowLambda = true)` returns the
declaration of the enclosing lambda.
Stumbled with this issue when refactoring some code in CK.
CK is using either inline assembly or inline LLVM-IR builtins to
generate buffer_load_dword lds instructions.
This patch exposes this instruction as a Clang builtin available on gfx9 and gfx10.
Related to SWDEV-519702 and SWDEV-518861
Recent change applied too strict check for old and src operands match.
These shall be compatible, but not necessarily exactly the same.
Fixes: SWDEV-493072
This patch introduces `SemaAMDGPU`, `SemaARM`, `SemaBPF`, `SemaHexagon`,
`SemaLoongArch`, `SemaMIPS`, `SemaNVPTX`, `SemaPPC`, `SemaSystemZ`,
`SemaWasm`. This continues previous efforts to split Sema up. Additional
context can be found in #84184 and #92682.
I decided to bundle target-specific components together because of their
low impact on `Sema`. That said, their impact on `SemaChecking.cpp` is
far from low, and I consider it a success.
Somewhat accidentally, I also moved Wasm- and AMDGPU-specific function
from `SemaDeclAttr.cpp`, because they were exposed in `Sema`. That went
well, and I consider it a success, too. I'd like to move the rest of
static target-specific functions out of `SemaDeclAttr.cpp` like we're
doing with built-ins in `SemaChecking.cpp` .
