539 Commits

Author SHA1 Message Date
Mirko Brkušanin
c1a6974d6b
[AMDGPU][MC] Add GFX12 SMEM encoding (#75215) 2023-12-15 09:00:54 +01:00
Mirko Brkušanin
47615ddc84
[AMDGPU][MC] Add GFX12 VFLAT, VSCRATCH and VGLOBAL encodings (#75193) 2023-12-14 14:22:04 +01:00
Mirko Brkušanin
ac406b4817
[AMDGPU][MC] Add GFX12 VBUFFER encoding (#75195) 2023-12-14 12:58:18 +01:00
Mariusz Sikora
7f55d7de1a
[AMDGPU] GFX12: Add Split Workgroup Barrier (#74836)
Co-authored-by: Vang Thao <Vang.Thao@amd.com>
2023-12-13 15:01:13 +01:00
Piotr Sobczak
fac093dd08
[AMDGPU] Update IEEE and DX10_CLAMP for GFX12 (#75030)
Co-authored-by: Stanislav Mekhanoshin <Stanislav.Mekhanoshin@amd.com>
2023-12-13 13:52:40 +01:00
Mariusz Sikora
a97028ac51
[AMDGPU] Update VOP instructions for GFX12 (#74853)
Co-authored-by: Mirko Brkusanin <Mirko.Brkusanin@amd.com>
2023-12-12 11:38:24 +01:00
Kazu Hirata
586ecdf205
[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
2023-12-11 21:01:36 -08:00
Mariusz Sikora
19c9f9c0bf
[AMDGPU] GFX12: Add s_prefetch_inst/data instructions (#74448)
Co-authored-by: Stanislav Mekhanoshin <Stanislav.Mekhanoshin@amd.com>
2023-12-06 10:01:29 +01:00
Mirko Brkušanin
f5868cb6a6
[AMDGPU][MC] Add GFX12 VIMAGE and VSAMPLE encodings (#74062) 2023-12-04 13:04:42 +01:00
Ivan Kosarev
c9cdaffe49
[AMDGPU] Fix operand definitions for atomic scalar memory instructions. (#71799)
CPol and CPol_GLC1 operand classes have identical predicates, which
means AsmParser cannot differentiate between the RTN and non-RTN
variants of the instructions. When it currently selects the wrong
instruction, a hack in cvtSMEMAtomic() corrects the op-code. Using the
new predicated-value operands makes this hack and the whole conversion
function not needed.

Other uses of CPol_GLC1 operands are to be addressed separately.

Resolves about half of the remaining ~1000 pairs of ambiguous
instructions.

Part of <https://github.com/llvm/llvm-project/issues/69256>.
2023-11-10 14:00:46 +00:00
Stanislav Mekhanoshin
f7ab79f33e
[AMDGPU] Allow lit() on operands which do not accept modifiers (#69527) 2023-10-19 10:41:44 -07:00
Konstantin Zhuravlyov
6cfb64276d
AMDGPU: Minor updates to program resource registers (#69525)
- Be explicit about which program resource register is supported by
which target
    - RSRC1
      - FP16_OVFL is GFX9+
      - WGP_MODE is GFX10+
      - MEM_ORDERED is GFX10+
      - FWD_PROGRESS is GFX10+
    - RSRC3
      - INST_PREF_SIZE is GFX11+
      - TRAP_ON_START is GFX11+
      - TRAP_ON_END is GFX11+
      - IMAGE_OP is GFX11+
  - Do not emit GFX11+ fields when disassembling GFX10 code objects
  - Tighten enforcement of reserved bits in disassembler

---------

Co-authored-by: Konstantin Zhuravlyov <kzhuravl@amd.com>
2023-10-19 12:40:19 -04:00
Ivan Kosarev
509b5708e9
[AMDGPU][AsmParser] Eliminate custom predicates for named-bit operands. (#69243)
isGDS() and isTFE() need special treatment, because they may be both
named-bit and token operands.

Part of #62629.
2023-10-17 13:59:42 +01:00
Stanislav Mekhanoshin
06cd6485ae
[AMDGPU] Make ubsan happy (#68959) 2023-10-13 00:41:35 -07:00
Stanislav Mekhanoshin
ab6c3d5034
[AMDGPU] Change the representation of double literals in operands (#68740)
A 64-bit literal can be used as a 32-bit zero or sign extended operand.
In case of double zeroes are added to the low 32 bits. Currently asm
parser stores only high 32 bits of a double into an operand. To support
codegen as requested by the
https://github.com/llvm/llvm-project/issues/67781 we need to change the
representation to store a full 64-bit value so that codegen can simply
add immediates to an instruction.

There is some code to support compatibility with existing tests and asm
kernels. We allow to use short hex strings to represent only a high 32
bit of a double value as a valid literal.
2023-10-12 14:45:45 -07:00
Stanislav Mekhanoshin
e724c7e841
[AMDGPU] Add lit() asm operand modifier for SP3 compatibility. (#68839)
A literal can be optionally wrapped in a lit() expression. This does not
force literal encoding for an inline immediate, but neither does SP3.
The syntax is dummy for compatibility only.
2023-10-12 00:39:56 -07:00
Mirko Brkušanin
2cd2445c21
[AMDGPU] Src1 of VOP3 DPP instructions can be SGPR on supported subtargets (#67461)
In order to avoid duplicating every dpp pseudo opcode that has src1, we
allow it for all opcodes and add manual checks on subtargets that do not
support it.
2023-09-29 11:54:49 +02:00
Ivan Kosarev
9310baa596 [AMDGPU][NFC] Add True16 operand definitions.
Reviewed By: Joe_Nash

Differential Revision: https://reviews.llvm.org/D156103
2023-09-25 16:48:46 +01:00
Pierre van Houtryve
fe2f67e4ba
[AMDGPU] Remove Code Object V2 (#65715)
Code Object V2 has been deprecated for more than a year now. We can
safely remove it from LLVM.

- [clang] Remove support for the `-mcode-object-version=2` option.
- [lld] Remove/refactor tests that were still using COV2
- [llvm] Update AMDGPUUsage.rst
- Code Object V2 docs are left for informational purposes because those
code objects may still be supported by the runtime/loaders for a while.
- [AMDGPU] Remove COV2 emission capabilities.
- [AMDGPU] Remove `MetadataStreamerYamlV2` which was only used by COV2
- [AMDGPU] Update all tests that were still using COV2 - They are either
deleted or ported directly to code object v4 (as v3 is also planned to
be removed soon).
2023-09-21 12:00:45 +02:00
Austin Kerbow
69447d6afe [AMDGPU] Add ASM and MC updates for preloading kernargs
Add assembler directives for preloading kernel arguments that correspond
to new fields in the kernel descriptor for the length and offset of
arguments that will be placed in SGPRs prior to kernel launch. Alignment
of the arguments in SGPRs is equivalent to the kernarg segment when
accessed via the kernarg_segment_ptr. Kernarg SGPRs are allocated
directly after other user SGPRs.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D159459
2023-09-19 15:45:16 -07:00
jwanggit86
37b538819b
[AMDGPU] Incorrect error message regarding SCC modifier (#65660)
For the AMD GFX90A GPU, the SCC instruction modifier is allowed for
certain classes of instructions. However, the current assembler
generates an error message, "scc is not supported on this GPU",
regardless of the instruciton. This fix modifies the message as well as
the logic for generating the message. Related tests are moved from
gfx90a_err.s to gfx90a_asm_features.s.

Co-authored-by: Jun Wang <jun.wang7@amd.com>
2023-09-08 13:07:06 -07:00
Sergei Barannikov
a479be0f39 [MC] Change tryParseRegister to return ParseStatus (NFC)
This finishes the work of replacing OperandMatchResultTy with
ParseStatus, started in D154101.
As a drive-by change, rename some RegNo variables to just Reg
(a leftover from the days when RegNo had 'unsigned' type).
2023-09-06 10:28:12 +03:00
Stanislav Mekhanoshin
cfe9a134bb [AMDGPU] Rename 64BitDPP feature and fix the checks
Names '64BitDPP' and especially 'DPP64' were found misleading, and
DPP64 can easily be mixed with DPP16 and DPP8 while these are
different concepts. DPP16 and DPP8 refers to lanes where DPP64
refers to the operand size.

In fact the essential part here is that these instructions are
executed on the DP ALU, so rename the feature accordingly.

I have also found a bug in a check for these instructions, which is
fixed here and a common utility function is now used.

Differential Revision: https://reviews.llvm.org/D158465
2023-08-22 11:00:10 -07:00
Jay Foad
18919ee759 [AMDGPU] Validate GDS in the assembler
GFX90A DS instructions cannot use the gds modifier, except for the
DS_GWS_* instructions where it is still mandatory.

Differential Revision: https://reviews.llvm.org/D157100
2023-08-11 09:21:55 +01:00
Ivan Kosarev
6b693f5e34 [AMDGPU][AsmParser][NFC] Translate parsed DS instructions to MCInsts automatically.
Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D155620
2023-07-19 11:30:56 +01:00
Ivan Kosarev
7b6e606dac [AMDGPU][AsmParser][NFC] Translate parsed MIMG instructions to MCInsts automatically.
Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D155061
2023-07-13 19:47:31 +01:00
Ivan Kosarev
289ae6525d [AMDGPU][MC] Fix handling of A16 operands in intersect_ray instructions.
The patch adds the support for 'noa16' operands in non-A16 variants of
the instructions, fixes validation of A16 operands and eliminates the
custom conversion to MCInst.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D155057
2023-07-13 19:46:03 +01:00
Scott Linder
986001c827 [AMDGPU] Improve assembler + disassembler handling of kernel descriptors
* Relax the AsmParser to accept `.amdhsa_wavefront_size32 0` when the
  `.amdhsa_shared_vgpr_count` directive is present.
* Teach the KD disassembler to respect the setting of
  KERNEL_CODE_PROPERTY_ENABLE_WAVEFRONT_SIZE32 when calculating the
  value of `.amdhsa_next_free_vgpr`.
* Teach the KD disassembler to disassemble COMPUTE_PGM_RSRC3 for gfx90a
  and gfx10+.
* Include "pseudo directive" comments for gfx10 fields which are not
  controlled by any assembler directive.
* Fix disassembleObject failure diagnostic in llvm-objdump to not
  hard-code a comment string, and to follow the convention of not
  capitalizing the first sentence.

Reviewed By: rochauha

Differential Revision: https://reviews.llvm.org/D128014
2023-07-06 21:20:51 +00:00
Ivan Kosarev
12460cf90f [AMDGPU][AsmParser] Simplify the implementation of SWZ operands.
Those are implicit helper operands and therefore don't need any parsers
or printers.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: piotr, foad

Differential Revision: https://reviews.llvm.org/D154432
2023-07-05 10:45:12 +01:00
Ivan Kosarev
dac5957ca9 [AMDGPU][AsmParser][NFC] Clean up the implementation of KImmFP operands.
addKImmFPOperands() duplicates the KImmFP-specific logic implemented in
addLiteralImmOperand() and therefore can be removed.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D154427
2023-07-05 10:43:41 +01:00
Ivan Kosarev
ee165cdb1b [AMDGPU] Eliminate SIMCCodeEmitter and de-virtualise encoding methods.
Simplifies some future changes needed for
<https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D154337
2023-07-05 10:13:33 +01:00
Sergei Barannikov
d73c8c8fab [AMDGPU] Replace OperandMatchResultTy with ParseStatus (NFC)
ParseStatus is slightly more convenient to use due to implicit
conversion from bool, which allows to do something like:
```
  return Error(L, "msg");
```
when with MatchOperandResultTy it had to be:
```
  Error(L, "msg");
  return MatchOperand_ParseFail;
```
It also has more appropriate name since parse* methods are not only for
parsing operands.

Reviewed By: kosarev

Differential Revision: https://reviews.llvm.org/D154293
2023-07-04 22:29:56 +03:00
Ivan Kosarev
1ef47165fd [AMDGPU][AsmParser][NFC] Remove an unused function.
Was added in <https://reviews.llvm.org/D63293>, but never used.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D154331
2023-07-04 11:33:17 +01:00
Ivan Kosarev
b7e8a55a83 [AMDGPU][AsmParser][NFC] Simplify parsing of sopp_brtarget operands.
Also refine the definitions while there.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: mbrkusanin

Differential Revision: https://reviews.llvm.org/D154061
2023-06-30 16:35:46 +01:00
Ivan Kosarev
59fd48d71e [AMDGPU][AsmParser][NFC] Simplify instruction operand definitions.
This addresses the trivial cases that only require removing the
operand classes and renaming related entities.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D153965
2023-06-29 10:51:44 +01:00
Ivan Kosarev
5183ca8779 [AMDGPU][AsmParser] Eliminate cvtMtbuf().
Now that we have proper support for optional operands, the standard LLVM
machinery can take care of converting parsed instructions to MCInsts.
There are likely more cases where the conversion can be done
automatically, probably with some additional treatment. The plan is to
address them separately.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm, foad

Differential Revision: https://reviews.llvm.org/D153565
2023-06-23 12:43:52 +01:00
Ivan Kosarev
e95457d25b [AMDGPU][AsmParser][NFC] Simplify v_interp-related operand definitions.
Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D152897
2023-06-15 11:04:56 +01:00
Ivan Kosarev
024e540896 [AMDGPU][AsmParser][NFC] Get rid of custom default operand handlers.
Removes the need to add and remove them manually depending on whether
they are used in cvt*() functions. Also removes the compiler warnings
about unused handlers when it happens to be the case.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D151688
2023-06-14 12:10:56 +01:00
Ivan Kosarev
616d30c2cc [AMDGPU][AsmParser][NFC] Simplify the EndpgmImm operand definition.
Clears the road to eliminating custom default operand handlers. Also
unifies naming of related entities.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D151687
2023-06-13 18:46:08 +01:00
Ivan Kosarev
a3a8cbffcf [AMDGPU][AsmParser][NFC] Refine parsing of NamedOperandU32 operands.
Eliminates the need for the custom code in parseCustomOperand().

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D150980
2023-05-29 17:01:44 +01:00
Ivan Kosarev
212af2c081 [AMDGPU][AsmParser] Refine parsing of some 32-bit instruction operands.
Eliminates the need for the custom code in parseCustomOperand().

The remaining uses of NamedOperandU32 are to be addressed separately.

Part of <https://github.com/llvm/llvm-project/issues/62629>.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D150204
2023-05-19 16:54:30 +01:00
Diana Picus
bcbd9b021a [AMDGPU][MC] Don't accept attr > 32 for param_load
The docs say the interpolation attribute should be between 0..32 [1][2],
but we currently accept values all the way up to 63.

This patch makes the ASMParser error out for values > 32. It does not
touch codegen though because we're currently not checking anything at
all for codegen (llvm.amdgcn.lds.param.load will happily accept even 128
as an attr, although that won't fit in the encoding).

[1] https://llvm.org/docs/AMDGPU/gfx8_attr.html#amdgpu-synid-gfx8-attr
[2] https://llvm.org/docs/AMDGPU/gfx11_attr.html#amdgpu-synid-gfx11-attr

Differential Revision: https://reviews.llvm.org/D150261
2023-05-15 08:32:11 +02:00
Diana Picus
7591a7b6ea [AMDGPU][MC] Clean up DPP bound_ctrl handling
At the moment, we set the BC bit in DPP for both bound_ctrl:0 and
bound_ctrl:1, for compatibility with sp3 (see PR35397). However, this
hack is only needed for GFX8. For newer GFXs, sp3 behaves as expected,
i.e. it sets the bit when bound_ctrl:1 and clears it when bound_ctrl:0.

This patch updates LLVM to do the same for GFX11 or newer. We preserve
the current behaviour for GFX9 and 10 so we don't break any existing
code.

Differential Revision: https://reviews.llvm.org/D149254
2023-05-10 11:40:53 +02:00
NAKAMURA Takumi
c1221251fb Restore CodeGen/MachineValueType.h from Support
This is rework of;

  - rG13e77db2df94 (r328395; MVT)

Since `LowLevelType.h` has been restored to `CodeGen`, `MachinveValueType.h`
can be restored as well.

Depends on D148767

Differential Revision: https://reviews.llvm.org/D149024
2023-05-03 00:13:20 +09:00
Jay Foad
735cb7b1f8 [AMDGPU] Avoid using tuple where pair does suffice
Fixes the following building errors, happening with official Android prebuilt clang 14 shipped with Android 13:

external/llvm-project/llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp:5491:13: error: no viable constructor or deduction guide for deduction of template arguments of 'tuple'
          ? std::tuple(HSAMD::V3::AssemblerDirectiveBegin,
            ^
...
external/llvm-project/llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp:5493:13: error: no viable constructor or deduction guide for deduction of template arguments of 'tuple'
          : std::tuple(HSAMD::AssemblerDirectiveBegin,
            ^

Fixes: 6443c0e ("[AMDGPU] Stop using make_pair and make_tuple. NFC.")

Patch by Mauro Rossi!

Differential Revision: https://reviews.llvm.org/D142839
2023-04-21 08:30:35 +01:00
Ivan Kosarev
32f6b6bff5 [AMDGPU][AsmParser] Refine parsing SDWA operands.
Removes the need for the custom code in parseCustomOperand().

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D147241
2023-04-18 15:26:15 +01:00
Ivan Kosarev
5114843983 [AMDGPU][AsmParser] Refine SMRD offset definitions.
- Fixes the type of default 8-bit offset operands.
- Adds a test for optional offsets.

This is effectively an NFC.

Reviewed By: dp

Differential Revision: https://reviews.llvm.org/D142231
2023-03-30 15:29:41 +01:00
Kazu Hirata
7ada7bbee1 [Target] Use *{Set,Map}::contains (NFC) 2023-03-14 18:06:55 -07:00
Jay Foad
dc3882eace [AMDGPU] Fix .amdhsa_shared_vgpr_count error checking for GFX11
Differential Revision: https://reviews.llvm.org/D145936
2023-03-14 09:05:32 +00:00
Ivan Kosarev
b06e5ad8a6 [AMDGPU][AsmParser][NFC] Simplify parsing cache policies.
Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D144954
2023-03-01 12:34:21 +00:00