77363 Commits

Author SHA1 Message Date
Craig Topper
0fe0ef4d7f [RISCV] Remove unused tablegen class. NFC 2024-03-25 13:40:26 -07:00
Jeffrey Byrnes
b761137049
[AMDGPU] Use correct VGPR threshold for flagging ExcessRP regions in unified register file case (#85860)
`ST.getMaxNumVGPRs(MF)` lowers to `AMDGPUBaseInfo.cpp:getTotalNumVGPRs`
which returns 512 for gfx90a. This is subsequently limited by
`AMDGPUBaseInfo:getAddressableNumVGPRs()`, which also returns 512 for
gfx90a. The ISA states we can have a total of 512 registers, but a
maximum of only 256 of each of AGPR and VGPR (gfx90a 3.6.4).

Therefore, in unified register file case, `ST.getMaxNumVGPRs(MF)`
calculates the maximum number of combined VGPR + AGPR. But, it is
currently used as the limit for accvgpr and as the limit for archvgpr.

This patch uses it as the combined limit, and accounts for the maximum addressable arch/acc VGPRs when calculating the per RegClass limits.

It is not unreasonable to think other clients of getTotalNumVGPRs are
using it in the wrong way.
2024-03-25 13:11:58 -07:00
Craig Topper
6ee39fb4c1 [RISCV] Remove unused forceMergeOpRead from SchedTernary class. NFC
Ternary operations cannot have a merge op in addition to their 3 sources
so there is no need for this option.
2024-03-25 12:50:05 -07:00
Craig Topper
25bcdab478 [RISCV] Fix indentation and 80 columns in RISCVInstrInfoZvk.td. NFC
I didn't fix all the 80 column errors just the ones in template
arguments.
2024-03-25 12:42:07 -07:00
Michael Maitland
9056ce8804 Revert "[RISCV][GISEL] Legalize G_VSCALE"
This reverts commit 47681506ded30fada68f180b5e80f740bc76abcd. It is not
consistent with SelectionDAG.
2024-03-25 11:46:02 -07:00
Simon Pilgrim
3dcf62b5ee [X86] HSW/BDW - vector splat shifts don't use Port5 when loading the shift amount
Noticed while trying to compare splat vs per-element shift perf stats for #39424

Confirmed with uops.info
2024-03-25 18:22:29 +00:00
Craig Topper
ce37a7131f
[RISCV] Add integer RISCVISD::SELECT_CC to canCreateUndefOrPoison and isGuaranteedNotToBeUndefOrPoison. (#84693)
Integer RISCVISD::SELECT_CC doesn't create poison. If none of the,
operands are poison, the result is not poison.

This allows ISD::FREEZE to be hoisted above RISCVISD::SELECT_CC.
2024-03-25 11:10:58 -07:00
Michael Maitland
3305f5a980 [RISCV][GISEL] Check for legal element count in typeIsLegalBoolVec
https://github.com/llvm/llvm-project/pull/84553 added
typeIsLegalBoolVec but it was missing a check for minimum element count.
This patch adds that.
2024-03-25 10:49:41 -07:00
Michael Maitland
47681506de [RISCV][GISEL] Legalize G_VSCALE
G_VSCALE should be lowered using VLENB.
2024-03-25 10:44:58 -07:00
Michael Maitland
973e9dbd57 [RISCV][GISEL] Regbank select for scalable G_SELECT 2024-03-25 10:35:21 -07:00
Michael Maitland
8c51ac9ddb [RISCV][GISEL] Legalize G_SELECT for scalable vectors 2024-03-25 10:35:21 -07:00
Michael Maitland
ea798a7900 [RISCV][GISEL] Legalize and regbankselect vector typed G_IMPLICIT_DEF 2024-03-25 10:19:14 -07:00
Michael Maitland
33cf27b0a3 [RISCV][GISEL][NFC] Add typeIsLegalBoolVec and typeIsLegalIntOrFPVec helpers 2024-03-25 10:19:13 -07:00
David Green
96819daa3d
[AArch64] Handle v2i16 and v2i8 in concat load combine. (#86264)
This extends the concat load patch from
https://reviews.llvm.org/D121400, which was later moved to a combine, to
handle v2i8 and v2i16 concat loads too.
2024-03-25 17:10:23 +00:00
Justin Bogner
a83ed0423d
[DirectX] Follow naming conventions for enumerators in DXILABI.h. NFC (#86237)
These all-caps names differ from the llvm naming conventions for no good
reason, and `VOID` in all caps can cause problems in windows
environments (see [1]). Rename them to UpperCamelCase.

[1]: https://github.com/clangd/clangd/issues/1983
2024-03-25 09:49:32 -07:00
Evgenii Kudriashov
fb394562a3
[X86][GlobalISel] Fix referencing nonexistent operand in G_ICMP (#86221)
Fixes #86203
2024-03-25 16:46:12 +01:00
David Stuttard
06cfbe3cfd
[AMDPU] Add support for idxen and bothen buffer load/store merging in SILoadStoreOptimizer (#86285)
Added more buffer instruction merging support
2024-03-25 14:44:22 +00:00
David Green
4e29c6acd3
[AArch64] Correct Neoverse V1 SVE 16-bit sdot/udot schedule pipelines. (#86142)
Fixes #86102
2024-03-25 11:20:56 +00:00
Nathan Gauër
f0eb908340
[SPIR-V] Add WaveGetLaneIndex() intrinsic support (#85979)
Add support to generate valid SPIR-V for the WaveGetLaneIndex() HLSL
builtin.

To implement this, I had to fix a few small issues in the backend, like
the i8* pointer type being emitted, even if we have the type information
elsewhere.

Signed-off-by: Nathan Gauër <brioche@google.com>
2024-03-25 11:30:47 +01:00
Mariusz Sikora
94a550dab2
[AMDGPU][NFC] Rename Feature GFX11FullVGPRs to 1_5xVGPRs (#86468) 2024-03-25 11:00:59 +01:00
Shih-Po Hung
3cb024198f
[RISCV][CostModel] Estimate cost of llvm.vector.reduce.fmaximum/fminimum (#80697)
The ‘llvm.vector.reduce.fmaximum/fminimum.*’ intrinsics propagate NaNs
if any element of the vector is a NaN.
Following #79402, the patch adds the cost for NaN check (vmfne + vcpop)
2024-03-25 17:17:36 +08:00
Vyacheslav Levytskyy
b0d03ccc08
[SPIR-V] Fix illegal OpConstantComposite instruction with non-const constituents in SPIR-V Backend (#86352)
This PR fixes illegal use of OpConstantComposite with non-constant
constituents. The test attached to the PR is able now to satisfy
`spirv-val` check. Before the fix SPIR-V Backend produced for the
attached test case a pattern like
```
%a = OpVariable %_ptr_CrossWorkgroup_uint CrossWorkgroup %uint_123
%11 = OpConstantComposite %_struct_6 %a %a
```
so that `spirv-val` complained with
```
error: line 25: OpConstantComposite Constituent <id> '10[%a]' is not a constant or undef.
  %11 = OpConstantComposite %_struct_6 %a %a
```
2024-03-25 10:14:46 +01:00
Vyacheslav Levytskyy
1d250d9099
[SPIR-V] Improve type inference in SPIR-V Backend for opaque pointers (#86283)
This PR improves type inference in SPIR-V Backend for opaque pointers,
accounting or a case when there is a chain of function calls that allows
to deduce formal parameter types from actual arguments. The attached
test demonstrates the case.
2024-03-25 10:14:08 +01:00
Vyacheslav Levytskyy
99c40f6ba6
[SPIR-V] Introduce a command line option to support compatibility with Khronos SPIRV Translator (#86101)
SPIRV-LLVM-Translator project
(https://github.com/KhronosGroup/SPIRV-LLVM-Translator) from Khronos
Group is a tool and a library for bi-directional translation between
SPIR-V and LLVM IR. In its backward translation from SPIR-V to LLVM IR
SPIRV-LLVM-Translator isn't necessarily able to cover the same SPIR-V
patterns/instructions set that SPIRV Backend produces, even if we target
the same SPIR-V version in both SPIRV-LLVM-Translator and SPIRV Backend
projects.

To improve interoperability and ability to apply SPIRV Backend output in
different products this PR introduces a notion of a mode of SPIR-V
output that is compatible with a subset of SPIR-V supported by
SPIRV-LLVM-Translator. This includes a new command line option that
doesn't influence default behavior of SPIRV Backend and one test case
that demonstrates how this command line option may be used to get a
practical benefit of producing that one of two possible and similar
output options that can be understood by SPIRV-LLVM-Translator.
2024-03-25 10:13:42 +01:00
David Stuttard
75e528fdd9
[AMDGPU] Extend zero initialization of return values for TFE (#85759)
buffer_load instructions that use TFE also need to zero initialize
return values similar to how the image instructions currently work. Add
support for this with standard zero init of all results + zero init of
just TFE flag when enable-prt-strict-null subtarget feature is disabled.
2024-03-25 09:01:46 +00:00
Pierre van Houtryve
babbdad15b
[AMDGPU] Handle non-register operands for S_SUB/ADD_U64_PSEUDO (#86104)
This pseudo uses SSrc_b64 so it allows both an immediate or a register,
but the lowering crashed on immediate operands.
2024-03-25 09:23:40 +01:00
Luke Lau
373e77b4c0
[RISCV] Generalize (sub zext, zext) -> (sext (sub zext, zext)) to add (#86248)
This generalizes the combine added in #82455 to other binary ops,
beginning with adds in this patch.

Because the two zext operands are always +ve when treated as signed, and
we don't get any overflow since the add is carried out in at least N * 2
bits of the narrow type, the result of the add will always be +ve. So we
can use a zext for the outer extend, unlike sub which may produce a -ve
result from two +ve operands.

Although we could still use sext for add, I plan to add support for
other binary ops like mul in a later patch, but mul requires zext to be
correct (because the maximum value will take up the full N * 2 bits). So
I've opted to use zext here too for consistency.

Alive2 proof: https://alive2.llvm.org/ce/z/PRNsUM
2024-03-25 13:08:56 +08:00
Wang Pengcheng
d9746a6a5d
[RISCV][NFC] Pass LMUL to copyPhysRegVector
The opcode will be determined by LMUL.

Reviewers: preames, lukel97, topperc

Reviewed By: lukel97, topperc

Pull Request: https://github.com/llvm/llvm-project/pull/84448
2024-03-25 12:42:59 +08:00
Wang Pengcheng
6af6416e89
[RISCV] Add a tune feature to disable stripping W suffix (#86255)
We have a hidden option to disable it, but I'd like to make it a
tune feature.

For some implementations, instructions with W suffix would be less
costly as they only perform on 32 bits data. Though we may lose some
chances to compress.
2024-03-25 11:44:16 +08:00
Sergei Barannikov
5e5b656102
[MC] Make MCParsedAsmOperand::getReg() return MCRegister (#86444) 2024-03-25 05:13:48 +03:00
Phoebe Wang
2e4e04c590
[X86][BF16] Do not lower to VCVTNEPS2BF16 without AVX512VL (#86395)
Fixes: #86305
2024-03-25 10:06:12 +08:00
Min Hsu
67f2267ae0 [M68k][NFC] Suppress warning on an unused variable when assertion is disabled
NFC.
2024-03-24 14:24:44 -07:00
yingopq
5d7fd6a04a
[Mips] Restore wrong deletion of instruction 'and' in unsigned min/max processing. (#85902)
Fix #61881
2024-03-24 02:35:42 -04:00
Fangrui Song
87c7f4a12b [MC] Remove unnecessary reversal of relocations. NFC
Commit f44db24e1fd948c75c87aea017646f16553d3361 (2015) enabled this
simplication.
2024-03-23 10:03:09 -07:00
Harvin Iriawan
57146daeaa
[CodeGen] Update for scalable MemoryType in MMO (#70452)
Remove getSizeOrUnknown call when MachineMemOperand is created.  For Scalable
TypeSize, the MemoryType created becomes a scalable_vector.

2 MMOs that have scalable memory access can then use the updated BasicAA that
understands scalable LocationSize.

Original Patch by Harvin Iriawan
Co-authored-by: David Green <david.green@arm.com>
2024-03-23 12:56:25 +00:00
Evgenii Kudriashov
d365a45cb3
[GlobalISel] Introduce G_TRAP, G_DEBUGTRAP, G_UBSANTRAP (#84941)
Here we introduce three new GMIR instructions to cover a set of trap
intrinsics. The idea behind it is that generic intrinsics shouldn't be
used with G_INTRINSIC opcode.

These new instructions can match perfectly with existing trap ISD nodes.
It allows X86, AArch64, RISCV and Mips to reuse SelectionDAG patterns for
selection and avoid manual selection. However AMDGPU is an exception. It
selects traps during legalization regardless SelectionDAG or GlobalISel.

Since there are not many places where traps are used, this change
attempts to clean up all the usages of G_INTRINSIC with trap intrinsics. So,
there is no stage when both G_TRAP and
G_INTRINSIC_W_SIDE_EFFECTS(@llvm.trap) are allowed.
2024-03-23 13:12:44 +01:00
paperchalice
ef57977f2a
[NewPM][Hexagon] Add HexagonPassRegistry.def (#86244)
Prepare for dag-isel, also migrate some test case
2024-03-23 15:02:27 +08:00
paperchalice
635ea257ec
[NewPM] Fix BPF build (#86379)
Add Passes in dependency list
2024-03-23 13:06:58 +08:00
paperchalice
2aa5bae0c0
[NewPM][BPF] Add BPFPassRegistry.def NFCI (#86241)
Prepare migration for dag-isel.
2024-03-23 12:53:26 +08:00
paperchalice
7ac7d418ac
[NewPM][NVPTX] Add NVPTXPassRegistry.def NFCI (#86246)
Prepare for dag-isel migration.
2024-03-23 11:20:18 +08:00
paperchalice
76fdb5902f
[NewPM][DirectX] Add DirectXPassRegistry.def NFCI (#86242)
Prepare migration for dag-isel
2024-03-23 10:37:53 +08:00
Philipp Tomsich
b621269d4a
[AArch64] Adjust ROBsize for Ampere1B (NFC) (#86331)
To align more closely with common usage, we now use the size of the
reorder-buffer for MicroOpBufferSize instead of the entries of the
global micro-op scheduler.
2024-03-22 16:30:12 -07:00
Philipp Tomsich
af63c6e5d0
[AArch64] Adjust ROBsize for Ampere1/Ampere1A (NFC) (#86330)
To align more closely with common usage, we now use the size of the
reorder-buffer for MicroOpBufferSize instead of the entries of the
global micro-op scheduler.
2024-03-22 16:29:58 -07:00
Paul Kirth
dfe4ca9b7f
[RISCV][lld] Set the type of TLSDESC relocation's referenced local symbol to STT_NOTYPE
When adding fixups for RISCV_TLSDESC_ADD_LO and RISCV_TLSDESC_LOAD_LO,
the local label added for RISCV TLSDESC relocations have STT_TLS set,
which is incorrect. Instead, these labels should have `STT_NOTYPE`.

This patch stops adding such fixups and avoid setting the STT_TLS on
these symbols. Failing to do so can cause LLD to emit an error `has an
STT_TLS symbol but doesn't have an SHF_TLS section`. We additionally,
adjust how LLD services these relocations to avoid errors with
incompatible relocation and symbol types.

Reviewers: topperc, MaskRay

Reviewed By: MaskRay

Pull Request: https://github.com/llvm/llvm-project/pull/85817
2024-03-22 12:27:41 -07:00
Ulrich Weigand
4b907414d2 [SystemZ] Add support for llvm.readcyclecounter
The llvm.readcyclecounter intrinsic can be implemented via the
STORE CLOCK FAST (STCKF) instruction.
2024-03-22 20:01:02 +01:00
Craig Topper
e64e15ee59 [RISCV] Move the RISCVSchedule.td include after RISCVRegisterInfo.td. NFC
Registers shouldn't depend on the scheduler, but a scheduler
predicate could depend on a register.

This would make it possible to move VLDSX0Pred out of the SiFive7
scheduler model to RISCVSchedule.td if another model needed it.
2024-03-22 11:42:12 -07:00
Craig Topper
fb329f1844
[Target] Move SubRegIdxRanges from MCSubtargetInfo to TargetInfo. (#86245)
I'm planning to add HwMode support to SubRegIdxRanges for RISC-V GPR
pairs. The MC layer is currently unaware of the HwMode for registers and
I'd like to keep it that way.

This information is not used by the MC layer so I think it is safe to
move it.
2024-03-22 11:15:45 -07:00
Sacha Coppey
d2f8ba7d6d
[RISCV][NFC] Add generateMCInstSeq in RISCVMatInt (#84462)
This allows to avoid duplicating the code handling the instructions
outputted by `generateInstSeq` when emitting `MCInst`s.
2024-03-23 01:08:13 +08:00
Farzon Lotfi
79c32eb03d
[DXIL] Add lowerings for cosine and floor (#86173)
Completes #86170
Completes #86172
- `DXIL.td` - Add changes to lower the cosine and floor intrinsics to
dxilOps.
2024-03-22 07:02:47 -04:00
Farzon Lotfi
d8e5c0b4e5
[DXIL] Complete abs lowering (#86158)
This change completes #86155
- `DXIL.td` - lowering `fabs` intrinsic to the float dxil op.
- `DXILIntrinsicExpansion.cpp` - Add intrinsic expansion for the abs
case.
2024-03-22 07:01:01 -04:00