4591 Commits

Author SHA1 Message Date
quic_hchandel
171d3edd05
[RISCV] Add Qualcomm uC Xqciint (Interrupts) extension (#122256)
This extension adds eleven instructions to accelerate interrupt
servicing.

The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest

This patch adds assembler only support.

---------

Co-authored-by: Harsh Chandel <hchandel@qti.qualcomm.com>
2025-01-13 16:36:05 +05:30
Pengcheng Wang
681c4a2068 Reapply "[RISCV] Rework memcpy test (#120364)"
Use descriptive names and add more cases.

This recommits 59bba39 which was reverted in 4637c77.
2025-01-13 12:06:26 +08:00
Pengcheng Wang
4637c77746
Revert "[RISCV] Rework memcpy test" (#122662)
Reverts llvm/llvm-project#120364

The test should be updated due to some recent changes.
2025-01-13 11:36:37 +08:00
Pengcheng Wang
59bba39a69
[RISCV] Rework memcpy test (#120364)
Use descriptive names and add more cases.
2025-01-13 11:28:24 +08:00
Raphael Moreira Zinsly
6f53886a9a
[RISCV] Add stack clash vector support (#119458)
Use the probe loop structure to allocate vector code in the stack as
well. We add the pseudo instruction RISCV::PROBED_STACKALLOC_RVV to
differentiate from the normal loop.
2025-01-10 09:48:21 -08:00
Philip Reames
24bb180e8a
[RISCV] Attempt to widen SEW before generic shuffle lowering (#122311)
This takes inspiration from AArch64 which does the same thing to assist
with zip/trn/etc.. Doing this recursion unconditionally when the mask
allows is slightly questionable, but seems to work out okay in practice.

As a bit of context, it's helpful to realize that we have existing logic
in both DAGCombine and InstCombine which mutates the element width of in
an analogous manner. However, that code has two restriction which
prevent it from handling the motivating cases here. First, it only
triggers if there is a bitcast involving a different element type.
Second, the matcher used considers a partially undef wide element to be
a non-match. I considered trying to relax those assumptions, but the
information loss for undef in mid-level opt seemed more likely to open a
can of worms than I wanted.
2025-01-10 07:12:24 -08:00
Nikita Popov
eeac0ffaf4 Revert "[MachineLICM] Use RegisterClassInfo::getRegPressureSetLimit (#119826)"
This reverts commit b4e17d4a314ed87ff6b40b4b05397d4b25b6636a.

This causes a large compile-time regression.
2025-01-10 09:05:06 +01:00
Michael Maitland
d0373dbe7c
[RISCV][VLOPT] Add vadc to isSupportedInstr (#122345) 2025-01-09 19:44:40 -05:00
Michael Maitland
04e54cc19f
[RISCV][VLOPT] Add Vector Single-Width Averaging Add and Subtract to isSupportedInstr (#122351) 2025-01-09 19:39:12 -05:00
Michael Maitland
328c3a843f
[RISCV][VLOPT] Add vmerge to isSupportedInstr (#122340) 2025-01-09 16:10:40 -05:00
Michael Maitland
5f70fea79f [RISCV][VLOPT] Add Vector Floating-Point Compare Instructions to getSupportedInstr 2025-01-09 10:50:32 -08:00
Michael Maitland
b419edeec3 [RISCV][VLOPT] Add widening floating point multiply to isSupportedInstr 2025-01-09 10:50:32 -08:00
Michael Maitland
a484fa1d0a [RISCV][VLOPT] Add floating point multiply divide instructions to getSupportedInstr 2025-01-09 10:50:32 -08:00
Michael Maitland
8beb9d393d [RISCV][VLOPT] Add vector widening floating point add subtract instructions to isSupportedInstr 2025-01-09 10:50:31 -08:00
Michael Maitland
c036a9a2c2 [RISCV][VLOPT] Add vector single width floating point add subtract instructions to isSupportedInstr 2025-01-09 10:50:31 -08:00
Philip Reames
031f33cca3 [RISCV] Add tests for legalization of <N x i128> and <N x i256> shuffles 2025-01-09 10:48:45 -08:00
Michael Maitland
d5145715f7
[RISCV][VLOPT] Add vfirst and vcpop to getOperandInfo (#122295) 2025-01-09 13:31:02 -05:00
Michael Maitland
6beaa45526
[RISCV][VLOPT] Fix VCFCT incompatible EEW test (#122327)
https://github.com/llvm/llvm-project/pull/122151 added this test with an
invalid SEW. Use a valid SEW here.
2025-01-09 13:25:10 -05:00
Michael Maitland
550841f839
[RISCV][VLOPT] Add fp-reductions to getOperandInfo (#122151) 2025-01-09 09:43:26 -05:00
Michael Maitland
f77a7dd875
[RISCV][VLOPT] Add getOperandInfo for integer and floating point widening reductions (#122176) 2025-01-09 09:35:06 -05:00
Pengcheng Wang
b4e17d4a31
[MachineLICM] Use RegisterClassInfo::getRegPressureSetLimit (#119826)
`RegisterClassInfo::getRegPressureSetLimit` is a wrapper of
`TargetRegisterInfo::getRegPressureSetLimit` with some logics to
adjust the limit by removing reserved registers.

It seems that we shouldn't use
`TargetRegisterInfo::getRegPressureSetLimit`
directly, just like the comment "This limit must be adjusted
dynamically for reserved registers" said.

Separate from https://github.com/llvm/llvm-project/pull/118787
2025-01-09 21:05:52 +08:00
Luke Lau
c8ee1164bd
[RISCV] Fix masked->unmasked peephole handling masked pseudos with no passthru (#122253)
Some masked pseudos like PseudoVCPOP_M_B8_MASK don't have a passthru,
but in the masked->unmasked peephole we assumed the masked pseudo always
had one.

This checks for a passthru first and fixes #122245.
2025-01-09 19:54:37 +08:00
Craig Topper
b0f11dfc75
[RISCV] Add call preserved regmask to tail calls. (#122181)
Every call should have regmask operand to indicate what registers are
preserved or clobbered by the call. VirtRegRewriter uses this to tell
MachineRegisterInfo what registers are clobbered by a function. If the
mask isn't present the registers potentially clobbered by a tail called
function aren't counted. I have checked ARM, AArch64, and X86 and they
all have a regmask operand on their tail calls.

I believe this fixes an issue I'm seeing with IPRA.
2025-01-08 16:19:31 -08:00
Michael Maitland
e93181bf13
[RISCV][VLOPT] Add vector fp-conversion instruction to isSupportedInstr (#122033)
When these instructions are marked nofpexcept, we can optimize them.
There are some added toggles in the output, likley because other
noexcept fp instructions are not part of isSupportedInstr yet. We may
want to avoid marking an instruction as isSupported in the future if any
of its FP users are missing nofpexcept to avoid added toggles. However,
we seem to get some GPRs back as a result of this change, which may
outweigh the cost of avoiding extra toggles.

The plan is to follow this patch up with added support for more FP
instructions in the same way. The instructions in this patch are a
natural starting point because they allow us to test with integer
instructions which have good support already.
2025-01-08 13:30:40 -05:00
Craig Topper
9020d193e6 [RISCV][GISel] Re-generate combine-neg-abs.ll to fix build bot failure. NFC
Test was added while #121587 was in review.
2025-01-07 19:56:56 -08:00
Michael Maitland
b253a80f54
[RISCV][VLOPT] Add mask load to isSupported and getOperandInfo (#122030)
Add mask store to getOperandInfo since it has the same behavior.
2025-01-07 22:07:57 -05:00
Luke Quinn
dde5546b79
[RISCV] GISel custom lowering for G_ADD/G_SUB (#121587)
Custom lowering for s32 G_ADD/SUB to help match selection dag better.
Specifically for RV64 a s32 is produced as a add+sext the output this
allows for fewer instructions to sign extend a couple patterns. Allows
for the generation of addiw,subw,negw to reduce required instructions to
load values quicker

Log2_ceil_i32 in rvzbb.ll shows a more obvious improvement case.
2025-01-07 18:53:10 -08:00
Michael Maitland
142787d368
[RISCV][VLOPT] Add support for checkUsers when UserMI is a Single-Width Integer Reduction (#120345)
Reductions are weird because for some operands, they are vector
registers but only read the first lane. For these operands, we do not
need to check to make sure the EEW and EMUL ratios match. The EEWs,
however, do need to match.
2025-01-07 17:56:07 -05:00
Michael Maitland
36e4176f1d
[RISCV][VLOPT] Add strided, unit strided, and indexed loads to isSupported (#121705)
Add to getOperandInfo too since that is needed to reduce the VL.
2025-01-07 17:45:06 -05:00
Mikhail R. Gadelha
51c9c823ca
[RISCV] Added test case for PR119527. NFC (#121816)
Co-authored-by: Luke Lau <luke_lau@icloud.com>
2025-01-07 13:33:22 -03:00
Craig Topper
785b16ad04
[RISCV][GISel] Support G_MERGE_VALUES/G_UNMERGE_VALUES with Zfa. (#120379)
Without Zfa we use pseudos that are lowered to a stack load/store. With
Zfa we have instructions that can move a pair of registers to an FPR. Or
move the high or low half of an FPR to a GPR.

I've used a GINodeEquiv to make use of 3 of the 4 tablegen patterns. The
split case with Zfa requires 2 instructions which I'm doing through
custom isel like we do in SelectionDAG.
2025-01-07 07:50:50 -08:00
quic_hchandel
737d6ca44d
[RISCV] Add Qualcomm uC Xqcicm (Conditional Move) extension (#121752)
The Qualcomm uC Xqcicm extension adds 13 conditional move instructions.

The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest

This patch adds assembler only support.

---------

Co-authored-by: Harsh Chandel <hchandel@qti.qualcomm.com>
2025-01-07 08:25:00 +05:30
Craig Topper
c8d435f9af
[RISCV] Use ISD::XOR instead of RISCVISD::VMXOR_VL in lowerVectorMaskVecReduction of scalable ISD::VECREDUCE_AND (#121812)
This allows combining the XOR with earlier ISD::ANDs inserted by type
legalization.
2025-01-06 14:26:21 -08:00
Luke Lau
b359c84f3a
[RISCV] Don't commute with shift if it would break sh{1,2,3}add pattern (#119527)
This fixes a regression from #101294 by checking if we might be
clobbering a sh{1,2,3}add pattern.

Only do this is the underlying add isn't going to be folded away into an
address offset.
2025-01-06 19:23:21 +08:00
Michael Maitland
b48e5f0ff3
[RISCV][VLOPT] Add Vector FP instructions to getOperandInfo (#121609)
Although we cannot reduce the VL of these instructions (i.e. add to
isSupported) we can add them to getOperandInfo to enable optimization
where the FP vector instruction are users. Most of the instructions are
covered by existing tests, and I added tests for the narrowing
conversions because I was a little unsure whether the dest or the source
was 2*SEW and 2*LMUL.
2025-01-05 11:19:08 -05:00
Craig Topper
a4e47586b9
[ExpandMemCmp] Recognize canonical form of (icmp sle/sge X, 0) in getMemCmpOneBlock. (#121540)
This code recognizes special cases where the result of memcmp is
compared with 0. If the compare is sle/sge, then InstCombine
canonicalizes to (icmp slt X, 1) or (icmp sgt X, -1). We should
recognize those patterns too.
2025-01-03 10:23:13 -08:00
Alexey Bataev
6d321530af [CG][RISCV]Add more RVV tests with exact vlen and linear/quadratic number of shuffles 2025-01-03 10:06:21 -08:00
Craig Topper
bad0f98bda [ExpandMemCmp][AArch][RISCV][X86] Pre-commit tests for recognizing canonical form of (icmp sle/sge X, 0). NFC
Pre-commit for #121540.
2025-01-03 10:02:42 -08:00
Craig Topper
13cf5c9c22 [RISCV] Re-generate memcmp test checks missed in #121530. NFC
A patch landed to these tests while #121530 was in review and I
forgot to rebase.
2025-01-03 09:48:41 -08:00
Craig Topper
4dfea22e77
[ExpandMemCmp][AArch64][PowerPC][RISCV][X86] Use llvm.ucmp instead of (sub (zext (icmp ugt)), (zext (icmp ult))). (#121530)
AArch64 and PowerPC look like a improvements.
RISC-V is neutral.
X86 trades a dependency breaking xor before a seta for a movsx after a
sbbb. Depending on how the result is used, this movsx might go away.
2025-01-03 09:19:32 -08:00
Shao-Ce SUN
2fae5bdea7
[RISCV] Add support of Sdext,Sdtrig extentions (#120936)
`Sdext` and `Sdtrig` are RISC-V extensions related to debugging.

The full specification can be found at

https://github.com/riscv/riscv-debug-spec/releases/download/1.0.0-rc4/riscv-debug-specification.pdf
2025-01-03 17:25:42 +08:00
Pengcheng Wang
72db3f989e
[RISCV] Allow tail memcmp expansion (#121460)
This optimization was introduced by #70469.

Like AArch64, we allow tail expansions for 3 on RV32 and 3/5/6
on RV64.

This can simplify the comparison and reduce the number of blocks.
2025-01-03 14:05:02 +08:00
Sudharsan Veeravalli
532a2691bc
[RISCV] Add Qualcomm uC Xqcicli (Conditional Load Immediate) extension (#121292)
This extension adds 12 instructions that conditionally load an immediate
value.

The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest

This patch adds assembler only support.
2025-01-03 06:33:27 +05:30
Min-Yih Hsu
3cac26f541
[GISel] Combine (neg (min/max x, (neg x))) into (max/min x, (neg x)) (#120998)
This is the GISel version of #120666. Also supports both unsigned and
signed version of min & max.
2025-01-02 16:29:34 -08:00
Min-Yih Hsu
2291d0aba9
[DAGCombiner] Turn (neg (max x, (neg x))) into (min x, (neg x)) (#120666)
This pattern was originally spotted in 429.mcf by @topperc.

We already have a DAGCombiner pattern to turn `(neg (abs x))` into `(min
x, (neg x))`. But in some cases `(neg (max x, (neg x)))` is formed by an
expanded `abs` followed by a `neg` that is generated only after the
`abs` expansion. This patch adds a separate pattern to match cases like
this, as well as its inverse pattern: `(neg (min X, (neg X))) --> (max
X, (neg X))`.

This pattern is applicable to both signed and unsigned min/max.
2025-01-02 16:28:55 -08:00
Philip Reames
6840521739 Revert "[RISCV][CG]Use processShuffleMasks for per-register shuffles"
This reverts commit b8952d4b1b0c73bf39d6440ad3166a088ced563f.

spec x264 fails to build in all VLS configurations, with the assertion
failure: clang: ../llvm-project/llvm/lib/Target/RISCV/RISCVISelLowering.cpp:5246: llvm::SDValue lowerShuffleViaVRegSplitting(llvm::ShuffleVectorSDNode*, llvm::SelectionDAG&, const llvm::RISCVSubtarget&): Assertion `RegCnt == NumOfDestRegs && "Whole vector must be processed"' failed.

I can reduce a failing piece of IR, but the failure appears pretty
broad, so I suspect any reasonable vls build will hit it.
2025-01-01 10:53:24 -08:00
Philipp van Kempen
f590963db8
[RISCV] Implement RISCVTTIImpl::getPreferredAddressingMode for HasVendorXCVmem (#120533)
For a simple matmult kernel this heuristic reduces the length of the
critical basic block from 15 to 20 instructions, resulting in a 20%
speedup.

**Without heuristic:**

```
       13688: 001b838b      cv.lb   t2, (s7), 0x1
       1368c: 09cdbcab      cv.lb   s9, t3(s11)
       13690: 089db62b      cv.lb   a2, s1(s11)
       13694: 092dbdab      cv.lb   s11, s2(s11)
       13698: 001d028b      cv.lb   t0, (s10), 0x1
       1369c: 00f282b3      add     t0, t0, a5
       136a0: 9072b52b      cv.mac  a0, t0, t2
       136a4: 9192bfab      cv.mac  t6, t0, s9
       136a8: 90c2beab      cv.mac  t4, t0, a2
       136ac: 91b2bf2b      cv.mac  t5, t0, s11
       136b0: fffc0c13      addi    s8, s8, -0x1
       136b4: 018e0633      add     a2, t3, s8
       136b8: 91b2b0ab      cv.mac  ra, t0, s11
       136bc: 000b8d93      mv      s11, s7
       136c0: fc0614e3      bnez    a2, 0x13688 <muriscv_nn_vec_mat_mult_t_s8+0x2f0>

       #instrs = 15
```

**With heuristic:**

```
        7bc0: 001c860b      cv.lb   a2, (s9), 0x1
        7bc4: 001e0d0b      cv.lb   s10, (t3), 0x1
        7bc8: 001e808b      cv.lb   ra, (t4), 0x1
        7bcc: 0015038b      cv.lb   t2, (a0), 0x1
        7bd0: 001c028b      cv.lb   t0, (s8), 0x1
        7bd4: 00f282b3      add     t0, t0, a5
        7bd8: 90c2bfab      cv.mac  t6, t0, a2
        7bdc: 91a2b92b      cv.mac  s2, t0, s10
        7be0: 9012b5ab      cv.mac  a1, t0, ra
        7be4: 9072b9ab      cv.mac  s3, t0, t2
        7be8: 9072b72b      cv.mac  a4, t0, t2
        7bec: fc851ae3      bne     a0, s0, 0x7bc0 <muriscv_nn_vec_mat_mult_t_s8+0x338>

        #instrs = 12

        improvement = 1 - 12/15 = 0.2 = 20%
```
2024-12-31 10:56:28 +08:00
Craig Topper
c557ce9f27 [RISCV] Use add_like_non_imm12 in XTheadba patterns to match Zba. 2024-12-29 12:39:37 -08:00
quic_hchandel
1557eeda73
[RISCV] Add Qualcomm uC Xqciac (Load-Store Adress calculation) extension (#121037)
This extension adds 3 instructions that perform load-store address
calculation.

The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest

This patch adds assembler only support.

---------

Co-authored-by: Harsh Chandel <hchandel@qti.qualcomm.com>
Co-authored-by: Sudharsan Veeravalli <quic_svs@quicinc.com>
2024-12-29 11:14:12 +05:30
Craig Topper
814902a03a [RISCV] Fix XTheadba patterns broken since cfc574a6cd13d2d0b77110b579c5cfcec744129f.
Adding an OperandTransform to CSImm12MulBy4 and CSImm12MulBy8 for
Zba broke these patterns. They should have been changed in the same,
but we lacked sufficient testing.
2024-12-26 21:50:35 -08:00