When comparing multi-word integers with Zicond, we generate:
(or (czero_eqz (lo1 < lo2), (hi1 == hi2)),
(czero_nez (hi1 < hi2), (hi1 == hi2)))
The czero_nez is redundant because when hi1 == hi2 is true, hi1 < hi2 is
already 0. This patch adds a DAG combine to recognize:
czero_nez (setcc X, Y, CC), (setcc X, Y, eq) -> (setcc X, Y, CC)
when CC is a strict inequality (lt, gt, ult, ugt).
This saves one instruction in 128-bit comparisons on RV64 with Zicond.
Note the czero_nez becomes a czero.eqz in the final assembly because the
seteq is replaced by an xor that produces 0 when the values are equal.
Part of #179584
Assisted-by: claude
Compressing to a single shuffle doesn't remove any information and the backend can better apply specific optimizations to a single shuffle.
Addresses #176218.
---------
Co-authored-by: Luke Lau <luke_lau@igalia.com>
Extend the existing combineADDDToWMACC DAG combine to also match
RISCVISD::WMULSU and produce RISCVISD::WMACCSU. This is similar to
how ADDD+UMUL_LOHI is combined to WMACCU and ADDD+SMUL_LOHI is
combined to WMACC.
This patch was generated by AI, but I reviewed it.
We add pseudos/patterns for `vabs.v` instruction and handle the
lowering in `RISCVTargetLowering::lowerABS`.
Reviewers: topperc, 4vtomat, mshockwave, preames, lukel97, tclin914
Reviewed By: mshockwave
Pull Request: https://github.com/llvm/llvm-project/pull/180142
We directly lower `ISD::ABDS`/`ISD::ABDU` to `Zvabd` instructions.
Note that we only support SEW=8/16 for `vabd.vv`/`vabdu.vv`.
Reviewers: mshockwave, lukel97, topperc, preames, tclin914, 4vtomat
Reviewed By: lukel97, topperc
Pull Request: https://github.com/llvm/llvm-project/pull/180141
Combine the pattern:
ADDD(addlo, addhi, UMUL_LOHI(x, y).0, UMUL_LOHI(x, y).1)
into:
WMACCU(x, y, addlo, addhi)
And similarly for SMUL_LOHI -> WMACC.
This patch was written with AI, but I reviewed it carefully.
We already do all the checks necessary in order to prioritize
MULHU/MULHS/UMUL_LOHI/SMUL_LOHI over MULHSU/WMULSU. We might as
well just emit the nodes instead of letting generic type legalization
redo the checks.
This is slightly different than the default legalization because we
don't have access to ExpandInteger so we have to emit TRUNCATES and
BUILD_PAIR. Not sure if this will result in any differences in practice.
Add custom lowering for INSERT_VECTOR_ELT on P extension vector types
using the MVM instruction.
TODO: Handle <4 x i8> on RV64 which is constructed to extract_vector_elt
+ build_vector instead of insert_vector_elt.
Order the operands so the the low and high part of the rs1 regpair are
first, followed by the low and high part of the rs2 regpair.
Also change the type to use v4i8 for the result so that it's only
shuffling elements not combining elements into a larger elment.
I'm planning to add ADDD and SUBD opcodes that will be defined with the
same operand order allowing RISCVISelDAGToDAG.cpp code to be shared.
There's a good chance we'll want to use these for scalar too.
Drop vector type from SDTypeProfile. Remove PMULHSU since we already
have RISCVISD::MULHSU for scalars in the base ISA.
If the source of an fpto*i doesn't fit in the destination type, the
result is poison. For i1 destinations, this means the result needs to be
0 or 1/-1, so we can just compare the result to 0 directly instead of
truncating.
The VP lowering for fpto*i already does this.
XOR should be OR to match the comment.
Found while reviewing #179622 which deletes this function. I would like
to commit this first so we have a correct baseline for reviewing that
patch.
This is part of the work to remove trivial VP intrinsics.
In the combineVectorSizedSetCCEquality combine, used for the compares
that ExpandMemcmp generates, we currently emit a VP_SETCC. We can just
emit a regular SETCC and let RISCVVLOptimizer take care of reducing the
VL.
This commit enables code generation for RISC-V targeting Mach-O:
- Implement RISCVMachOTargetObjectFile::getNameWithPrefix method to
handle Mach-O symbol naming requirements.
- Use shouldAssumeDSOLocal() in RISCVTargetLowering::lowerGlobalAddress
instead of isDSOLocal() for proper Mach-O semantics in global address
lowering. Note that this is a NFC for RISCV when targeting ELF.
- Add comprehensive tests for various relocation types (direct globals,
GOT-based addressing, static vs PIC models).
- Test function calls, tail calls, and various symbol reference patterns
including addends and subtractions.
This patch is based on code originally written by Tim Northover.
We also get some i32->i64 promotion for CLMULH. The DAGCombiner
change is to prevent an infinite loop from that.
Test file was rewritten to cover all types and split between clmul
and clmulh.
I added a couple masked tests to show that VectorPeephole works.
The test outputs were already large so I didn't want to add more than a couple.
This reverts commit e3156c531da5aa4ec604605ed4e19638879d773c.
We need to resolve a crash on trunk and LLVM 22. Reverting makes it
easier to backport.
Fixes#176637.
I did not replace riscv.clmulh/clmulr since those require a multiple
instruction pattern match. I wanted to ensure that -O0 will select the
correct instructions without relying on combines.
This patch does the minimum to remove RISCVISD::CLMUL*. It does not
remove existing intrinsics.
There's some missed optimizations for i32 CLMULH/CLMULR on RV64, but
those may be generic issues.
I've put the test cases in the existing files so it's more obvious what
the missed optimizations are by comparing within the file.
This prevents getCommonSubClass from finding them before VR/VRNoV0.
Fixes a crash reported post-commit in #171231. getCommonSubClass
returned one of these classes, but it doesn't have the same VTs as
VR/VRNoV0 leading to an assertion failure.
The subregister-undef-early-clobber.mir still ends up finding these
register classes in the InitUndef pass.
For sub-register width vectors (v2i16, v4i8) on RV64 with P-extension,
the type legalizer widens them to legal types, i.e. v4i16, v8i8, before
they're getting unrolled, so they'll be redundant computation for higher
part of register.
The correct way to handle is similar to widening div/rem where there's
undef padded for high part.
stack on: https://github.com/llvm/llvm-project/pull/176093
P extension packed SIMD types are passed in GPRs. For types larger than
XLen (e.g. v8i8 on RV32), they are split and passed via the 2XLen
mechanism, similar to i64 on RV32.
FIXME: Need to figure out the mechanism when P and V are enabled at the
same time.
stack on: https://github.com/llvm/llvm-project/pull/176193
Splits out change from https://github.com/llvm/llvm-project/pull/176015
Changes shouldExpandAtomicRMWInIR to take a constant argument: This is
to allow some other TargetLowering constant-argument functions to call
it. This change touches several backends. An alternative solution
exists, but to me, this seems the "right" way.
fa57074d146925a303263905af415cc78f58f353 constraint the RISCV BitInt
with 128 bits.
It is due to fp <-> int convension will crash in backend.
(https://godbolt.org/z/9o1qr4rje)
This patch enable larger than 128 bits BitInt type by
`setMaxLargeFPConvertBitWidthSupported`.
The test case added in this patch crashes on rv32v without this fix. We
attempt to trunc the i32 type of the select produced by lowerCttzElts to
i64, which asserts. Use getZExtOrTrunc instead.
Result type of P extension's comparison instructions is same as operands
and the result bits are all 1s or 0s so we need to set
ZeroOrNegativeOneBooleanContent to make sext(setcc) auto combined.
The P extension requires us to use base ISA load/store instructions for
small vectors. We need to make sure we don't generate misaligned
instructions.
We'll need to do more work here if we want P and V to be enabled at the
same time, but that's a future problem.
Introduced in #170483 this was only ever meant to trigger for scalar
types. We get an error on rv32gcv for some inputs "Cannot implicitly
convert a scalable size to a fixed-width size in `TypeSize::operator
ScalarTy()`".
While we're editing this function, delete an accidentally repeated
comment.
Minimal reproducer:
```
define <vscale x 4 x ptr> @ham() {
bb:
%load = load <vscale x 4 x i32>, ptr null, align 4
%ashr = ashr <vscale x 4 x i32> %load, splat (i32 1)
%getelementptr = getelementptr i32, ptr null, <vscale x 4 x i32> %ashr
ret <vscale x 4 x ptr> %getelementptr
}
```
Added without test as the fix is trivial and the test is rather awkward/specialised. We will get coverage for this and similar issues from llvm-test-suite rv32gcv in CI.
We use fli+fneg to generate negative float, eliminate the fneg for fma.
Fold fma to vfnmsac.vf,vfnmsub.vf, vfnmacc.vf, vfnmadd.vf
---------
Co-authored-by: Craig Topper <craig.topper@sifive.com>
