This extension adds 3 instructions that perform load-store address
calculation.
The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest
This patch adds assembler only support.
---------
Co-authored-by: Harsh Chandel <hchandel@qti.qualcomm.com>
Co-authored-by: Sudharsan Veeravalli <quic_svs@quicinc.com>
Adding an OperandTransform to CSImm12MulBy4 and CSImm12MulBy8 for
Zba broke these patterns. They should have been changed in the same,
but we lacked sufficient testing.
Patch adds usage of processShuffleMasks in in codegen
in lowerShuffleViaVRegSplitting. This function is already used for X86
shuffles estimations and in DAGTypeLegalizer::SplitVecRes_VECTOR_SHUFFLE
functions, unifies the code.
Reviewers: preames, topperc, lukel97, wangpc-pp
Reviewed By: wangpc-pp
Pull Request: https://github.com/llvm/llvm-project/pull/120803
Check if (add r, imm) can be optimized to (ADDI (ADDI r, imm0), imm1),
in which imm = imm0 + imml and both imm0 and imm1 are simm12. We make
imm0 as large as possible and imm1 as small as possible so that we might
be able to use c.addi for the small immediate.
SEWGreaterThanOrEqualAndLessThan64 is a stricter constraint so it should
have a higher value than SEWGreaterThanOrEqual.
Found by our random test generator.
Prior to this patch, we required that all users had the same VL in order
to optimize. But as the FIXME said, we can use the largest VL to
optimize, as long as we can determine what the largest is. This patch
implements the FIXME.
This patch adds basic support of `MachinePipeliner` and disable
it by default.
The functionality should be OK and all llvm-test-suite tests have
passed.
The default legalization uses vmslt with a vector of XLen to compute a
mask. This doesn't work if the type isn't legal. For fixed vectors it
will scalarize. For scalable vectors it crashes the compiler.
This patch uses an alternate strategy that promotes the i1 vector to an
i8 vector and does the merge. I don't claim this to be the best
lowering. I wrote it quickly almost 3 years ago when a crash was
reported in our downstream.
Fixes#120405.
Scalarize vector FPOWI instead of promoting the type. This allows the
scalar FPOWIs to be visited and converted to libcalls before promoting
the type.
FIXME: This should be done in LegalizeVectorOps/LegalizeDAG, but call
lowering needs the unpromoted EVT.
Without this patch, in some backends, such as RISCV64 and LoongArch64,
the i32 type is illegal and will be promoted. This causes exponent type
check to fail when ISD::FPOWI node generates a libcall.
Fix https://github.com/llvm/llvm-project/issues/118079
Use `MO.getOperandNo() == 0` instead of `IsMODef` so naming is clear for the store, since the store should treat its operand 0 like that even though it is not a def.The load should treat its operand 0 def in the same way.
Now that we have testing of all instructions in the isSupportedInstr
switch, and better coverage of getOperandInfo, I think it is a good time
to enable this by default.
This is a follow up to 2af2634 to use vmv.v.x of i8 constants instead of
the prior vid/vand/vmsne sequence. The advantage of the vmv.v.x sequence
is that it's always m1 (so cheaper at high LMUL), and can be
rematerialized by the register allocator if needed to locally reduce
register pressure.
I found this crash when trying to enable the VLOptimizer pass. We need
this patch before we can enable by default. The old assert was not
checking that USE and DEF were vector registers. The correct condition
is guarded at the callsite of tryReduceVL.
This adds a new helper `canFoldStoreIntoLibCallOutputPointers()` to
check that it is safe to fold a store into a node that will expand to a
library call that takes output pointers. This requires checking for two
(independent) properties:
1. The store is not within a CALLSEQ_START..CALLSEQ_END pair
* If it is, the expansion would lead to nested call sequences (which is
invalid)
2. The node does not appear as a predecessor to the store
* If it does, attempting to merge the store into the call would result
in a cycle in the DAG
These two properties are checked as part of the same traversal in
`canFoldStoreIntoLibCallOutputPointers()`
Lowering to load-acquire/store-release for RISCV Zalasr.
Currently uses the psABI lowerings for WMO load-acquire/store-release
(which are identical to A.7). These are incompatable with the A.6
lowerings currently used by LLVM. This should be OK for now since Zalasr
is behind the enable experimental extensions flag, but needs to be fixed
before it is removed from that.
For TSO, it uses the standard Ztso mappings except for lowering seq_cst
loads/store to load-acquire/store-release, I had Andrea review that.
Convert the LLT to EVT and call
TargetLowering::shouldExpandCmpUsingSelects to determine if we should do
this.
We don't have a getSetccResultType, so I'm boolean extending the
compares to the result type and using that. If the compares legalize to
the same type, these extends will get removed. Unfortunately, if the
compares legalize to a different type, we end up with truncates or
extends that might not be optimally placed.
Mask only instructions like vmand and vmsbf should always have 0 for
their Log2SEW operand. Non-mask instructions should only have
3, 4, 5, or 6 for their Log2SEW operand.
Split the operand type so we can verify these cases separately.
I had to fix the SEW for whole register move to vmv.v.v copy
optimization and update an mir test. The vmv.v.v change isn't
functional since we have already done vsetvli insertion before and
nothing else uses the field after copy expansion. I can split these
changes off if desired.
These instructions should always be created with Log2SEW=0 and an LMUL
based on SEW=8. This is used by the vsetvli pass to know these
instructions only care about the ratio and not the specific value.
Looks like I fixed riscv_vmsge(u)_mask intrinsics years ago, but forgot
the unmasked intrinsics.
I'm working on an enhancement to our MachineVerifier checks that will
require VMNAND and VMSET to have Log2SEW=0.
Replace LMUL suffixes with _B1, _B2, etc. This matches what we do
for other mask only instructions like VCPOP_M, VFIRST_M, VMSBF_M,
VLM, VSM, etc.
Now all pseudoinstructions that use Log2SEW=0 will be consistently
named.
These are suffixed with B1, B2, B4, B8, B16, B32, or B64 which I think
these were supposed to match the naming of the vbool types from C where
the number should be SEW/LMUL. So the smallest mask is 64 and the
largest is 1. This provides a compact syntax for describing the 7
possible ratios between LMUL and SEW.
We had the instruction names in the opposite order.
This allows us to support i128 G_ICMP on RV32. I'm not sure how to test
the "left over" part of this as RISC-V always widens to a power of 2
before narrowing.
We need to adjust getEMULEqualsEEWDivSEWTimesLMUL to account for the
fact that Log2EEW for mask instructions is 0 but their EMUL is
calculated using Log2EEW=3.
Masking out most significant bits can be done with shl followed by srl
with same shift amount. If this is followed by a shl, we could instead
srl by a smaller amount of bits.
This transform is already implemented in tablegen for masking out
32 most significant bits.
Emits better code for e.g.
float *index(float *p, int i)
{
return p + (i & (1 << 30) - 1);
}
The Qualcomm uC Xqcics extension adds 8 conditional select instructions.
The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/latest
This patch adds assembler only support.
---------
Co-authored-by: Harsh Chandel <hchandel@qti.qualcomm.com>
In x264, there's a few kernels with shuffles like this:
%41 = add nsw <16 x i32> %39, %40
%42 = sub nsw <16 x i32> %39, %40
%43 = shufflevector <16 x i32> %41, <16 x i32> %42, <16 x i32> <i32 11,
i32 15, i32 7, i32 3, i32 26, i32 30, i32 22, i32 18, i32 9, i32 13, i32
5, i32 1, i32 24, i32 28, i32 20, i32 16>
Because this is a complex two-source shuffle, this will get lowered as
two vrgather.vvs that are blended together.
vadd.vv v20, v16, v12
vsub.vv v12, v16, v12
vrgatherei16.vv v24, v20, v10
vrgatherei16.vv v24, v12, v16, v0.t
However the indices coming from each source are disjoint, so we can
blend the two together and perform a single source shuffle instead:
%41 = add nsw <16 x i32> %39, %40
%42 = sub nsw <16 x i32> %39, %40
%43 = select <0,0,0,0,1,1,1,1,0,0,0,0,1,1,1,1> %41, %42
%44 = shufflevector <16 x i32> %43, <16 x i32> poison, <16 x i32> <i32
11, i32 15, i32 7, i32 3, i32 10, i32 14, i32 6, i32 2, i32 9, i32 13,
i32 5, i32 1, i32 8, i32 12, i32 4, i32 0>
The select will likely get merged into the preceding instruction, and
then we only have to do one vrgather.vv:
vadd.vv v20, v16, v12
vsub.vv v20, v16, v12, v0.t
vrgatherei16.vv v24, v20, v10
This patch bails if either of the sources are a broadcast/splat/identity
shuffle, since that will usually already have some sort of cheaper
lowering.
This improves performance on 525.x264_r by 4.12% with -O3 -flto
-march=rva22u64_v on the spacemit-x60.
For pre-ra scheduling, we use two options `-misched-topdown` and
`-misched-bottomup` to force the direction.
While for post-ra scheduling, we use `-misched-postra-direction`
with enumerated values (`topdown`, `bottomup` and `bidirectional`).
This is not unified and adds some mental burdens. Here we replace
these two options `-misched-topdown` and `-misched-bottomup` with
`-misched-prera-direction` with the same enumerated values.
To avoid the condition of `getNumOccurrences() > 0`, we add a new
enum value `Unspecified` and make it the default initial value.
These options are hidden, so we needn't keep the compatibility.
This follows up #115495 by enabling merging of external globals by
default, which had been left as a next step in order to make the
previous change more incremental and so we can more easily narrow down
on any identified regressions.
Enabling merging of external globals matches what Arm does (for non
mach-o targets), though AArch64 doesn't as there were [some
concerns](https://reviews.llvm.org/D61947) it might cause regressions in
some cases.
See https://github.com/llvm/llvm-project/pull/117880 for benchmark figures and discussion.
I don't include getOperandInfo for the loads, since they don't take a
vector use operand, and we don't include the loads in isSupportedInstr
so we will never call getOperandInfo on the vector destination of these
instructions.
Don't add support for VSM since we don't have any mask producing
instructions in isSupportedInstr at the moment.
This pull request modifies the behavior of the
`@llvm.experimental.stackmap` intrinsic to require that its two first
operands (`id` and `numShadowBytes`) be **immediate values**. This
change ensures that variables cannot be passed as two first arguments to
this intrinsic.
Related Issue: https://github.com/llvm/llvm-project/issues/115733
### Testing
- Added new test cases to ensure errors are emitted for non-immediate
operands.
- Ran the full LLVM test suite to verify no regressions were introduced.
Previously we created an FP_TO_FP16 and legalized it in
SoftenFloatOp_FP_ROUND. This caused i16 to be sent to call lowering
instead of f16. This results in the ABI not being followed if f16 is
supposed to be passed in a different register than i16.
Looking at the libgcc binary for the library function it appears the value
is returned in xmm0 so the X86 test was being miscompiled before.
Fixes#107607.
