Previously, llvm IR is hard to create a scalable vector splat with a
specific vector length, so we use riscv.vmv.v.x and riscv.vmv.v.f to do
this work. But the two rvv intrinsics needs strict type constraint which
can not support fixed vector types and illegal vector types. Using
vp.splat could preserve old functionality and also generate more
optimized code for vector types and illegal vectors.
This patch also fixes crash for getEVT not serving ptr types.
A splat of a <n x i64> on RV32 will get lowered as a zero strided load
anyway (and won't match any .vx splat patterns), so don't expand it to a
scalar load + splat to avoid writing it to the stack.
This patch makes zero strided VP loads always be expanded to a scalar
load and splat even if +optimized-zero-stride-load is present.
Expanding it allows more .vx splat patterns to be matched, which is
needed to prevent regressions in #98111.
If the feature is present, RISCVISelDAGToDAG will combine it back to a
zero strided load.
The RV32 test diff also shows how need to emit a zero strided load
either way after expanding an SEW=64 strided load. We could maybe fix
this in a later patch by not doing the expand if SEW>XLEN.
This is a recommit of #98140. The old commit should be rebased on #98205
which changes the feature of hardware zero stride optimization.
It's a similar patch as a214c521f8763b36dd400b89017f74ad5ae4b6c7 for
vp.stride.load. Some targets prefer pattern (vmv.v.x (load)) instead of
vlse with zero stride.
It's a similar patch as a214c521f8763b36dd400b89017f74ad5ae4b6c7 for
vp.stride.load. Some targets prefer pattern (vmv.v.x (load)) instead of
vlse with zero stride.
It's IR version of #97798.
We want to know the upper 33 bits of the And Input are zero. SExt
only guarantees they are the same.
We originally checked for SExt or ZExt when we were using
isImpliedByDomCondition because a ZExt may have been changed to SExt
before we visited the And.
We are no longer using isImpliedByDomCondition so we can only look
for zext with the nneg flag.
While here, switch to PatternMatch to simplify the code.
Fixes#78783
LLVM vector reduction intrinsics return a scalar result, but on RISC-V
vector reduction instructions write the result in the first element of a
vector register. So when a reduction in a loop uses a scalar phi, we end
up with unnecessary scalar moves:
loop:
vfmv.s.f v10, fa0
vfredosum.vs v8, v8, v10
vfmv.f.s fa0, v8
This mainly affects ordered fadd reductions, which has a scalar accumulator
operand.
This tries to vectorize any scalar phis that feed into a fadd reduction
in RISCVCodeGenPrepare, converting:
loop:
%phi = phi <float> [ ..., %entry ], [ %acc, %loop]
%acc = call float @llvm.vector.reduce.fadd.nxv4f32(float %phi, <vscale x 2 x float> %vec)
```
to
loop:
%phi = phi <vscale x 2 x float> [ ..., %entry ], [ %acc.vec, %loop]
%phi.scalar = extractelement <vscale x 2 x float> %phi, i64 0
%acc = call float @llvm.vector.reduce.fadd.nxv4f32(float %x, <vscale x 2 x float> %vec)
%acc.vec = insertelement <vscale x 2 x float> poison, float %acc.next, i64 0
Which eliminates the scalar -> vector -> scalar crossing during
instruction selection.
This stacks on #70725. Once we have lowering for zext nneg, we can
rewrite all of the existing RISCVCodeGenPrepare login in terms of zext
nneg instead of sext. The change isn't NFC from the perspective of the
individual pass, but should be from the perspective of codegen as a
whole.
As noted in the TODO, one piece can be moved to instcombine, but I'll
leave that to a separate commit.
(abs(i32 X, i1 1) always produces a positive result. The 'i1 1'
means INT_MIN input produces poison. If the result is sign extended,
InstCombine will convert it to zext. This does not produce ideal
code for RISCV.
This patch reverses the zext back to sext which can be folded
into a subw or negw. Ideally we'd do this in SelectionDAG, but
we lose the INT_MIN poison flag when llvm.abs becomes ISD::ABS.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D130412
If X is known positive by a dominating condition, we can fill in
ones into the upper bits of C1 if that would allow it to become an
simm12 allowing the use of ANDI.
This pattern often occurs in unrolled loops where the induction
variable has been widened.
To get the best benefit from this, I had to move the pass above
ConstantHoisting which is in addIRPasses. Otherwise the AND constant
is often hoisted away from the AND.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D129888
We're creating single instruction to replace another instruction.
We can insert using the InsertBefore operand of the constructor.
Then copy the debug location.
Initial optimization is to convert (i64 (zext (i32 X))) to
(i64 (sext (i32 X))) if the dominating condition for the basic block
guaranteed the sign bit of X is zero.
This frequently occurs in loop preheaders where a signed induction
variable that can never be negative has been widened. There will be
a dominating check that the 32-bit trip count isn't negative or zero.
The check here is not restricted to that specific case though.
A i32->i64 sext is cheaper than zext on RV64 without the Zba
extension. Later optimizations can often remove the sext from the
preheader basic block because the dominating block also needs a sext to
evaluate the greater than 0 check.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D129732
