The optimization of (mul x, c) to (ADD (SLLI x, i0), (SLLI x, i1))
is only enabled for i32 multiplication on rv64, because of
the regression in i64 multiplication on rv32.
However we can change the condition to that the immediate 'c'
should only be used once, then the above regression can also be
avoided, and ohter chances of optimization can be enabled.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147410
It seems that flang uses this for "nint" and expects this i32
to work. On the C side we think lround should only work for "long"
which is i64 on rv64.
It's easy for us to support i32 when we have native FP instructions.
I fell back to i64 and truncated the result otherwise. The
documentation for lround says it returns an unspecified value if
doesn't fit in the integer type. I have no idea what flang is
expecting. I really only did the libcall to avoid forking a test.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D147195
The result of sub + setcc is 0 or 1 for all bits.
The sra instruction get the same result.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147538
Loads of fixed length vectors with irregular element counts are
sometimes emitted as a scalar load + scalar_to_vector.
Previously the scalar_to_vector wasn't legal and so was scalarized
further. This patch handles it by lowering it to a vmv.s.x.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147608
If we're inserting a fixed length subvector into a fixed length vector,
then we can use a tail agnostic policy as long as we're inserting up to
or past the end of the main vector.
I.e., because we're overwriting all of the main vector's tail elements,
and we don't care what the elements after that are.
As noted by Philip in https://reviews.llvm.org/D146711#4220341
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147347
This constructs a proper memory operand for these intrinsics.
Segment load/store will be added in a separate patch.
Reviewed By: kito-cheng
Differential Revision: https://reviews.llvm.org/D147119
Also the patch loose the fixed vector contraint in llvm/lib/IR/Verifier.cpp.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147380
Compiler hits infinite loop in DAGCombine. For force-streaming-compatible-sve
mode we have custom lowering for 128-bit vector splats and later in
DAGCombiner::SimplifyVCastOp() we scalarized SPLAT because we have custom
lowering for SME. Later, we restored SPLAT opertion via performMulCombine().
This enables the interleaved access pass on O1 and above, and causes
interleaving/deinterleaving shuffles of fixed length vectors with
stores/loads to be lowered into vssegN/vlsegN.
We need to be careful and make sure that we only lower vsseg/vlseg
whenever we know the fixed vector type will fit within the minimum vlen,
and that the interleaving factor is supported for the given LMUL.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145085
This enables the interleaved access pass on O1 and above, and causes
interleaving/deinterleaving shuffles of fixed length vectors with
stores/loads to be lowered into vssegN/vlsegN.
We need to be careful and make sure that we only lower vsseg/vlseg
whenever we know the fixed vector type will fit within the minimum vlen,
and that the interleaving factor is supported for the given LMUL.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145085
Similar to the existing SelectionDAG::SplitVector helper, this helper creates the EXTRACT_ELEMENT nodes for the LO/HI halves of the scalar source.
Differential Revision: https://reviews.llvm.org/D147264
StartIndexes[0] Tells exactly which source element is in element 0,
the even source. Nothing needs to be swapped.
Since we're dealing with power of 2 vector lengths, StartIndexes[0]
is almost always even so the condition here was never true. The
exception is when we're interleaving two 1 element vectors. In that
case StartIndexes[0] could be 1.
We recently hit a failure from this on a pulldown. I don't have
the reduced reproducer yet and my naive attempts at making an
interleave of 1 element vectors produces a slideup instead so don't
go through this path.
Reviewed By: luke
Differential Revision: https://reviews.llvm.org/D147268
This directly matches the codegen for xventanacondops with vt.maskcn =>
czero.nez and vt.maskc => czero.eqz. An additional difference is that
zicond is available on RV32 in addition to RV64 (xventanacondops is RV64
only).
Differential Revision: https://reviews.llvm.org/D147147
The patch customized lower vector type ISD::STRICT_FP_ROUND to RISCVISD::STRICT_FP_ROUND.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147113
Preparation for adding the other RVV load/store intrinsics we use
for the C API.
Reviewed By: asb, kito-cheng
Differential Revision: https://reviews.llvm.org/D147004
As discussed earlier in the [GitHub
issue](https://github.com/llvm/llvm-project/issues/59500), currently
LLVM generates invalid code when emulated TLS is used. There were
attempts to resolve this previously (D102527), but they were not merged
because the component owners raised concerns about emulated TLS
efficiency.
The current state of the art is that:
- OpenBSD team, which raised the initial issue, simply has [patches
downstream](https://github.com/openbsd/src/blob/a0747c9/gnu/llvm/llvm/lib/Target/RISCV/RISCVISelLowering.cpp#L2850-L2852).
- Our team, which raised the GH issue, has patches downstream as well.
We also do not use `malloc` or any [dynamic
allocations](https://github.com/llvm/llvm-project/issues/59500#issuecomment-1349046835)
with emulated TLS, so the concerns raised in the original issue does not
apply to us.
- GCC compatibility is broken, because GCC supports emulated TLS.
- RISC-V is the only architecture in LLVM that does not support emulated
TLS, and work is being done to at least warn the users about it
(D143619).
With all these in mind I believe it is important to address the
consumers' needs especially given that there is little to no maintenance
downsides.
Differential Revision: https://reviews.llvm.org/D143708
Like D145900, the patch also supports fixed vector strict_fma nodes in RISC-V by
customized lowering them to riscv_strict_vfmadd_vl nodes. riscv_strict_vfmadd_vl
is created to avoid some riscv_vfmadd_vl optimizations happening to original
strict_fma nodes. The patch also adds combine patterns for riscv_strict_fmadd_vl
nodes with negation operands.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D146939
A shuffle with an insert subvector mask is functionally equivalent to:
(insert_subvector v0, (extract_subvector v1, len), index)
We can emulate by doing a vslideup on v1 into the right index, and
carefully selecting VL so that we don't overwrite any more destination
elements than what we have to.
This avoids the need for a select with a mask.
We custom isel for ConstantFP that has higher priority than isel
patterns. We were previously detecting valid FP constants for fli
to early exit from the custom code. This detection called
getLoadFPImm. Then we would run the isel patterns which would call
getLoadFPImm a second time.
With a little bit more code we can directly select the fli instruction
in the custom handler and avoid a second call.
Remove the incorrect mayRaiseFPException flag from the FLI instructions.
Reviewed By: joshua-arch1
Differential Revision: https://reviews.llvm.org/D146093
We've a argument lowering logic to prevent floating-point value pass
passed with bit-conversion, but that rule should not applied to vector
arguments.
---
How to pass argument to `foo`:
```
tail call void @foo(i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0,
<vscale x 16 x float> zeroinitializer,
<vscale x 16 x float> zeroinitializer,
<vscale x 16 x float> zeroinitializer)
```
`foo` take 13 arguments, first 8 argument pass in GPR, and next 2 LMUL 8 vector
arguments passed in v8-v23, and now we run out of argument register for GPR and
vector register, so we must pass last LMUL 8 vector argument by stack.
Which means we should reserve `vlenb * 8` byte for stack for the last
vector argument.
Reviewed By: craig.topper, asb
Differential Revision: https://reviews.llvm.org/D145938
The patch handles fixed type strict-fp by new RISCVISD::STRICT_ prefixed
isd nodes.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145900
We currently have 3 functions and 3 lookup tables. This was the
most expediant and obvious way to fix several bugs.
This patch uses a single function and single lookup
table. It uses APFloat::convert to convert from the half or double
to single precision. If the conversion doesn't have any errors or
lose any information we use the f32 table to finish the lookup.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D145897
This adds two new methods to ShuffleVectorInst, isInterleave and
isInterleaveMask, so that the logic to check if a shuffle mask is an
interleave can be shared across the TTI, codegen and the interleaved
access pass.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145971
I don't have a test case that fails for this, but it seemed like
we should only handle legal types. The callers I looked at in
DAGCombine either check the type is legal or don't even call
isFPImmLegal unless LegalOperations is true.
Written in a slightly odd way because switches on EVT require
an additional isSimple check so an if/else chain is easier. Used a bool
to shorten the code instead of having multiple ifs and returns.
AArch64 uses a similarish structure.
-Return false from RISCVDAGToDAGISel::selectFPImm for fli
constants so we don't try to use integer expansion.
-Support fli.h with Zvfh+Zfhmin.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D145766
The patch mainly does two things. The first is allowing scalable vector
ISD::STRICT_FP_EXTEND. The second is making RISC-V customized lower
strict_fpextend to riscv_strict_fpextend_vl, the strict version of
riscv_fpextend_vl.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145548
fli.h requires Zfh or Zvfh. We need to check for this in
isFPImmLegal. Zvfh support will come in another patch.
I had to split the test file because there are other issues with
Zfhmin and some intrinsics.
These intrinsics are equivalent to the regular @llvm.riscv.vssegNF
intrinsics, only they accept fixed length vectors in their overloaded
types: The regular intrinsics only operate on scalable vectors.
These intrinsics convert the fixed length vectors to scalable ones, and
then lower it on to the regular scalable intrinsic.
This mirrors the intrinsics added in 0803dba7dd998ad073d75a32b65296734c10ae70
This will be used in a later patch with the Interleaved Access pass.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D145022
Lower the two intrinsics introduced in D141924.
These intrinsics can be combined with loads and stores into the much more efficient segmented load and store instructions in a following patch.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D144092
A mistake in the control flow of performMemPairCombine resulted in paired
loads/stores for types that were not supported by the instructions (i8/i16).
These loads/stores could not match the constraints of the patterns defined
in the THead td file and the compiler would throw a 'Cannot select' error.
This is now fixed and two new test functions have been added in xtheadmempair.ll
which would previously crash the compiler. The compiler was additionally tested
with a wide range of benchmarks and no issues were observed.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D144559
Not entirely sure we'll end up reusing the body of the transform, but personally I find this structure easier to follow anyways.
Differential Revision: https://reviews.llvm.org/D144532
