Even though we only need to write to the bottom NumElts - Rotation
elements for the vslidedown.vi, we can save an extra vsetivli toggle if
we just keep the wide VL.
(I may be missing something here: is there a reason why we want to explicitly keep the vslidedown narrow?)
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D151390
This is an attempt to reland D42600 and enabling this optimisation by default.
This also resolves the issue pointed out in the context of PGO build.
Differential Revision: https://reviews.llvm.org/D42600
This adds the vfslide1down (and vfslide1up for consistency) nodes. These mostly parallel the existing vslide1down/up nodes. (See note below on instruction semantics.) We then use the vfslide1down in build_vector lowering instead of going through the stack.
The specification is more than a bit vague on the meaning of these instructions. All we're given is "The vfslide1down instruction is defined analogously, but sources its scalar argument from an f register."
We have to combine this with a general note at the beginning of section 10. Vector Arithmetic Instruction Formats which reads: "For floating-point operations, the scalar can be taken from a scalar f register. If FLEN > SEW, the value in the f registers is checked for a valid NaN-boxed value, in which case the least-signicant SEW bits of the f register are used, else the canonical NaN value is used. Vector instructions where any floating-point vector operand’s EEW is not a supported floating-point type width (which includes when FLEN < SEW) are reserved.".
Note that floats are NaN-boxed when D is implemented.
Combining that all together, we're fine as long as the element type matches the vector type - which is does by construction. We shouldn't have legal vectors which hit the reserved encoding case. An assert is included, just to be careful.
Differential Revision: https://reviews.llvm.org/D151347
The immediate field on the vsetivli is fairly limited. For larger vectors, we end up having to materialize a constant in a register. We hadn't plumbed the infrastructure to treat such materialized constants as constants for purpose of vsetvli elimination.
I only bothered to handle LI. We could extend this to LUI sequences, but well, 2048 elements is probably enough for all practical fixed length vector codegen. :)
The test delta does point out a related problem. At LMUL8, we see increased register allocation pressure, and we should probably either a) address register allocation remat, or b) be less aggressive about eliminating vsetvlis at high lmul. Note that high LMUL code is not generated much by default.
Differential Revision: https://reviews.llvm.org/D151212
For stores of small fixed-length vector constants, we can store them
with a sequence of lui/addi/sh/sw to avoid the cost of building the
vector and the vsetivli toggle, provided the constant materialization
cost isn't too high.
This subsumes the optimisation for stores of zeroes in
4dc9a2c5b93682c12d7a80bbe790b14ddb301877
(This is a reapply of 0ca13f9d2701e23af2d000a5d8f48b33fe0878b7)
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D151221
For stores of small fixed-length vector constants, we can store them
with a sequence of lui/addi/sh/sw to avoid the cost of building the
vector and the vsetivli toggle.
Note that this only handles vectors that are 32 bits or smaller, but
could be expanded to 64 bits if we know that the constant
materialization cost isn't too high.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D151221
For small fixed-length vector copies like
vsetivli zero, 2, e16, m1, ta, ma
vle16.v v8, (a0)
vse16.v v8, (a1)
We can scalarize them if the total vector size < XLEN:
lw a0, 0(a0)
sw a0, 0(a1)
This patch adds a DAG combine to do so, reusing much of the existing
logic in https://reviews.llvm.org/D150717
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D151103
Add minimal support to lower return, and introduce an OutgoingValueHandler and an OutgoingValueAssigner for returns.
Supports return values with integer, pointer and aggregate types.
(Update of D69808 - avoiding commandeering that revision)
Co-authored By: lewis-revill
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D117318
We can compute a simpler expression for Lo for these cases. This
is an alternative for the test cases in D151180 that works for
more targets.
This is similar to some of the special cases we have for expanding
setcc operands.
Differential Revision: https://reviews.llvm.org/D151182
For always poison shifts, any KnownBits return value is valid.
Currently we return unknown, but returning zero is generally more
profitable. We had some code in ValueTracking that tried to do this,
but was actually dead code.
Differential Revision: https://reviews.llvm.org/D150648
Normally a mul by a power 2 will be converted to shift by InstCombine.
Hoping this will make D127115 not affect this test so much.
I've verified the change still fails with the original DAGCombiner
bug.
This patch-set aims to simplify the existing RVV segment load/store
intrinsics to use a type that represents a tuple of vectors instead.
To achieve this, first we need to relax the current limitation for an
aggregate type to be a target of load/store/alloca when the aggregate
type contains homogeneous scalable vector types. Then to adjust the
prolog of an LLVM function during lowering to clang. Finally we
re-define the RVV segment load/store intrinsics to use the tuple types.
The pull request under the RVV intrinsic specification is
riscv-non-isa/rvv-intrinsic-doc#198
---
This is the 1st patch of the patch-set. This patch is originated from
D98169.
This patch allows aggregate type (StructType) that contains homogeneous
scalable vector types to be a target of load/store/alloca. The RFC of
this patch was posted in LLVM Discourse.
https://discourse.llvm.org/t/rfc-ir-permit-load-store-alloca-for-struct-of-the-same-scalable-vector-type/69527
The main changes in this patch are:
Extend `StructLayout::StructSize` from `uint64_t` to `TypeSize` to
accommodate an expression of scalable size.
Allow `StructType:isSized` to also return true for homogeneous
scalable vector types.
Let `Type::isScalableTy` return true when `Type` is `StructType`
and contains scalable vectors
Extra description is added in the LLVM Language Reference Manual on the
relaxation of this patch.
Authored-by: Hsiangkai Wang <kai.wang@sifive.com>
Co-Authored-by: eop Chen <eop.chen@sifive.com>
Reviewed By: craig.topper, nikic
Differential Revision: https://reviews.llvm.org/D146872
Provides MC layer support for Zvfbfwma: vector BF16 widening mul-add.
As currently specified, Zvfbfwma does not appear to have a dependency on
Zvfbfmin or Zfbfmin.
Differential Revision: https://reviews.llvm.org/D147612
Provides MC layer support for Zfbfmin: vector BF16 conversions.
Zvfbfmin does not appear to have a dependency on Zfbfmin as currently
specified.
Differential Revision: https://reviews.llvm.org/D147611
Provides MC layer support for Zfbfmin: scalar BF16 conversions.
As documented, this extension includes FLH, FSH, FMV.H.X, and FMH.X.H as
defined in Zfh/Zfhmin, but doesn't require either extension.
No Zfbfinxmin has been defined (though you would expect one in the
future, for symmetry with Zfhinxmin). See issue
https://github.com/riscv/riscv-bfloat16/issues/27.
Differential Revision: https://reviews.llvm.org/D147610
In general, VL=2 vectors are very questionable profitability wise. For constants specifically, our inability to materialize many vector constants cheaply biases us strongly towards unprofitability at VL=2.
This hook is very close to the x86 implementation. The difference is that X86 whitelists stores of zeros, and we're better off letting that stay scalar at VL=2.
Differential Revision: https://reviews.llvm.org/D150798
This is a follow-up to b71edfaa4ec3c998aadb35255ce2f60bba2940b0
since I forgot the lit.local.cfg files in that one.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: barannikov88, kwk
Differential Revision: https://reviews.llvm.org/D150762
The direct motivation here is to undo an unprofitable vectorization performed by SLP, but the transform seems generally useful as well. If we are storing a zero to memory, we can use a single scalar store (from X0) for all power of two sizes up to XLen.
Differential Revision: https://reviews.llvm.org/D150717
This patch does the following things:
1, Add accurate Predicates in intrinsic-pseudo pattern class depending on the vector type.
2, Make vmulh, vmulhsu, vmulhu and vsmul intrinsic only able to select in v extension.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D150550
These aren't currently created from anything that has a passthru
operand. If we need to support this in the future, we should add
a policy operand to the ISD node definition.
We can apply the same logic as for multiply since a left shift is just a multiply by a power of two. Note that since shl is not commutative, we do need to be careful to match sure that the splat is the RHS of the instruction.
Differential Revision: https://reviews.llvm.org/D150471
This has been present from the original patch which added the pass, and doesn't appear to be strongly justified. We do need to be careful of commutativity.
Differential Revision: https://reviews.llvm.org/D150468
For the "C" Standard Extension/Zca, D45560 enabled 2-byte alignment for
assembly output (e.g. `clang -S a.c`) and D102052 enabled 2-byte alignment for
assembly input and object file output (e.g. `clang -c a.s`).
This patch ports the behavior for code generation and object file output by
adding RISCVELFTargetObjectFile::getTextSectionAlignment (e.g. `clang -c a.c`).
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D150240
This patch adds a check for whether the memory operand is known to be
a jump table and, if so, allows shrinkwrapping to continue. In the
case that we are looking at a jump table, I believe it is safe to
assume that the access will not be to the stack (but please correct me
if I am wrong here).
In the test attached, this is helpful in that we are able to generate
only one instruction for each non-default case in the original switch
statement.
Differential Revision: https://reviews.llvm.org/D149886
This reverts commit d9683a70fef48cfaee2c83147a3b26f4f90162a2. A regression was reported in the review, revert until author is back from conference and can investigate.
The original change had a bug where it allowed SEW mutation. This is wrong in multiple ways, but an easy example is that the slide amount is in units of SEW, and thus that changing SEW changes the slide offset.
I'd reverted this in 33314693 intending to more majorly rework the patch because in addition to the bug, I'd noticed a potential oppurtunity to increase scope. After implementing that variant, and realizing it triggered nowhere, I decided to go back to the prior patch with the minimal fix.
Note there's no separate test case for the fix. This is because we already had multiple, and I just didn't realize the impact of the original test diff. Adding one more test would have been unlikely to catch that human error.
Original commit message..
Noticed this while looking at some SLP output. If we have an extractelement, we're probably using a slidedown into an destination with no contents. Given this, we can allow the slideup to use a larger VL and clobber tail elements of the destination vector. Doing this allows us to avoid vsetvli toggles in many fixed length vector examples.
Differential Revision: https://reviews.llvm.org/D148834
In D79537, `nomerge` was made to only apply to non-tail calls. This fixes it by also applying it to tail calls.
For ARM, I only made the new MI to inherit the flag under `TCRETURNdi` and `TCRETURNri`, because that's the place tail calls got replaced. Not sure if there's any other place needed.
Fixes#61545.
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D146749
This patch enables signed truncation check transforms for i8 on rv32 when XVT is i64 and Zbb is enabled.
It is a small improvement of D149977.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D150177
