My initial goal was to name bits {19-15} as either rs1, vs1, or imm as
appropriate.
The VCIX instructions effectively have 3 properties that are combined:
the type of bits 19-15 (determined by funct3), how many vector registers
are sources (determined by funct6), and whether vd is a written or not
(determined by vm).
I found the VCIXInfo class very hard to read and understand. This patch
breaks it up into simpler steps and moves some properties to flags in
the existing VCIXType classs. A new VCIXRs1Info class is added to
contain the properties for bits 19-15. The VCIXInfo now combines the
properties from these 2 classes and the HaveOutputDst flag to create the
various things needed to build the final instruction. To reduce the
number of template arguments, the VCIXInfo is passed all the way down to
the base class.
Much of the refactoring in this patch was accomplished with AI. I have
looked at the diffs in the output tablegen files to verify it works as
expected. RISCVGenInstrInfo.inc only changes line number comments.
RISCVGenDisassembler.inc doesn't change at all as expected.
RISCVGenMCCodeEmitter.inc changes due to the different field names and
orders in the record.
optimizeCondBranch is looking for immediates that are 1 apart to rewrite
the branch by sharing a constant. BSETI x0, 11 can be used to produce
2048 which is one more than the largest positive constant produced by
addi.
When comparing multi-word integers with Zicond, we generate:
(or (czero_eqz (lo1 < lo2), (hi1 == hi2)),
(czero_nez (hi1 < hi2), (hi1 == hi2)))
The czero_nez is redundant because when hi1 == hi2 is true, hi1 < hi2 is
already 0. This patch adds a DAG combine to recognize:
czero_nez (setcc X, Y, CC), (setcc X, Y, eq) -> (setcc X, Y, CC)
when CC is a strict inequality (lt, gt, ult, ugt).
This saves one instruction in 128-bit comparisons on RV64 with Zicond.
Note the czero_nez becomes a czero.eqz in the final assembly because the
seteq is replaced by an xor that produces 0 when the values are equal.
Part of #179584
Assisted-by: claude
Compressing to a single shuffle doesn't remove any information and the backend can better apply specific optimizations to a single shuffle.
Addresses #176218.
---------
Co-authored-by: Luke Lau <luke_lau@igalia.com>
Extend the existing combineADDDToWMACC DAG combine to also match
RISCVISD::WMULSU and produce RISCVISD::WMACCSU. This is similar to
how ADDD+UMUL_LOHI is combined to WMACCU and ADDD+SMUL_LOHI is
combined to WMACC.
This patch was generated by AI, but I reviewed it.
Currently vector splice intrinsics are costed through getShuffleCost
when the offset is fixed. When the offset is variable though we can't
use a shuffle mask so it currently returns invalid.
This implements the cost in RISCVTTIImpl::getIntrinsicInstrCost as the
cost of a slideup and a slidedown, which matches the codegen.
It also implements the type based cost whenever the offset argument
isn't available.
It may be possible to reduce the cost in future when one of the vector
operands is known to be poison, in which case we only generate a single
slideup or slidedown.
When `Zvabd` exists, `llvm.abs` is lowered to `vabs.v` so the cost
is 1.
Reviewers: mshockwave, topperc, lukel97, skachkov-sc, preames
Reviewed By: topperc
Pull Request: https://github.com/llvm/llvm-project/pull/180146
We add pseudos/patterns for `vabs.v` instruction and handle the
lowering in `RISCVTargetLowering::lowerABS`.
Reviewers: topperc, 4vtomat, mshockwave, preames, lukel97, tclin914
Reviewed By: mshockwave
Pull Request: https://github.com/llvm/llvm-project/pull/180142
We directly lower `ISD::ABDS`/`ISD::ABDU` to `Zvabd` instructions.
Note that we only support SEW=8/16 for `vabd.vv`/`vabdu.vv`.
Reviewers: mshockwave, lukel97, topperc, preames, tclin914, 4vtomat
Reviewed By: lukel97, topperc
Pull Request: https://github.com/llvm/llvm-project/pull/180141
SpacemiT X100 is a 4-issue, out-of-order, RVA23 processor. This patch
introduces the base scheduling model for scalar instructions. The
scheduling model for RVV will be added in a future update.
We should add used callee-saved registers as implicit used to save
libcall and as implicit defined to restore libcall. It likes what we did
for CM_PUSH/CM_POPRET. That can help to construct correct dataflow. In
entry bb, save libcall implicitly uses the callee-saved registers which
live in. And in return bb, restore libcall implicitly defines the
callee-saved registers which live out.
Combine the pattern:
ADDD(addlo, addhi, UMUL_LOHI(x, y).0, UMUL_LOHI(x, y).1)
into:
WMACCU(x, y, addlo, addhi)
And similarly for SMUL_LOHI -> WMACC.
This patch was written with AI, but I reviewed it carefully.
We already do all the checks necessary in order to prioritize
MULHU/MULHS/UMUL_LOHI/SMUL_LOHI over MULHSU/WMULSU. We might as
well just emit the nodes instead of letting generic type legalization
redo the checks.
This is slightly different than the default legalization because we
don't have access to ExpandInteger so we have to emit TRUNCATES and
BUILD_PAIR. Not sure if this will result in any differences in practice.
Add custom lowering for INSERT_VECTOR_ELT on P extension vector types
using the MVM instruction.
TODO: Handle <4 x i8> on RV64 which is constructed to extract_vector_elt
+ build_vector instead of insert_vector_elt.
Order the operands so the the low and high part of the rs1 regpair are
first, followed by the low and high part of the rs2 regpair.
Also change the type to use v4i8 for the result so that it's only
shuffling elements not combining elements into a larger elment.
I'm planning to add ADDD and SUBD opcodes that will be defined with the
same operand order allowing RISCVISelDAGToDAG.cpp code to be shared.
There's a good chance we'll want to use these for scalar too.
Drop vector type from SDTypeProfile. Remove PMULHSU since we already
have RISCVISD::MULHSU for scalars in the base ISA.
This patch proposes new a tuning feature string format that helps users
to build a performance model by "configuring" an existing tune CPU,
along with its scheduling model. For example, this string
```
"sifive-x280:single-element-vec-fp64"
```
takes ``sifive-x280`` as the "base" tune CPU and configured it with
``single-element-vec-fp64``. This gives us a performance model that
looks exactly like that of ``sifive-x280``, except some of the 64-bit
vector floating point instructions now produce only a single element per
cycle due to ``single-element-vec-fp64``.
This string could eventually be used in places like ``-mtune`` at the
frontend. Right now, this patch only implements the parser part, which
is put under the TargetParser library.
The grammar for this string is:
```
tune-cpu ::= 'tuning CPU name in lower case'
directive ::= "[a-zA-Z0-9_-]+"
tune-features ::= directive ["," directive]*
```
A *directive* can and can only _enable_ or _disable_ a certain tuning
feature from the tuning CPU. A **positive directive**, like the
``single-element-vec-fp64`` we just saw, enables an additional tuning
feature in the associated tuning model.
A **negative directive**, on the other hand, removes a certain tuning
feature. For example, ``sifive-x390`` already has the
``single-element-vec-fp64`` feature, and we can use
"sifive-x390:no-single-element-vec-fp64" to create a new performance
model that looks nearly the same as ``sifive-x390`` except
``single-element-vec-fp64`` being cut out. In this case,
``no-single-element-vec-fp64`` is a negative directive.
There are additional restrictions on what we can put in the list of
directives, please refer to the documentations for more details.
Right now, this string only accepts directives that are explicitly
supported by the tune CPU. For example, "sifive-x280:prefer-w-inst" is
not a valide string as ``prefer-w-inst`` is not supported by
``sifive-x280`` at this moment. Vendors of these processors are expected
to maintain the compatibility of their supported directives across
different versions.
---------
Co-authored-by: Sam Elliott <aelliott@qti.qualcomm.com>
Currently we only fold an op into vmerge if we know the smaller of the
two vls, because we can't increase the vl.
However if the op's passthru is undef, then we can just use vmerge's vl
because the lanes past op's vl were undef anyway. We need to make sure
that the op doesn't access memory though. Other instructions where the
result depends on the VL should already be handled by the
RISCVII::elementsDependOnVL check below.
This is probably always profitable because even though we increase vl,
we remove a vmerge which needs to process vl elements anyway.
This removes some regressions in #179622.
If the source of an fpto*i doesn't fit in the destination type, the
result is poison. For i1 destinations, this means the result needs to be
0 or 1/-1, so we can just compare the result to 0 directly instead of
truncating.
The VP lowering for fpto*i already does this.
XOR should be OR to match the comment.
Found while reviewing #179622 which deletes this function. I would like
to commit this first so we have a correct baseline for reviewing that
patch.
The only caller of this function (`PeepholeOptimizer::optimizeSelect`)
did not use most of the parameters, was broadly equivalent to
`MI->isSelect()`, and the `optimizeSelect` hook can return `nullptr`
anyway.
Update `optimizeSelect` to return `nullptr` by default rather than
asserting when not implemented.
Replace with hasStdExtZcd() and hastStdExtZcf().
Creation of RISCVSubtarget/MCSubtargetInfo handles implication of Zcf
and Zcd now. The exception is .option arch handling which will require
+zcf and +zcd to be listed explicitly. I'll try to fix this in a follow
up. #155035 had the same issue.
I've left the error messages mentioning both Zcf and C+F/D. We can
consider changing that in a follow up.
This patch fixes https://github.com/llvm/llvm-project/issues/150737.
The original computed CSRCost is too small, so the optimization of
spilling instead of using CSR is rarely triggered.
Also the original cost model is too difficult to be understood and too
hard to be tuned by backend developers and users.
So this patch changes the CSRCost to be
CSRCost = TRI->getCSRFirstUseCost() * EntryFreq * Scale
TRI->getCSRFirstUseCost() is the raw cost of save/restore a CSR. Usually
we don't need to tune this number.
EntryFreq is the BlockFrequency of the entry block.
Scale is used to scale down the CSRCost, because we usually prefer a CSR
register instead of spilling if we have similar CSRCost and spill cost,
so it should be less than 100%. We usually tune this number.
Another problem is the original function RAGreedy::calcSpillCost()
actually computes a cost for block split, so this patch also implements
a correct RAGreedy::calcSpillCost() function.
This new behavior is not enabled by default. This optimization is used
by 3 targets (AArch64 / AMDGPU / RISCV), I will change them one by one
in following patches.
This ensures the feature bits and RISCVSubtarget flags match what
RISCVISAInfo would do.
I'm not excited about the code duplication, but I need to set the
RISCVSubtarget flags along with calling ToggleFeature. I'll think about
how to improve this.
`constrainSelectedInstRegOperands` always returns `true`; so it can be
safely transformed to return `void` instead.
A follow-up patch should update `MachineInstrBuilder::constrainAllUses`.
This is part of the work to remove trivial VP intrinsics.
In the combineVectorSizedSetCCEquality combine, used for the compares
that ExpandMemcmp generates, we currently emit a VP_SETCC. We can just
emit a regular SETCC and let RISCVVLOptimizer take care of reducing the
VL.
This correctly names the operands vd, vs1, and vs2 instead of rd, rs1,
and rs2. RVInstVCCustom2 is now only used for VCIX which has its own
operand naming problems.
I'm considering using named operand indices in
RISCVAsmParser::validateInstruction for the RVVConstraints, but first I
would have to make vs1, vs2 named correctly across all vector
instructions.
When working on #177238 I found some cases where machine SSA
optimizations (or any optimizations that run before the current
VLOptimizer, really) can benefit from reduced VL operands. In addition,
by running VLOptimizer early, in the future we can further remove the
mini VL reduction currently in RISCVVectorPeephole, once we teach
VLOptimizer some corner cases (e.g. handle vector stores).
This patch therefore moves VLOptimizer to be (basically) right after the
ISel phase.
If we know the shift amount is greater than or equal to the
incoming EEW, the zext will have been converted to an anyext by
SimplifyDemandedBits. Treat this case the same as zext.
Rather than using lets around classes/defs, override them in the class
def/body.
Some of these lets were around single class/def were I thought it was
better to be inside. Some were around multiple unrelated classes where
it seemed better not to link their encodings like that.
For vmv, I added a multiclass to better encapsulate them but still kept
the let scope to avoid repetition. The encodings are closely related
enough that I thought this was ok.
