This just reorders existing so we do the legal type check first.
In this particular test case we're also protected by the i1 check that I
also moved earlier.
Fixes#157177.
This is purely stylistic, but I think makes the code easier to follow.
It isn't quite NFC because it undoes the arithmetic lowering added yesterday in #156957 for the select c, simm12, 0 cases for a processor with both conditional move forwarding and zicond. What the right code is for that combination of features is currently an open question.
These instructions extract and range of bits, move them to the LSB
then sign or zero extend the result to fill the upper bits. If
the extracted bits is small enough then the result will have 33
sign bits.
These instructions are being used by the scalar efficiency SIG to
determine if we should have a standard instruction. Improving our
support for these instructions may give better data for their analysis.
I'll follow up by adding this to RISCVOptWInstrs too which will remove
the 'w' suffixes added here.
This is only used in the small code model and when `Xqcili` is enabled,
where addresses would otherwise use `LUI/ADDI`. Other code models need
to use pc-relative addressing. This patch does this for global/block
addresses, constant pools and jumptables.
Overall, this gives a better code size saving as` QC.E.LI` is easier to
relax to `QC.LI/LI` etc than` LUI/ADDI` (especially when the `LUI/ADDI`
might have sharing, or be split apart). `QC.E.LI` has the
`RISCV_QC_E_32` local relocation attached to it.
If a VL operand is > 31 then it will be materialized into an ADDI $x0,
imm. We can reason about it by peeking at the virtual register
definition which allows RISCVVectorPeephole and RISCVVLOptimizer to
catch more cases.
There's a separate issue with RISCVVLOptimizer where the materialized
immediate may not always dominate the instruction we want to reduce the
VL of, but this is left to another patch.
The arithmetic expansion requires fewer registers, and is often fewer
instructions. The critical path does increase by (up to) one
instruction.
This is a sub-case of the expansion we do without zicond, but restricted
specifically to the simm12 case. In the general case where the other
source is a register using zicond is likely better. (Edit: While
technically true, this is a bit misleading, we do this in
combineSelectToBinOp which is also used in the zicond path, just further
down.)
Add RISCVVLOptimizer supported for unit-stride, strided, and indexed
strided segmented stores. The biggest change was adding the capability
to look through INSERT_SUBREG, which was used for composing segmented
register class values.
Fix#149350
This is similar to the recently added (X & -4096) == 0 -> (X >> 12) ==
0,
but operating only on the lower 32 bits.
This also removes the (X & (1 << 31)) == 0 -> (xor (srliw X, 31), 1)
isel pattern. seqz and xori 1 should have similar cost and encoding
size.
To a slide1up, if the scalar value we're sliding in was extracted from
the first element of a vector, we can use a normal vslideup of 1 instead
with its passthru being that vector. This can eliminate an
extract_element instruction (i.e. vfmv.f.s, vmv.x.s).
---------
Co-authored-by: Craig Topper <craig.topper@sifive.com>
If an instruction's demanded VL is a virtual register defined by a vleff
instruction, it might not dominate and fail to have its VL reduced.
In leiu of the output VL, we can try and use the AVL passed to the vleff
itself since it will be at least greater than or equal the original VL.
I tried to create an LLVM IR test for this in but didn't have any luck
because the scheduler kept on moving the instruction past the vleff, so
it always dominated. So I've just included some mir tests instead.
The existing test case only exhausted the GPRs so the stack wasn't tested if FPRs were also available for arguments. This new test exhausts the GPRs and FPRs.
This patch implements direct N-way splitting for wide scalar shifts
instead
of recursive binary splitting. For example, an i512 G_SHL can now be
split
directly into 8 i64 operations rather than going through i256 -> i128 ->
i64.
The main motivation behind this is to alleviate (although not entirely
fix)
pathological compile time issues with huge types, like i4224. The
problem
we see is that the recursive splitting strategy combined with our messy
artifact combiner ends up with terribly long compiles as tons of
intermediate
artifacts are generated, and then attempted to be combined ad-nauseum.
Going directly from the large shifts to the destination types
short-circuits
a lot of these issues, but it's still an abuse of the backend and
front-ends
should never be doing this sort of thing.
RISCV does not provide a native atomic subtract instruction, so this
patch lowers `G_ATOMICRMW_SUB` by negating the RHS value and performing
an atomic add. The legalization rules in `RISCVLegalizerInfo` are
updated accordingly, with libcall fallbacks when `StdExtA` is not
available, and intrinsic legalization is extended to support
`riscv_masked_atomicrmw_sub`.
For example, lowering
`%1 = atomicrmw sub ptr %a, i32 1 seq_cst`
on riscv32a produces:
```
li a1, -1
amoadd.w.aqrl a0, a1, (a0)
```
On riscv64a, where the RHS type is narrower than XLEN, it currently
produces:
```
li a1, 1
neg a1, a1
amoadd.w.aqrl a0, a1, (a0)
```
There is still a constant-folding or InstConbiner gap. For instance,
lowering
```
%b = sub i32 %x, %y
%1 = atomicrmw sub ptr %a, i32 %b seq_cst
```
generates:
```
subw a1, a1, a2
neg a1, a1
amoadd.w.aqrl a0, a1, (a0)
```
This sequence could be optimized further to eliminate the redundant neg.
Addressing this may require improvements in the Combiner or Peephole
Optimizer in future work.
---------
Co-authored-by: Kane Wang <kanewang95@foxmail.com>
Before this patch, the selection for `QC_INSB` and `QC_INSBI` entirely
happens in C++, and does not support more than one non-constant input.
This patch seeks to rectify this shortcoming, by moving the C++ into a
target-specific DAGCombine, and adding `RISCV::QC_INSB`. One advantage
is this simplifies the code for handling `QC_INSBI`, as the C++ no
longer needs to choose between the two instructions based on the
inserted value (this is still done, but via ISel Patterns).
Another advantage of the DAGCombine is that this introduction can also
shift the inserted value to the `QC_INSB`, which our patterns need (and
were previously doing to the constant), and this shift can be
CSE'd/optimised with any prior shifts, if they exist. This allows the
inserted value to be variable, rather than a constant.
We had these RUN lines in our downstream and I couldn't tell for sure
that we had another Zfh calling convention test upstream.
Note we should fix the stack test to also exhaust the GPRs to make it
test the stack for ilp32f/lp64f. This was an existing issue in the
testing when F was enabled.
This patch tries to fix
[#155014](https://github.com/llvm/llvm-project/issues/155014). The
pattern of `ctlz`/`cttz` -> `icmp` -> `select` can occur when accounting
for targets which don't support `cttz(0)` or `ctlz(0)`. We can replace
this with a mask, but **only on power-of-2 bitwidths**.
In order to fold a vmerge into a pseudo, the pseudo's passthru needs to
be the same as vmerge's false operand.
If they don't match we can try and commute the instruction if possible,
e.g. here we can commute v9 and v8 to fold the vmerge:
vsetvli zero, a0, e32, m1, ta, ma
vfmadd.vv v9, v10, v8
vsetvli zero, zero, e32, m1, tu, ma
vmerge.vvm v8, v8, v9, v0
vsetvli zero, a0, e32, m1, tu, mu
vfmacc.vv v8, v9, v10, v0.t
Previously this wasn't possible because we did the peephole in
SelectionDAG, but now that it's been migrated to MachineInstr in #144076
we can reuse the commuting infrastructure in TargetInstrInfo.
This fixes the extra vmv.v.v in the "mul" example here:
https://github.com/llvm/llvm-project/issues/123069#issuecomment-3137997141
It should also allow us to remove the isel patterns described in #141885
later.
The general lowering of build_vector starts with splatting the first
operand before sliding down other operands one-by-one. However, if the
every operands is an extract_element from the first vector element, we
could use the original _vector_ (source of extraction) from the last
build_vec operand as start value before sliding up other operands (in
reverse order) one-by-one. By doing so we can avoid the initial splat
and eliminate the vector to scalar movement later, which is something we
cannot do with vslidedown/vslide1down.
---------
Co-authored-by: Craig Topper <craig.topper@sifive.com>
Co-authored-by: Luke Lau <luke@igalia.com>
This node reads the lower 32 bits, shifts it right arithmetically
then sign extends to i64. If we know some of the lower 32 bits we
can propagate that information.
For the test case I had to find something that didn't get optimized
before type legalizaton and didn't get type legalized to a sign
extended value. The bswap gets type legalized to (lshr (bswap), 32).
This case is the inverse of the one introduced in #155644. The
complexity with the inversion is that we need to also invert the
condition before shifting it. I had originally planned to only do so
when the condition was "cheaply" invertible (i.e. didn't require the
xori), but when looking more closely at the diffs I noticed that while
the XORI prevents this from being an icount improvement, and actually
lengthens slightly the critical path, it does still reduce the number of
registers needed.
This block of code is currently conditional on the fusions being enabled
but as far as I can tell, does no harm to generally enable. The net
effect is the generically compiled code runs slightly better on machines
with this fusion.
The actual motivation is merely to stop confusing myself when I see the
sequence in code; the register allocators choice to sometimes blow two
registers instead of one is just generally weird, and my eyes spot it
when scanning disassembly.
(Note that this is just the regalloc hint; the scheduling changes remain
conditional, and probably should remain so.)
