Extending to f32 first (as is done for f16) results in better generated
code for RISC-V (and affects no other in-tree tests). Additionally,
performing the FP_EXTEND first seems equally justified for bf16 as for
f16.
Differential Revision: https://reviews.llvm.org/D156944
This ports over the test cases half-convert.ll and implements patterns
or RISCVISelLowering.cpp changes for all of the most straight-forward
cases (those that don't require changes outside of lib/Target/RISCV).
The remaining cases and noted poor codegen for saturating conversions
will be handled in follow-up patches.
Differential Revision: https://reviews.llvm.org/D156943
This will prevent the `assert(!O.readsReg())` from firing in
SIOptimizeVGPRLiveRange::optimizeLiveRange
Fix for #64163
Reviewed By: arsenm, #amdgpu
Differential Revision: https://reviews.llvm.org/D156893
This now reuses the existing lowering for G_FMIN/MAX for G_FABS too, which can
handle more type successfully. We can hopefully reuse the same pattern action
definition for other fp operations too.
Extend the existing 128-bit -> 64-bit truncation handling by widening/narrowing the src/dst vectors and use the lower half operand/result for PACKSS/PACKUS instructions.
We currently just scalarize sub-128-bit vector truncations, but if the input vector has sufficient signbits/zerobits then we should try to use PACKSS/PACKUS with a widened vector with don't care upper elements. Shuffle lowering will struggle to detect this if we wait until the scalarization has been revectorized as a shuffle.
Another step towards issue #63710
Tweak the immediate on two vror.vi test cases to use a uimm6 immediate
that would have failed before D156974 when we were looking for a simm6
immediate.
Part of this test file was stolen from D156895. We should merge them
when committing.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D156926
Previous heuristics had a big flaw: they only looked at single PHI at a time, and didn't take into account the whole "chain".
The concept of "chain" is important because if we only break a chain partially, we risk forcing regalloc to reserve twice as many registers for that vector.
We also risk adding a lot of copies that shouldn't be there and can inhibit backend optimizations.
The solution I found is to consider the whole "PHI chain" when looking at PHI.
That is, we recursively look at the PHI's incoming value & users for other PHIs, then make a decision about the chain as a whole.
The currrent threshold requires that at least `ceil(chain size * (2/3))` PHIs have at least one interesting incoming value.
In simple terms, two-thirds (rounded up) of the PHIs should be breakable.
This seems to work well. A lower threshold such as 50% is too aggressive because chains can often have 7 or 9 PHIs, and breaking 3+ or 4+ PHIs in those case often causes performance issue.
Fixes SWDEV-409648, SWDEV-398393, SWDEV-413487
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D156414
When compiling Rust code we may end up with calls to functions provided
by other code units. Presently this code crashes on a null pointer
dereference - this patch avoids that crash and adds a test.
Reviewed By: ast
Differential Revision: https://reviews.llvm.org/D156446
When function has different attributes from module, emit the .option <attribute> before the function body. This allows non-integrated assemblers to properly assemble the functions (which may contain instructions dependent on the extra target features).
Reviewed By: craig.topper, reames
Differential Revision: https://reviews.llvm.org/D155155
When generating unwind tables for code which uses return-address
signing, we need to toggle the RA_SIGN_STATE DWARF register around any
tail-calls, because these require the return address to be authenticated
before the call, and could throw an exception. This is done using the
.cfi_negate_ra_state directive before the call, and .cfi_restore_state
at the start of the next basic block.
However, since D153098, the .cfi_restore_state isn't being inserted,
because the CFIFixup pass isn't being run. This re-enables that pass
when return-adress signing is enabled.
Reviewed By: ikudrin, MaskRay
Differential Revision: https://reviews.llvm.org/D156428
Backwards frame index elimination uses backwards register scavenging,
which is preferred because it does not rely on accurate kill flags.
Differential Revision: https://reviews.llvm.org/D156691
Similar to D156350, if we were going to create 2 x i16 insertions (MOVD+PINSRW), try to merge them into a single MOVD to reduce the amount of GPR<->VEC traffic
Previously we used the number of registers needed saved and pushable as the
number of pushed registers. We also use pushed register number to caculate
the stack size. It is not correct because Zcmp pushes registers from $ra to the
max register needed saved and there is no gurantee that the needed saved
registers are a sequenced list from $ra.
There is an example about that. PushPopRegs should be 6 (ra,s0 - s4)= instead of 1.
```
; llc -mtriple=riscv32 -mattr=+zcmp
define void @foo() {
entry:
; Old: .cfi_def_cfa_offset 16
; New: .cfi_def_cfa_offset 32
tail call void asm sideeffect "li s4, 0", "~{s4}"()
ret void
}
```
Reviewed By: Jim, kito-cheng
Differential Revision: https://reviews.llvm.org/D156407
This doesn't bring us to parity with the test/CodeGen/RISCV/half-* test
cases, it simply picks off an initial set that can be supported
especially easy. In order to make the review more manageable, I'll
follow up with other cases.
There is zero innovation in the test cases - they simply take the
existing half/float cases and replace f16->bf16 and half->bfloat.
Differential Revision: https://reviews.llvm.org/D156895
isOperationLegalOrCustomOrPromote returns true only if VT is other or legal
and operation action is Legal, Custom or Promote.
Permit a vector binary operation can be converted to scalar binary operation which is custom lowered with illegal type.
One of cases is i32 isn't a legal type on RV64 and its ALU operations is set to custom lowering,
so vadd for element type i32 can be converted to addw.
Reviewed By: jacquesguan, craig.topper
Differential Revision: https://reviews.llvm.org/D156692
fp-imm.ll and zfh-imm.ll test 0.0 and -0.0 while float/double/half-imm.ll
tested other non-zero constants. It seems like they should all be
tested together.
There are slight coverage changes due to different command lines,
but I'm not sure its meaningful. For example, we now don't test
double 0.0 and -0.0 with only the F extension.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D156929
D155929 teach lowerScalarInsert to handl start value (extractelement scalable_vector, 0)
and specifically converts fixed extracted vectors to scalable vectors when
lowering vector reduction. It's not enough because there is another way to
create (extractelement fixed_vector, 0) as a start value of lowerScalarInsert
like #64327.
#64327: https://github.com/llvm/llvm-project/issues/64327.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D156863
The first trivial example I tried failed to merge due to the user scan
logic. Remove the complicated scan of users handling with distance
thresholds, with a same block restriction. The actual expansion of
sincos is basically the same size as sin or cos individually. Copy the
technique the generic optimization uses, which is to just use the
input instruction as the insert point or just insert at the start of
the entry block.
https://reviews.llvm.org/D156706
Ran across this when making a change to RISCV memset lowering. Seems very odd that manually merging a store into a vector prevents it from being further merged.
Differential Revision: https://reviews.llvm.org/D156349
These test cases previously caused an error. RISCVInstrInfo::copyPhysReg also needed a tweak in order to account for copying bf16 values in FPR16 registers.
Differential Revision: https://reviews.llvm.org/D156883
For G_ABS with type v2s16 and sgpr inputs break down into two s32 G_ABS
instructions.
Patch by: Acim Maravic
Differential Revision: https://reviews.llvm.org/D155867
There is no need to increase the size of odd sized vectors if they are
going to be scalarized by a different rule.
Patch by: Acim Maravic
Differential Revision: https://reviews.llvm.org/D155865
As noted in <https://github.com/llvm/llvm-project/issues/64090>, it's
more efficient to lower a partword 'atomicrmw xchg a, 0` to and amoand
with appropriate mask. There are a range of possible ways to go about
this - e.g. writing a combine based on the
`llvm.riscv.masked.atomicrmw.xchg` intrinsic, or introducing a new
interface to AtomicExpandPass to allow target-specific atomics
conversions, or trying to lift the conversion into AtomicExpandPass
itself based on querying some target hook. Ultimately I've gone with
what appears to be the simplest approach - just covering this case in
emitMaskedAtomicRMWIntrinsic. I perhaps should have given that hook a
different name way back when it was introduced.
This also handles the `atomicrmw xchg a, -1` case suggested by Craig
during review.
Fixes https://github.com/llvm/llvm-project/issues/64090
Differential Revision: https://reviews.llvm.org/D156801
MachineVerifier does not check that DBG_VALUE, DBG_VALUE_LIST and
DBG_INSTR_REF have the expected number of operands, so printing them
(e.g. with -print-after-all) should not crash.
Differential Revision: https://reviews.llvm.org/D156226
Issue mentioned: https://github.com/riscv/riscv-code-size-reduction/issues/182
The order of callee-saved registers stored by Zcmp push in memory is reversed.
Pseudo code for cm.push in https://github.com/riscv/riscv-code-size-reduction/releases/download/v1.0.4-1/Zc.1.0.4-1.pdf
```
if (XLEN==32) bytes=4; else bytes=8;
addr=sp-bytes;
for(i in 27,26,25,24,23,22,21,20,19,18,9,8,1) {
//if register i is in xreg_list
if (xreg_list[i]) {
switch(bytes) {
4: asm("sw x[i], 0(addr)");
8: asm("sd x[i], 0(addr)");
}
addr-=bytes;
}
}
```
The placement order for push is s11, s10, ..., ra.
CFI offset should be calculed as reversed order for correct stack unwinding.
Reviewed By: fakepaper56, kito-cheng
Differential Revision: https://reviews.llvm.org/D156437
