We should add used callee-saved registers as implicit used to save
libcall and as implicit defined to restore libcall. It likes what we did
for CM_PUSH/CM_POPRET. That can help to construct correct dataflow. In
entry bb, save libcall implicitly uses the callee-saved registers which
live in. And in return bb, restore libcall implicitly defines the
callee-saved registers which live out.
This preparatory patch introduces an additional argument to the target hook
loadRegFromStackSlot. Ths is essential for targets to handle subregister-specific
reload in the future. See how this is used for AMDGPU target with PR #175002.
This removes the tracking of the MinCSFrameIndex, and MaxCSFrameIndex markers, simplifying the target API. This brings the tracking for callee save spill slots in line with how we handle other properties of stack locations.
A couple notes:
1) This requires doing scans of the entire object range, but we have other such instances in the code already, so I doubt this will matter in practice.
2) This removes the requirement that callee saved spill slots be contiguous in the frame index identified space.
I marked this as NFCI because if prior code violated the contiguous range assumption - I can't find a case where we did - then this change might adjust frame layout in some edge cases.
The motivation for this is mostly code readability, but I might use this as a primitive for something in an upcoming patch series around shrink wrapping. Haven't decided yet.
This splits out "ScalablePredicateVector" from the "ScalableVector"
StackID this is primarily to allow easy differentiation between vectors
and predicates (without inspecting instructions).
This new stack ID is not used in many places yet, but will be used in a
later patch to mark stack slots that are known to contain predicates.
Co-authored-by: Kerry McLaughlin <kerry.mclaughlin@arm.com>
Zicfiss (Shadow Stack) instructions are implemented using the encoding
space defined in the "Zimop" May-Be-Operations Extension, where the
instruction behaviors turn into NOP if Zimop is implemented but the
redefining extension does not present.
This means we can safely loosen the codegen requirement of shadow stack
to Zimop, in that those processors with Zimop but no Zicfiss can still
execute the instructions without problem.
This patch add new pseudo instructions to model MOPs that are expanded
into Zicfiss instructios, and change to emit them in the codegen
In TargetFrameLowering::determineCalleeSaves, any vector register is
marked
as saved if any of its subregister is clobbered, this is not correct in
vector registers. We only want the vector register to be marked as saved
only if all of its subregisters are clobbered.
This patch handles vector callee saved registers in target hook.
Currently, AsmPrinter skips CFI instructions created by a backend if
they are not needed. I'd like to change that so that it always
prints/encodes CFI instructions if a backend created them.
This change should slightly (perhaps negligibly) improve compile time as
post-PEI passes no longer need to skip over these instructions in
no-exceptions no-debug builds, and will allow to simplify convoluted
logic in AsmPrinter once other targets stop emitting CFI instructions
when they are not needed (that's my final goal).
The changes in a test seem to be caused by slightly different post-RA
scheduling in the absence of CFI instructions.
When not in the prologue we do not want to set the FrameSetup flag, by
passing the flag as argument we can use allocateStack correctly on those
cases.
This fixes the allocation and probe in eliminateCallFramePseudoInstr.
As of 20b5728b7b1ccc4509a316efb270d46cc9526d69, C always enables Zca, so
the check `C || Zca` is equivalent to just checking for `Zca`.
This replaces any uses of `HasStdExtCOrZca` with a new `HasStdExtZca`
(with the same assembler description, to avoid changes in error
messages), and simplifies everywhere where C++ needed to check for
either C or Zca.
The Subtarget function is just deprecated for the moment.
Stack clash protection code was missing from
RISCVFrameLowering::eliminateCallFramePseudoInstr, calling allocateStack
fixes it.
This patch also fixes the tests in stack-probing-dynamic.ll that should
be testing the stack allocation before a function call.
The expensive checks bots found issues with #132481, due to not setting
defs/uses correctly. In 31bd7a507152 I added verify flags, so that the
failure is reproduced without requiring expensive checks, and xfailed
the test.
This change:
- Ensures that registers are correctly marked as defs/uses.
- Removes the xfail.
- Leaves the tests with `-verify-machineinstrs` which should have been
present originally.
The `qc.c.mienter` and `qc.c.mienter.nest` instructions, broadly only
save the argument and temporary registers. The exceptions are that they
also save `fp` (`s0`) to construct a frame chain from the signal handler
to the frame below, and they also save `ra`. They are designed this way
so that (if needed) push and pop instructions can be used to save the
callee-saved registers.
This patch implements this optimisation, constructing the following
rather than a long sequence of `sw` and `lw` instructions for saving the
callee-saved registers:
```asm
qc.c.mienter
qc.cm.push {ra, s0-sN}, -M
...
qc.cm.pop {ra, s0-sN}, M
qc.c.mileaveret
```
There are some carefully-worked-out details here, especially around CFI
information. For any register saved by both `qc.c.mienter(.nest)` and
the push (which is `ra` and `s0` at most), we point the CFI information
at the version saved by `qc.c.mienter(.nest)`. This ensures the CFI
points at the same `fp` copy as a frame pointer unwinder would find.
This Change adds support for two SiFive vendor attributes in clang:
- "SiFive-CLIC-preemptible"
- "SiFive-CLIC-stack-swap"
These can be given together, and can be combined with "machine", but
cannot be combined with any other interrupt attribute values.
These are handled primarily in RISCVFrameLowering:
- "SiFive-CLIC-stack-swap" entails swapping `sp` with `sf.mscratchcsw`
at function entry and exit, which holds the trap stack pointer.
- "SiFive-CLIC-preemptible" entails saving `mcause` and `mepc` before
re-enabling interrupts using `mstatus`. To save these, `s0` and `s1`
are first spilled to the stack, and then the values are read into
these registers. If these registers are used in the function, their
values will be spilled a second time onto the stack with the generic
callee-saved-register handling. At the end of the function interrupts
are disabled again before `mepc` and `mcause` are restored.
This Change also adds support for the following two experimental
extensions, which only contain CSRs:
- XSfsclic - for SiFive's CLIC Supervisor-Mode CSRs
- XSfmclic - for SiFive's CLIC Machine-Mode CSRs
The latter is needed for interrupt support.
The CFI information for this implementation is not correct, but I'd
prefer to correct this in a follow-up. While it's unlikely anyone wants
to unwind through a handler, the CFI information is also used by
debuggers so it would be good to get it right.
Co-authored-by: Ana Pazos <apazos@quicinc.com>
We can't assume MBBI is still pointing at MBB if we've already expanded
a probe. We need to re-query the MBB from MBBI. Fixes#135206
Co-authored-by: Craig Topper <craig.topper@sifive.com>
Spimm in the spec refers to the 2-bit encoded value. All of the code
uses the 0, 16, 32, or 48 adjustment value.
Also remove the decodeZcmpSpimm as its identical to the default
behavior for no custom DecoderMethod.
This adds the Xqciint system registers from the Xqci-0.7 spec, as well
as two leftover instructions: `qc.c.mret` and `qc.c.mnret`
Co-authored-by: Sudharsan Veeravalli <quic_svs@quicinc.com>
Currently, we use `csrr` with `vlenb` to obtain the `VLEN`, but this is
not the only option. We can also use `vsetvli` with `e8`/`m1` to get
`VLENMAX`, which is equal to the VLEN. This method is preferable on some
microarchitectures and makes it easier to obtain values like `VLEN * 2`,
`VLEN * 4`, or `VLEN * 8`, reducing the number of instructions needed to
calculate VLEN multiples.
However, this approach is *NOT* always interchangeable, as it changes
the state of `VTYPE` and `VL`, which can alter the behavior of vector
instructions, potentially causing incorrect code generation if applied
after a vsetvli insertion. Therefore, we limit its use to the
prologue/epilogue for now, as there are no vector operations within the
prologue/epilogue sequence.
With further analysis, we may extend this approach beyond the
prologue/epilogue in the future, but starting here should be a good
first step.
This feature is gurded by the `+prefer-vsetvli-over-read-vlenb` feature,
which is disabled by default for now.
The Xqci 0.7.0 spec just came out, with some updates to Xqciint,
bringing it to v0.4. The main update of any relevance is that
`qc.c.mienter` and `qc.c.mienter.nest` now update both the stack pointer
and the frame pointer (before, they only updated the stack pointer).
They both remain compatible with the frame pointer convention.
This change bumps the Xqciint version, and ensures that we don't emit
the unneeded frame pointer adjustment instruction after
`qc.c.mienter(.nest)`.
This change adds support for `qci-nest` and `qci-nonest` interrupt
attribute values. Both of these are machine-mode interrupts, which use
instructions in Xqciint to push and pop A- and T-registers (and a few
others) from the stack.
In particular:
- `qci-nonest` uses `qc.c.mienter` to save registers at the start of the
function, and uses `qc.c.mileaveret` to restore those registers and
return from the interrupt.
- `qci-nest` uses `qc.c.mienter.nest` to save registers at the start of
the function, and uses `qc.c.mileaveret` to restore those registers and
return from the interrupt.
- `qc.c.mienter` and `qc.c.mienter.nest` both push registers ra, s0
(fp), t0-t6, and a0-a10 onto the stack (as well as some CSRs for the
interrupt context). The difference between these is that
`qc.c.mienter.nest` re-enables M-mode interrupts.
- `qc.c.mileaveret` will restore the registers that were saved by
`qc.c.mienter(.nest)`, and return from the interrupt.
These work for both standard M-mode interrupts and the non-maskable
interrupt CSRs added by Xqciint.
The `qc.c.mienter`, `qc.c.mienter.nest` and `qc.c.mileaveret`
instructions are compatible with push and pop instructions, in as much
as they (mostly) only spill the A- and T-registers, so we can use the
`Zcmp` or `Xqccmp` instructions to spill the S-registers. This
combination (`qci-(no)nest` and `Xqccmp`/`Zcmp`) is not implemented in
this change.
The `qc.c.mienter(.nest)` instructions have a specific register storage
order so they preserve the frame pointer convention linked list past the
current interrupt handler and into the interrupted code and frames if
frame pointers are enabled.
Co-authored-by: Pankaj Gode <quic_pgode@quicinc.com>
This adds support for Xqccmp to the following passes:
- Prolog Epilog Insertion - reusing much of the existing push/pop logic,
but extending it to cope with frame pointers and reorder the CFI
information correctly.
- Move Merger - extending it to support the `qc.` variants of the
double-move instructions.
- Push/Pop Optimizer - extending it to support the `qc.` variants of the
pop instructions.
The testing is based on existing Zcmp tests, but I have put them in
separate files as some of the Zcmp tests were getting quite long.
Previously we calculated the max register id. Then converted it to the number
of registers and encoding. Then converted number of registers to stack
size. Then saved number of registers, encoding, and stack size to
MachineFunctionInfo.
This patch removes the calculation of max register id, and instead
calculates the number of registers. The encoding is removed from
MachineFunctionInfo in favor of converting the number of registers to
encoding at the time of use.
Callee saved registers should always be phyiscal registers. They are
often passed directly to other functions that take MCRegister like
getMinimalPhysRegClass or TargetRegisterClass::contains.
Unfortunately, sometimes the MCRegister is compared to a Register which
gave an ambiguous comparison error when the MCRegister is on the LHS.
Adding a MCRegister==Register comparison operator created more ambiguous
comparison errors elsewhere. These cases were usually comparing against
a base or frame pointer register that is a physical register in a
Register. For those I added an explicit conversion of Register to
MCRegister to fix the error.
Found using https://github.com/codespell-project/codespell
```
codespell RISCV --write-changes \
--ignore-words-list=FPR,fpr,VAs,ORE,WorstCase,hart,sie,MIs,FLE,fle,CarryIn,vor,OLT,VILL,vill,bu,pass-thru
```
This is a tiny change that can save up to 16 bytes of stack allocation,
which is more beneficial on RV32 than RV64.
cm.push allocates multiples of 16 bytes, but only uses a subset of those
bytes for pushing callee-saved registers. Up to 12 (rv32) or 8 (rv64)
bytes are left unused, depending on how many registers are pushed.
Before this change, we told LLVM that the entire allocation was used, by
creating a fixed stack object which covered the whole allocation.
This change instead gives an accurate extent to the fixed stack object,
to only cover the registers that have been pushed. This allows the
PrologEpilogInserter to use any unused bytes for spills. Potentially
this saves an extra move of the stack pointer after the push, because
the push can allocate up to 48 more bytes than it needs for registers.
We cannot do the same change for save/restore, because the restore
routines restore in batches of `stackalign/(xlen/8)` registers, and we
don't want to clobber the saved values of registers that we didn't tell
the compiler we were saving/restoring - for instance `__riscv_restore_0`
is used by the compiler when it only wants to save `ra`, but will end up
restoring `ra` and `s0`.
The DwarfDebug.cpp implementation expects the epilogue instructions to
have source location of last non-debug instruction after which the epilogue
instructions are inserted. The epilogue_begin is set on location of the first
FrameDestroy instruction with source line information that has been seen in
the epilogue basic block.
In the trunk, the risc-v backend sets the epilogue_begin after the epilogue has
actually begun i.e. after callee saved register reloads and the source line
information is not set on those reload instructions. This is leading to #120553
where, while debugging, breaking on or single stepping to the epilogue_begin
location will make accessing the variables from wrong place as the FP has been
restored to the parent frame's FP.
To fix that, this patch sets FrameSetup/FrameDestroy flags on the callee saved
register spill/reload instructions which is actually correct. Then the
RISCVInstrInfo::loadRegFromStackSlot uses FrameDestroy flag to identify a
reload of the callee saved register in the epilogue and copies the source
line information from insert position instruction to that reload instruction.
Requires PR #120622Fixes#120553
Use the probe loop structure to allocate vector code in the stack as
well. We add the pseudo instruction RISCV::PROBED_STACKALLOC_RVV to
differentiate from the normal loop.
Without this patch ScalableVector frame index property is used before
assignment. More precisely, let's take a look at
RISCVFrameLowering::assignCalleeSavedSpillSlots. In this function we
divide callee saved registers on scalar and vector ones, based on
ScalableVector property of their frame indexes:
```
...
const auto &UnmanagedCSI = getUnmanagedCSI(*MF, CSI);
const auto &RVVCSI = getRVVCalleeSavedInfo(*MF, CSI);
...
```
But we assign ScalableVector property several lines below:
```
...
auto storeRegToStackSlot = [&](decltype(UnmanagedCSI) CSInfo) {
for (auto &CS : CSInfo) {
// Insert the spill to the stack frame.
Register Reg = CS.getReg();
const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
TII.storeRegToStackSlot(MBB, MI, Reg, !MBB.isLiveIn(Reg),
CS.getFrameIdx(), RC, TRI, Register());
}
};
storeRegToStackSlot(UnmanagedCSI);
...
```
Due to it, list of RVV callee saved registers will always be empty.
Currently this problem doesn't appear, but if you slightly change the
code and, for example, put some instructions between scalar and vector
spills, the resulting code will be ill formed.
Enable `-fstack-clash-protection` for RISCV and stack probe for function
prologues.
We probe the stack by creating a loop that allocates and probe the stack
in ProbeSize chunks.
We emit an unrolled probe loop for small allocations and emit a variable
length probe loop for bigger ones.
