Currently the CFI offset for RVV registers are not handled entirely,
this patch add those information for either stack unwinding or
debugger to work correctly on RVV callee-saved stack object.
Depends On D154576
Differential Revision: https://reviews.llvm.org/D156846
- Those special register stores are STORE and their memory operands are
input operands instead of output ones.
Reviewers:
JDevlieghere, arsenm, yinying-lisa-li, koachan, PeimingLiu, jyknight, aartbik, matthias-springer
Reviewed By: arsenm
Pull Request: https://github.com/llvm/llvm-project/pull/88971
- With the memory operand as the destination, 'mr' variants of CFCMOV
works like STORE and their memory operands should be input operands
instead of output ones.
Reviewers: XinWang10, arsenm
Pull Request: https://github.com/llvm/llvm-project/pull/88970
This is largely a revert of commit
e81796671890b59c110f8e41adc7ca26f8484d20.
As #88029 shows, there exists hardware that only supports unaligned
scalar.
I'm leaving how this gets exposed to the clang interface to a future
patch.
This vendor extension has the same shift_add as zba, and most of the same
patterns are duplicated. Enable it here too so the configurations don't
diverge.
Following #78348, we should treat functions with an explicit section as
small, unless the section name is (or has the prefix) ".ltext".
Clang emits global initializers into a ".text.startup" section on Linux.
If we mix small/medium code model object files with large code model
object files, we'll end up mixing sections with and without the large
section flag.
Reland of #87838 with a check for non-ELF platforms in
TargetMachine::isLargeGlobalValue(), otherwise MCJIT on Windows tests
fail.
With zba, we can expand this to (add (shl X, C1), (shXadd X, X)).
Note that this is our first expansion to a three instruction sequence. I
believe this to general be a reasonable tradeoff for most architectures,
but we may want to (someday) consider a tuning flag here.
I plan to support 2^n + (2/4/8 + 1) eventually as well, but that comes
behind 2^N - 2^M. Both are also three instruction sequences.
---------
Co-authored-by: Min-Yih Hsu <min@myhsu.dev>
The former is better as a zero extend can be folded into the sll,
whereas the later currently produces a seperate zext.w due to bad
interactions with other combines.
CASE_VFMA_OPCODE_VV and CASE_VFMA_CHANGE_OPCODE_VV need to match up if we are
are to avoid "Unexpected opcode" errors, but in CASE_VFMA_CHANGE_OPCODE_VV,
CASE_VFMA_CHANGE_OPCODE_LMULS_MF2 had mistakenly been used instead of
CASE_VFMA_CHANGE_OPCODE_LMULS_MF4.
We had some instances when LLVM would not inline fixed-count memcpy and
ended up
attempting to lower it a a libcall, which would not work on AMDGPU as
the
address space doesn't meet the requirement, causing compiler crash.
The patch relaxes the threshold used for -Os/-Oz compilation so we're
always allowed
to inline memory copy functions.
This patch basically does the same thing as
https://reviews.llvm.org/D158226 for
AMDGPU.
Fix#88497.
Bug fix: Handle RVV return type in calling convention correctly.
Return values are handled in a same way as function arguments.
One thing to mention is that if a type can be broken down into
homogeneous
vector types, e.g. {<vscale x 4 x i32>, {<vscale x 4 x i32>, <vscale x 4
x i32>}},
it is considered as a vector tuple type and need to be handled by tuple
type rule.
For G_LOAD and G_STORE we want this information during regbankselect.
Today we treat load dest as integer and insert converts.
---------
Co-authored-by: Evgenii Kudriashov <evgenii.kudriashov@intel.com>
Currently neither the SPIR nor the SPIRV targets specify the AS for
globals in their datalayout strings. This is problematic because
CodeGen/LLVM will default to AS0 in this case, which produces Globals
that end up in the private address space for e.g. OCL, HIPSPV or SYCL.
This patch addresses it by completing the datalayout string.
In LoongArch psABI v2.30, the R_LARCH_ALIGN requires symbol index to
support the third parameter of alignment directive. Create symbol for
each section is redundant because they have section symbol which can
also be used as symbol index. So use section symbol directly for
R_LARCH_ALIGN.
This gives us one extra SGPR to play with. The comment suggested that it
could cause bugs, but I have tested it with Vulkan CTS with the default
wave size for compute shaders set to 32 and did not find any problems.
This PR includes:
* vadd.vv/vand.vv/vor.vv/vxor.vv
* vmseq.vv/vmsne.vv
* vmin.vv/vminu.vv/vmax.vv/vmaxu.vv
* vmul.vv/vmulh.vv/vmulhu.vv
* vwadd.vv/vwaddu.vv
* vwmul.vv/vwmulu
* vwmacc.vv/vwmaccu.vv
* vadc.vvm
There is no test change, I may add it later.
Fixes part of #64422
Reviewers: michaelmaitland, preames, lukel97, topperc, asb
Reviewed By: topperc, lukel97
Pull Request: https://github.com/llvm/llvm-project/pull/88379
We rename `TuneNoStripWSuffix` to `TunePreferWInst`.
If all the users of an instruction just use the low 32 bits, we can
convert it to its W variant.
A quick test on Coremark (`-O3 -march=rv64gc`):
| | W instructions | code size(.text) |
|--------|----------------|------------------|
| before | 302 | 12257 |
| after | 343 | 12265 |
| | +13.58% | +0.065% |
Reviewers: asb, dtcxzyw, preames, lukel97, michaelmaitland, topperc
Reviewed By: topperc, dtcxzyw
Pull Request: https://github.com/llvm/llvm-project/pull/87237
This adds `RISCVScheduleXSf.td` with `SchedWrite` definitions for all
VCIX instructions and uses it in `RISCVSchedSiFive7.td` to set default
latencies for these instructions, helping with issue
https://github.com/llvm/llvm-project/issues/83391. Of course these
default latencies cannot be accurate (since each coprocessor will have
different latencies), but this seems to be enough to avoid some of the
problematic behavior described in the bug.
In any case, this seems to be enough to help with #83391 in our internal
testing.
A subsequent discussion is how to structure the code such that it's
easier for downstream consumers of this to use `SiFive7` scheduling
model with accurate VCIX latencies. But we can probably have a separate
issue to discuss that.
Copy the pointer info, flags, alignment, AAInfo, and ranges, but let
getLoad rebuild the MMO using the scalable type used for the the new
load/store. This makes sure the LLT minimum size matches the ContainerVT
minimum size. This is important since vscale_range may have been used to
determine that the fixed vector was the exact size of a scalable vector.
Fixes#88799
I'm not sure what this was supposed do. Maybe it was for the
disassembler, but that should be managed through DecoderNamespace.
So let's remove it and simplify the code.
This is currently being implied in RISCVISAInfo.cpp. Make it explicit.
I'm planning to move all extension information to RISCVFeatures.td and
have tablegen create the tables for RISCVISAInfo.cpp. This requires
making the creation of RISCVTargetParserDef.inc in tablegen independent
of RISCVISAInfo.cpp. So we need an accurate extension list for CPUs in
tablegen.
These should be already handled by other code.
Removing the kernel code model rules right above it cause
bss_pagealigned.ll to fail by using a movabsq to get the address of a
global, haven't figured out where that code is yet.
For experimental.cttz.elts, we can use a vfirst instruction, but we need
to correct the result if input vector can be 0. cttz.elts returns the
vector length while vfirst returns -1.
When build with option -msve-vector-bits=512, the return vaule of
Subtarget->getMinSVEVectorSizeInBits() is 512;
While the MinElts is still 4 for <vscale x 4 x double> in
getNumInterleavedAccesses, so it creates invalid
llvm.aarch64.sve.ld2.sret.nxv4f64, which need be splited.
Unlikely, the related custom spilting is not supported now.
Fix https://github.com/llvm/llvm-project/issues/88247
This expansion is directly inspired by the analogous code in the x86
backend for LEA. shXadd and (this sub-case of) LEA are largely
equivalent.
This is an alternative to
https://github.com/llvm/llvm-project/pull/87105.
This expansion is also supported via the decomposeMulByConstant
callback, but restricted because of interactions with other combines
since that code runs before legalization. As discussed in the other
review, my original plan had been to support post legalization expansion
through the same interface, but that ended up being more complicated
than seems justified.
Instead, lets go ahead and do the general expansion post-legalize. Other
targets use the combine approach, and matching that structure makes it
easier for us to adapt ideas from other targets to RISCV.
https://github.com/llvm/llvm-project/pull/79940 put calls to
recomputeLiveIns into
a loop, to repeatedly call the function until the computation converges.
However,
this repeats a lot of code. This changes moves the loop into a function
to simplify
the handling.
Note that this changes the order in which recomputeLiveIns is called.
For example,
```
bool anyChange = false;
do {
anyChange = recomputeLiveIns(*ExitMBB) || recomputeLiveIns(*LoopMBB);
} while (anyChange);
```
only begins to recompute the live-ins for LoopMBB after the computation
for ExitMBB
has converged. With this change, all basic blocks have a recomputation
of the live-ins
for each loop iteration. This can result in less or more calls,
depending on the
situation.
The purpose of the COALESCER_BARRIER pseudo node is to prevent the
register coalescer from coalescing certain COPY instructions around
smstart/smstop instructions, so that we spill only the (required) FPR
register rather than the encompassing ZPR register.
The pseudos are removed in the AArch64ExpandPseudo pass. However,
because the node itself is a _use_ of a register, this occassionally
leads to redundant spills/fills, because the register allocator thinks
the virtual register is actually used before an smstart/smstop
instruction, causing it to be filled, at which points it requires
immediate spilling again to ensure it stays live over the smstart/smstop
instruction.
We can avoid that by removing the pseudo nodes right after coalescing,
but before register allocation.
- `CGBuiltin.cpp` - Switch to using
`CGM.getHLSLRuntime().get##NAME##Intrinsic()`
- `CGHLSLRuntime.h` - Add any to backend intrinsic abstraction
- `IntrinsicsSPIRV.td` - Add any intrinsic to SPIR-V.
- `SPIRVInstructionSelector.cpp` - Add means of selecting any intrinsic.
Any and All share the same behavior up to the opCode. They are only
different in vector cases.
Completes #88045
When fed constant operands we try to lower WHILE intrinsics to PTRUE
using a fixed length pattern. This is not valid for the decrementing
variants of WHILE because they construct their result predicate vector
by traversing from high->low lanes whereas the incrementing variants and
PTRUE traverse from low->high lanes.
Whilst we can still utilise PTRUE by reversing its result I figure
replacing a single WHILE with multiple SVE instructions is likely
counterproductive.
The immediate forms of SQADD and SQSUB interpret their immediate
operand as unsigned and thus effectively only support positive
immediate operands.
The original code is only wrong for the i8 cases because they
previously accepted all values, however, the new patterns enable more
uses and this I've added tests for the larger element types as well.
Following https://github.com/llvm/llvm-project/pull/68313 this patch
extends the idea to M-profile PACBTI.
The Machine Scheduler can reorder instructions within a scheduling
region depending on the scheduling policy set. If a BTI-clearing
instruction happens to partake in one such region, it might be moved
around, therefore ending up where it shouldn't.
The solution is to mark all BTI-clearing instructions as scheduling
region boundaries. This essentially means that they must not be part of
any scheduling region, and as consequence never get moved:
- PAC
- PACBTI
- BTI
- SG
Note that PAC isn't BTI-clearing, but it's replaced by PACBTI late in
the compilation pipeline.
As far as I know, currently it isn't possible to organically obtain code
that's susceptible to the bug:
- Instructions that write to SP are region boundaries. PAC seems to
always be followed by the pushing of r12 to the stack, so essentially
PAC is always by itself in a scheduling region.
- CALL_BTI is expanded into a machine instruction bundle. Bundles are
unpacked only after the last machine scheduler run. Thus setjmp and BTI
can be separated only if someone deliberately run the scheduler once
more.
- The BTI insertion pass is run late in the pipeline, only after the
last machine scheduling has run. So once again it can be reordered only
if someone deliberately runs the scheduler again.
Nevertheless, one can reasonably argue that we should prevent the bug in
spite of the compiler not being able to produce the required conditions
for it. If things change, the compiler will be robust against this
issue.
The tests written for this are contrived: bogus MIR instructions have
been added adjacent to the BTI-clearing instructions in order to have
them inside non-trivial scheduling regions.
