This pattern shows up often in media libraries. The optimization should only
kick in for O3. Currently only supports a single family of accumulation
instructions, but can easily be expanded to support additional
instructions in the future.
This patch improves DAGCombiner's handling of potential store merges by
detecting function calls between loads and stores. When a function call
exists in the chain between a load and its corresponding store, we avoid
merging these stores if the spilling is unprofitable.
We had to implement a hook on TLI, since TTI is unavailable in
DAGCombine. Currently, it's only enabled for riscv.
This is the DAG equivalent of PR #129258
With a minor fix for the build failures.
Original message:
This extension adds nine instructions, eight for non-memory-mapped devices synchronization and delay instruction.
The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/tag/Xqci-0.7.0
This patch adds assembler only support.
Co-authored-by: Sudharsan Veeravalli quic_svs@quicinc.com
This extension adds nine instructions, eight for non-memory-mapped
devices synchronization and delay instruction.
The current spec can be found at:
https://github.com/quic/riscv-unified-db/releases/tag/Xqci-0.7.0
This patch adds assembler only support.
Co-authored-by: Sudharsan Veeravalli <quic_svs@quicinc.com>
Set the default processor version to v68 when the user does not specify
one in the command line. This includes changes in the LLVM backed and
linker (lld). Since lld normally sets the version based on inputs, this
change will only affect cases when there are no inputs.
Fixes#127558
Implemented the asint16 function and added test cases for codegen, Sema,
and SPIR-V backend.
fixes https://github.com/llvm/llvm-project/issues/99184
---------
Co-authored-by: Ashley Coleman <ascoleman@microsoft.com>
This is an adapted version of arsenm's
https://github.com/llvm/llvm-project/pull/120524.
The intention of the change is to enable dead stack slot copy
elimination in StackSlotColoring for vector loads and stores. In terms
of testing, see stack-slot-coloring.mir. This has little impact on in
tree tests otherwise.
This change has a different and smaller set of test diffs then then
@arsenm's patch because I'm using scalable sizes for the LMULs, not a
single signal value. His patch allowed vector load/store pairs of
different width to be deleted, mine does not. There's also simply been a
lot of churn in regalloc behavior on these particular tests recently, so
that may explain some of the diff as well.
When generating code for functions that have `__builtin_frame_address`
calls and `noinline` attribute, prologue was not emitted correctly
leading to an assertion failure in PowerPC. The issue was due to
improper insertion of prologue for a function that contain llvm
`__builtin_frame_address`.
Shrink-wrap pass computes the save and restore points of a function.
Default points are the entry and exit points of the function. During
shrink-wrapping the frame-pointer was not honored like the stack pointer
and it was considered as a callee-saved register. This change will treat
the FP similar to SP and will insert the prolog on top the instruction
containing FP.
---------
Co-authored-by: Tony Varghese <tony.varghese@ibm.com>
APX NDD instructions may be compressed when the result is also a source.
For 8/16b instructions, this may create partial register write hazards
if a previous super-register def is within the partial reg update
clearance, or incorrect code if the super-register is not dead.
This change prevents compression when the super-register is marked as an
implicit define, which the virtual rewriter already adds in the case
where a subregister is defined but the super-register is not dead.
The BreakFalseDeps interface is also updated to add implicit
super-register defs for NDD instructions that would incur partial-write
stalls if compressed to legacy ops.
We already fold concat(extract_subvector(x,0), extract_subvector(x,numsubelts)) -> (wider lower half) extract_subvector(x,0)
This patch extends this handling for the concat(extract_subvector(x,c), extract_subvector(x,c+numsubelts)) -> (wider upper half) extract_subvector(x,c) case as well
There was a case where `normalizeNonPipelinedInstructions` didn't
schedule unpipelineable instructions correctly, which could generate
illegal code. This patch fixes this issue by rejecting the schedule if
we fail to insert the unpipelineable instructions at stage 0.
Here is a part of the debug output for `sms-unpipeline-insts3.mir`
before applying this patch.
```
SU(0): %27:gpr32 = PHI %21:gpr32all, %bb.3, %28:gpr32all, %bb.4
Successors:
SU(14): Data Latency=0 Reg=%27
SU(15): Anti Latency=1
...
SU(14): %41:gpr32 = ADDWrr %27:gpr32, %12:gpr32common
Predecessors:
SU(0): Data Latency=0 Reg=%27
SU(16): Ord Latency=0 Artificial
Successors:
SU(15): Data Latency=1 Reg=%41
SU(15): %28:gpr32all = COPY %41:gpr32
Predecessors:
SU(14): Data Latency=1 Reg=%41
SU(0): Anti Latency=1
SU(16): %30:ppr = WHILELO_PWW_S %27:gpr32, %15:gpr32, implicit-def $nzcv
Predecessors:
SU(0): Data Latency=0 Reg=%27
Successors:
SU(14): Ord Latency=0 Artificial
...
Do not pipeline SU(16)
Do not pipeline SU(1)
Do not pipeline SU(0)
Do not pipeline SU(15)
Do not pipeline SU(14)
SU(0) is not pipelined; moving from cycle 19 to 0 Instr: ...
SU(1) is not pipelined; moving from cycle 10 to 0 Instr: ...
SU(15) is not pipelined; moving from cycle 28 to 19 Instr: ...
SU(16) is not pipelined; moving from cycle 19 to 0 Instr: ...
Schedule Found? 1 (II=10)
...
cycle 9 (1) (14) %41:gpr32 = ADDWrr %27:gpr32, %12:gpr32common
cycle 9 (1) (15) %28:gpr32all = COPY %41:gpr32
```
The SUs are traversed in the order of the original basic block, so in
this case a new cycle of each instruction is determined in the order of
`SU(0)`, `SU(1)`, `SU(14)`, `SU(15)`, `SU(16)`. Since there is an
artificial dependence from `SU(16)` to `SU(14)`, which is contradict to
the original SU order, the new cycle of `SU(14)` must be greater than or
equal to the cycle of `SU(16)` at that time. This results in the failure
of scheduling `SU(14)` at stage 0. For now, we reject the schedule for
such cases.
This reverts commit 0dba5381d8c8e4cadc32a067bf2fe5e3486ae53d.
YMM rounding was removed from AVX10 whitepaper. Ref:
https://cdrdv2.intel.com/v1/dl/getContent/784343
The MINMAX and SATURATING CONVERT instructions will be removed as a
follow up.
It is an architectural requirement that there must be no outstanding GDS
instructions when an "always GDS" instruction is issued, and also that
an always GDS instruction must be allowed to complete.
Insert waits on DScnt/LGKMcnt prior to (if necessary) and subsequent to
(unconditionally) any always GDS instruction, and an additional S_NOP if
the subsequent wait was followed by S_ENDPGM.
Always GDS instructions are GWS instructions, DS_ORDERED_COUNT,
DS_ADD_GS_REG_RTN, and DS_SUB_GS_REG_RTN (the latter two as considered
always GDS as of this patch).
Currently, we use `csrr` with `vlenb` to obtain the `VLEN`, but this is
not the only option. We can also use `vsetvli` with `e8`/`m1` to get
`VLENMAX`, which is equal to the VLEN. This method is preferable on some
microarchitectures and makes it easier to obtain values like `VLEN * 2`,
`VLEN * 4`, or `VLEN * 8`, reducing the number of instructions needed to
calculate VLEN multiples.
However, this approach is *NOT* always interchangeable, as it changes
the state of `VTYPE` and `VL`, which can alter the behavior of vector
instructions, potentially causing incorrect code generation if applied
after a vsetvli insertion. Therefore, we limit its use to the
prologue/epilogue for now, as there are no vector operations within the
prologue/epilogue sequence.
With further analysis, we may extend this approach beyond the
prologue/epilogue in the future, but starting here should be a good
first step.
This feature is gurded by the `+prefer-vsetvli-over-read-vlenb` feature,
which is disabled by default for now.
The aarch64-cond-br-tuning pass transforms a CBZX instruction into a
conditional branch (B.cond). One of the by products of the
transformation is that the source instruction of the CBZX, which is an
ANDXri instruction, gets transformed into a ANDSXri instruction, however
this transformation doesn't preserve it's debug instruction number.
This patch fixes that issue.
In [1], Nikita Popov suggested that during lowering 'unreachable' insn
should not generate extra code for naked functions, and this applies to
all architectures. Note that for naked functions, 'unreachable' insn is
necessary in IR since the basic block needs a terminator to end.
This patch checked whether a function is naked function or not. If it is
a naked function, 'unreachable' insn will not generate ISD::TRAP.
[1] https://github.com/llvm/llvm-project/pull/131731
Co-authored-by: Yonghong Song <yonghong.song@linux.dev>
Stack slot coloring already works, but we didn't have coverage.
Dead stack slot copy elimination (after stack slot coloring) does not, and will
be included in an upcoming change.
Given this shuffle:
```
shufflevector <8 x i8> %0, <8 x i8> %1, <8 x i32> <i32 0, i32 4, i32 8, i32 12, i32 undef, i32 undef, i32 undef, i32 undef>
```
#127272 lowers it with a bunch of vnsrl. If we describe the result in
terms of the shuffle mask, we expect:
```
<0, 4, 8, 12, u, u, u, u>
```
but we actually got:
```
<0, 4, u, u, 8, 12, u, u>
```
for factor larger than 2. This is caused by `CONCAT_VECTORS` on
incorrect (sub) vector types. This patch fixes the said issue by
building an aggregate vector with the correct sub vector types.
Fix#132071
