Summary:
This is to avoid using the callee saved registers for the return address
of the tail call return instruction.
Reviewers:
arsenm, cdevadas
Differential Revision:
https://reviews.llvm.org/D147096
`unpckps` has the same performance as `unpckpd` (only port5) wereas
`unpckdq` can run on p15 on some newer architectures.
`unpckdq` is in the integer domain, so only do the transform if the
target has no bypass delay on shuffles (SKL+).
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D147729
`unpckqdq` seems to be treated as a shuffle from bypass delay
perspective (which makes sense it appears to have shared shuffle units
for all micro-arch).
`unpckqdq` is slightly preferable to `shufpd` as it saves 1-byte of
code size and can be used to replace the micro-fused `rm` version. So,
if the target has no bypass delay, we should do `unpckpd` ->
`unpckqdq` instead of `shufpd.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D147728
We shouldn't do the transformation if we either have bypass delay OR
the new opcode has worse performance. Previous code was incorrectly
using AND.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D147727
1) Add tests for `unpckps`.
2) Add explicit test for fast shuffles (ICX+) but WITH bypass delay.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D147726
We add a field `IsOutlined` to indicate whether a MachineFunction
is outlined and set it true for outlined functions in MachineOutliner.
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D146191
The test was added in D69953, but since D67794 landed a bit later it no
longer catches the bug. Update the test to be representative again.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D146934
GlobalISel happens to insert some constant materializes before SI_END_CF
in one test. These need to be excluded from AliveBlocks since they
are defined in the original block and used in the split block,
so they aren't fully alive through either block.
The case where the value defined in the first block which was originally used
in a later block is still broken.
Avoids a verifier error in a future patch.
Outlining isn't always a win when the saved instruction count is >= 1.
The overhead of representing a new function in the binary depends on
exception metadata and alignment. So parameterize this for local tuning.
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D136774
set_inactive is actually a kind of operation that is passing certain
value from active threads to inactive threads. In later WWM operation,
the activated threads which were disabled before would read such
values passed to them by set_inactive operation. So I think the
set_inactive is a convergent operation.
Differential Revision: https://reviews.llvm.org/D147683
The set_inactive was originally inside a if branch, but was incorrectly
hoisted out of the loop. So the lanes that were disabled later after the
if() branch inside the loop were not set correctly. It should stay in
the if branch.
Differential Revision: https://reviews.llvm.org/D147682
The result of sub + setcc is 0 or 1 for all bits.
The sra instruction get the same result.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147538
Note that technically both M68000/010 can use M68881, despite the fact
that usually only M68020 and newer ISAs are equipped with M68881/2.
M68040 and newer processors have builtin M68882.
Differential Revision: https://reviews.llvm.org/D147479
Loads of fixed length vectors with irregular element counts are
sometimes emitted as a scalar load + scalar_to_vector.
Previously the scalar_to_vector wasn't legal and so was scalarized
further. This patch handles it by lowering it to a vmv.s.x.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147608
I was investigating why getMemoryOpCost was so expensive for these fixed length vector types, so I've added these tests to see what was going on.
It looks like there's some room for codegen improvement before we adjust the cost model.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147607
FCMP may use ISD::SETNE when nnan, we don't want to end up with cases where we mismatch signed zeros etc.
Thanks to @pengfei for the test case from D147688
When optimizing vmv.s.x/vmv.v.x's of scalar loads, if VL is known to be
1 then we don't need to perform a stride of x0, and can just do a
regular load.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147609
DAG::SplitVector only works with vectors with even numbers of elements, when splitting vectors with large (illegal) element widths, we are likely to split down to <1 x iXXX>.
In such cases, pre-bitcast to a <X x i64> type to ensure splitting will always succeed.
Thanks to @alexfh for identifying this.
If we're inserting a fixed length subvector into a fixed length vector,
then we can use a tail agnostic policy as long as we're inserting up to
or past the end of the main vector.
I.e., because we're overwriting all of the main vector's tail elements,
and we don't care what the elements after that are.
As noted by Philip in https://reviews.llvm.org/D146711#4220341
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147347
I personally find it handy to be able to run the check-llvm-codegen-riscv-rvv target to test anything that involves vector codegen, so this patch moves some more vector-based test cases into that directory to increase its coverage.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147644
The 'ELPM' instruction has three forms:
--------------------------
| form | feature |
| ----------- | -------- |
| ELPM | hasELPM |
| ELPM Rd, Z | hasELPMX |
| ELPM Rd, Z+ | hasELPMX |
--------------------------
The second form is always used in the expansion of pseudo instructions
LPMWRdZ/ELPMWRdZ. But for devices without ELPMX and with only ELPM,
only the first form can be used.
Reviewed By: aykevl, Miss_Grape
Differential Revision: https://reviews.llvm.org/D141264
This is a follow up D147507 which removed the prior transformation to
`shufps` which was incorrect as the mask was for 64-bit double
elements, not 32-bit float elements. Using `shufpd` for the
replacement, however, preserves the mask semantics and has the same
benefits as `shufps`.
Reviewed By: pengfei, RKSimon
Differential Revision: https://reviews.llvm.org/D147541
The inverse ballot intrinsic takes in a boolean mask for all lanes and
returns the boolean for the current lane. See SPIR-V's
`subgroupInverseBallot()` in the [[ https://github.com/KhronosGroup/GLSL/blob/master/extensions/khr/GL_KHR_shader_subgroup.txt | GL_KHR_shader_subgroup extension ]].
This allows decision making via branch and select instructions with a manually
manipulated mask.
Implemented in GlobalISel and SelectionDAG, since currently both are supported.
The SelectionDAG required pseudo instructions to use the custom inserter.
The boolean mask needs to be uniform for all lanes.
Therefore we expect SGPR input. In case the source is in a
VGPR, we insert one or more `v_readfirstlane` instructions.
Reviewed By: nhaehnle
Differential Revision: https://reviews.llvm.org/D146287
This constructs a proper memory operand for these intrinsics.
Segment load/store will be added in a separate patch.
Reviewed By: kito-cheng
Differential Revision: https://reviews.llvm.org/D147119
This patch is to update pr61315.ll what was needed as part of
D146632 and caused build failures.
Reviewed By: stefanp
Differential Revision: https://reviews.llvm.org/D147675
Also the patch loose the fixed vector contraint in llvm/lib/IR/Verifier.cpp.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147380
This patch is to fix the xxperm vector operand swap condition so that the
single-use operand is in V2 to prevent copying, it also fixes the subtarget
condition to exploit the xpperm.
Reviewed By: stefanp
Differential Revision: https://reviews.llvm.org/D146632
Implement emission of DWARF CFI instructions for MSP430. This includes descriptions of stack frame layout and location of callee-saved registers that could be used for backtracing.
Differential Revision: https://reviews.llvm.org/D146966
This was assuming all register operands were assigned to physical registers.
This should ignore the operands which weren't assigned in this run.
Fixes#61134
Pass disabled since approximately D104962 for miscompiling openmp
The functions under ReplaceConstant miscompile phis as noted in D112717 and
have no users in tree other than the disabled pass. It seems likely it has no
users out of tree.
Deletes the test cases associated with the disabled pass as well.
Reviewed By: rampitec
Differential Revision: https://reviews.llvm.org/D147586
Since the same encoded offset is used for both the vector
select offset and the address offset we have to spot two
patterns simulatenously in the ldr/str intrinsic inputs, i.e.
vector select = base + off
address = base + (off * VL)
whereas currently we only look for the address pattern. I
don't think this is possible in tablegen, so I suspect we'll
have to do this manually as part of lowering or as a target
DAG combine. For now, I've removed these tablegen patterns
so that we at least do the correct thing even if the code
quality isn't great.
I've also changed some of the ldr/str tests to pass in the
same vector select pattern (base + off) as the address
pattern.
Differential Revision: https://reviews.llvm.org/D147433
Two tests still had opaque-pointers=0:
(1) llvm/test/CodeGen/Hexagon/addrmode.ll
(2) llvm/test/CodeGen/Hexagon/swp-epilog-phi7.ll
Deleted (1) since it no longer exercised the original scenario, modified
(2) to reflect codegen changes.
This fixes https://github.com/llvm/llvm-project/issues/61928
