The AArch64 backend, during lowering, will convert an 64bit vector insert to a
128bit vector:
vector_insert %dreg, %v, %idx
=>
%qreg = insert_subvector undef, %dreg, 0
%ins = vector_insert %qreg, %v, %idx
EXTRACT_SUBREG %ins, dsub
This creates a bit of mess in the DAG, and the EXTRACT_SUBREG being a machine
nodes makes it difficult to simplify. This patch removes that, treating the
64bit vector insert as legal and handling them with extra tablegen patterns.
The end result is a simpler DAG that is easier to write tablegen patterns for.
Differential Revision: https://reviews.llvm.org/D144550
Address the dominating condition, the urem fold is benefit from the analytics improvements.
Fix https://github.com/llvm/llvm-project/issues/60546
NOTE: delete the calls in simplifyBinaryIntrinsic and foldICmpWithDominatingICmp
is used to reduce compile time.
Reviewed By: nikic, arsenm, erikdesjardins
Differential Revision: https://reviews.llvm.org/D144248
To make legalization easier, the operands and outputs have the same size for
these ISD Nodes. When legalizing the results in SplitVectorResult the operands
are legalized to the same size as the outputs.
The ISD Node has two output/results, therefore the legalizing functions update
both results/outputs.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D144744
SKX has an objectively faster shift than shuffle, on all other targets
the two have equal performance (with maybe a slight preference for
shifts because p5 is a more common bottleneck).
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D143786
`(xor/sub N-1, Y)` -> `(not Y)` is minorly preferable (especially for
`(sub N-1, Y)` where it saves an instruction), but isn't worth
potentially creating an extra instruction for.
So, only do the transformation if `(xor/sub N-1, Y)` has one use.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D144985
Previously not inserting the `-1` in `not Y` (`xor Y, -1`) into the
DAG. Not inserting `-1` as a DAG node comes up as a bug when doing
`(xor (shl 1, A), B)` -> `(btc A, B)`. `btc` requires `B` (dst) to be
a register.
Differential Revision: https://reviews.llvm.org/D144984
Ignoring freeze(undef) if it has multiple uses in LowerAVXCONCAT_VECTORS
causes the custom INSERT_SUBVECTOR for vector widening to be ignored.
Differential Revision: https://reviews.llvm.org/D14490
This patch adds the LLVM IR intrinsics for the following:
* uunpk (2 and 4 vectors)
* sunpk (2 and 4 vectors)
I have named the tests sve2p1-intrinsics-* because although
the instructions are added as part of the SME2 feature they
only operate on SVE vectors.
NOTE: These intrinsics are still in development and are subject to future changes.
Differential Revision: https://reviews.llvm.org/D142964
In AVRFrameLowering::spillCalleeSavedRegisters(), when a 16-bit
livein register is spilled, two PUSH instructions are generated
for the higher and lower 8-bit registers. But these two 8-bit
registers are marked as killed in the two PUSH instructions, so
any future use of them will cause a crash.
This patch fixes the above issue by adding the two sub 8-bit
registers to the livein list.
Fixes https://github.com/llvm/llvm-project/issues/56423
Reviewed By: jacquesguan
Differential Revision: https://reviews.llvm.org/D144720
`emitPrologue` may insert stack pointer adjustment in tail call optimized functions where the callee argument stack size is bigger than the caller's. In such a case, the adjustment must be taken into account when generating CFA directives.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D143618
There are a variety of cases where we want more control over the exact
instruction emitted. This commit creates a new pass to fixup
instructions after the DAG has been lowered. The pass is only meant to
replace instructions that are guranteed to be interchangable, not to
do analysis for special cases.
Handling these instruction changes in in X86ISelLowering of
X86ISelDAGToDAG isn't ideal, as its liable to either break existing
patterns that expected a certain instruction or generate infinite
loops.
As well, operating as the MachineInstruction level allows us to access
scheduling/code size information for making the decisions.
Currently only implements `{v}permilps` -> `{v}shufps/{v}shufd` but
more transforms can be added.
Differential Revision: https://reviews.llvm.org/D143787
This patch adds support for TargetExtType/target(...) representing
SPIR-V builtin types. After D135202, target(...) is the preferred way
for representing SPIR-V builtin types in LLVM IR and the only working
in the opaque pointer mode.
In order to maintain compatibility with LLVM IR generated by older
versions of Clang and LLVM/SPIR-V Translator, pointers-to-opaque-structs
denoting SPIR-V/OpenCL builtin types will be translated to equivalent
SPIR-V target extension types. This translation is only available in the
typed pointer mode (-opaque-pointers=0).
The relevant LIT tests with SPIR-V builtins were converted to use the
new target(...) notation.
Differential Revision: https://reviews.llvm.org/D144494
In post-isel code, there are cases where there were redundant copies from a source FPR to an intermediate GPR in order to copy to a destination FPR. In this patch, we identify these patterns in post-isel peephole optimization and replace them with a direct FPR-to-FPR copy.
One example for this will be the insertion of the scalar result of 'uaddlv' neon intrinsic function into a destination vector. During instruction selection phase, 'uaddlv' result is copied to a GPR, & a vector insert instruction is matched separately to copy the previous result to a destination SIMD&FP register.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D142594
This change reorders the stack up-adjustment and return value copying phases of
machine-ir generation on Aarch64. Doing so prevents a bug observed for fastcc
calls with >8 arguments, where the up-adjustment required from making that call
is placed in the wrong place relative to spill and reloading code.
See: https://github.com/llvm/llvm-project/issues/60972 for full issue
reproduction and context.
Patch contributed by Bruce Collie
Differential Revision: https://reviews.llvm.org/D144791
If we have sext_inreg(vector_extract(x)) but the top bits are not used, DAG
will try to remove the sext_inreg, using vector_extract(x) directly. This can
lead to multiple uses of both sext_inreg(vector_extract(x)) and
vector_extract(x), leading to the generation of both umov and smov extracts.
This adds a target hook to prevent that under AArch64 where the sext_inreg can
be considered free if there are multiple uses of the sext and no uses of the
vector_extract. This helps fix a small regression from D144550.
Differential Revision: https://reviews.llvm.org/D144850
Adds intrinsics for the following SME2 instructions (1, 2 & 4 vector):
- smlall
- umlall
- smlsll
- umlsll
- sumlall
- usmlall
NOTE: These intrinsics are still in development and are subject to future changes.
Reviewed By: david-arm
Differential Revision: https://reviews.llvm.org/D143278
During legalization of the SelectionDAG, some nodes are replaced with
arch-specific nodes. These may be complex nodes, where the root node no
longer corresponds to the node that should carry the extra info.
Fix the issue by copying extra info to the new node and all its new
transitive operands during RAUW. See code comments for more details.
This fixes the remaining pcsections-atomics.ll tests on X86.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D144677
Use MIMetadata() to propagate both DebugLoc and !pcsections metadata.
This fixes several of the non-native sized !pcsections tests in
pcsections-atomics.ll.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D144676
Extend pcsections-atomics.ll to exhaustively test all atomic ops up to
64 bits. This currently shows that some atomic operations do not end up
in PC sections. This will be addressed in a subsequent change.
Differential Revision: https://reviews.llvm.org/D144710
The pcsections.ll test primarily tests that the AsmPrinter produces the
right output in sections. This output is not easily covered by
update_llc_test_checks.py, and as such is hand written. This makes
maintenance rather burdensome. Instead, let's keep pcsections.ll as
simple as possible.
Move the more complex tests that primarily test that some atomic
operations end up in the PC section to pcsections-atomics.ll.
NFC.
Reviewed By: dvyukov, vitalybuka
Differential Revision: https://reviews.llvm.org/D144675
This patch adds new intrinsics for uniform dsp operations and changes the lowering for the following builtins to emit calls to the new aarch64.sve.###.u intrinsics.
svsqsub_x
svsqsub_n_x
svuqsub_x
svuqsub_n_x
svsqsubr_x
svsqsubr_n_x
svuqsubr_x
svuqsubr_n_x
Reviewed By: Paul Walker
Differential Revision: https://reviews.llvm.org/D144704
Inserting scalar result of 'uaddlv' neon intrinsic function to a destination vector currently makes use of the integer unit. Subsequent patches will eliminate the redundant use of the integer registers in a more generic way that will include this special case. This is an initial set of tests for this functionality.
Differential Revision: https://reviews.llvm.org/D143038
We use this combine in the AArch64 postlegalizer combiner, which causes this
function to query the legalizer rules for the action for an invalid opcode/type
combination (G_AND and p0). Moving the legalizer query until after the validity
check in matchHoistLogicOpWithSameOpcodeHands() fixes this.
Added: OR, SMAX, SMIN, UMAX, UMIN, ABS, SHL, SRL, SRA, MUL
Intentionally not generically using TLI.isBinOp as that causes
regressions as there are many binops that cannot combine with masked
instructions.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D143860
`{v}blend{d|ps|pd}` is preferable to `{v}blendw` so widen so that we
can match it.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D143789
