During legalization of the SelectionDAG, some nodes are replaced with
arch-specific nodes. These may be complex nodes, where the root node no
longer corresponds to the node that should carry the extra info.
Fix the issue by copying extra info to the new node and all its new
transitive operands during RAUW. See code comments for more details.
This fixes the remaining pcsections-atomics.ll tests on X86.
v2: Optimize copyExtraInfo() deep copy. For now we assume that only
NodeExtraInfo that have PCSections set require deep copy. Furthermore,
limit the depth of graph search while pre-populating the visited set,
assuming the to-be-replaced subgraph 'From' has limited complexity. An
assertion catches if the maximum depth needs to be increased.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D144677
This is guarding a check for isTypeLegal so it should check is
LegalTypes.
Fixes PR61111.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D145139
Add tests for more complicated scratch load and store patterns.
Includes:
- sign and zero extending loads of i8 and i16 to i32 into 32-bit register
- D16 instructions that affect only high or low 16 bits of 32-bit register
- D16 sign and zero extending loads of i8 to i16 into high or low 16 bits
of 32-bit register
- D16 loads of i16 to high or low 16 bits of 32-bit register
- D16 stores of i8 and i16 from high 16 bits of 32-bit register
Differential Revision: https://reviews.llvm.org/D145081
This patch adds AArch64 CodeGen support such that the type can be passed
and returned to/from functions, and also adds support to use this type in
load/store operations and PHI nodes.
Reviewed By: paulwalker-arm
Differential Revision: https://reviews.llvm.org/D136862
Move the opcode checks to after we have already verified we found a
valid shift instruction (`0 < ShiftAmt`) in `matchUnaryPermuteShuffle`
and `lowerShuffleAsShift`.
Reviewed By: pengfei, RKSimon
Differential Revision: https://reviews.llvm.org/D143786
This patch skips redundant explicit masks of the shift count since
it is implied inside wasm shift instruction.
Differential Revision: https://reviews.llvm.org/D144619
As noticed on D144789, when we have pairs of min/max nodes we often end up with multiple comparisons which we could reuse with commuted select ops, so check to see if a suitable SETCC already exists. This also allowed us to remove a similar X86 peephole.
There are other getSETCC cases where we could safely reuse other CondCodes as well - I've been trying to think of how we could reuse this logic in SelectionDAG but haven't found anything that always works well.
An alternative would be to have a TLI callback that returns a preferred CondCode from a list of options, I've noticed this helped fpclamptosat tests on some other targets (MVE + WebAssembly), but other tests suffered.
Differential Revision: https://reviews.llvm.org/D145065
Since the redundant BTI instructions emitted by jump tables are now
removed in the ARMBranchTargets pass, the fixupBTI function is not needed
in the ARMConstantIslandPass. Some related tests are removed as well.
The relevant patch that removes the redundant BTI instructions:
https://reviews.llvm.org/D144470
Differential Revision: https://reviews.llvm.org/D145048
Default MaxDivRemBitWidthSupported to 128, so that divisions larger
than 128 bits are always expanded, without requiring additional
configuration from the target.
Note that this may still emit calls to __udivti3 on 32-bit targets,
which likely don't have an implementation of that builtin. However,
I believe this is sufficient to fix
https://github.com/llvm/llvm-project/issues/60531, because Zig must
already be defining those builtins.
Differential Revision: https://reviews.llvm.org/D144871
A LDR will implicitly zero the rest of the vector, so vector_insert(zeros,
load, 0) can use a single load. This adds tablegen patterns for both scaled and
unscaled loads, detecting where we are inserting a load into the lower element
of a zero vector.
Differential Revision: https://reviews.llvm.org/D144086
We are missing patterns to generate vector splats using LD1R. A shuffle vector
with all 0s is a vector splat if the operands are a load and undef for which
we can generate a LD1R.
Differential Revision: https://reviews.llvm.org/D145004
The AArch64 backend, during lowering, will convert an 64bit vector insert to a
128bit vector:
vector_insert %dreg, %v, %idx
=>
%qreg = insert_subvector undef, %dreg, 0
%ins = vector_insert %qreg, %v, %idx
EXTRACT_SUBREG %ins, dsub
This creates a bit of mess in the DAG, and the EXTRACT_SUBREG being a machine
nodes makes it difficult to simplify. This patch removes that, treating the
64bit vector insert as legal and handling them with extra tablegen patterns.
The end result is a simpler DAG that is easier to write tablegen patterns for.
Differential Revision: https://reviews.llvm.org/D144550
Address the dominating condition, the urem fold is benefit from the analytics improvements.
Fix https://github.com/llvm/llvm-project/issues/60546
NOTE: delete the calls in simplifyBinaryIntrinsic and foldICmpWithDominatingICmp
is used to reduce compile time.
Reviewed By: nikic, arsenm, erikdesjardins
Differential Revision: https://reviews.llvm.org/D144248
To make legalization easier, the operands and outputs have the same size for
these ISD Nodes. When legalizing the results in SplitVectorResult the operands
are legalized to the same size as the outputs.
The ISD Node has two output/results, therefore the legalizing functions update
both results/outputs.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D144744
SKX has an objectively faster shift than shuffle, on all other targets
the two have equal performance (with maybe a slight preference for
shifts because p5 is a more common bottleneck).
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D143786
`(xor/sub N-1, Y)` -> `(not Y)` is minorly preferable (especially for
`(sub N-1, Y)` where it saves an instruction), but isn't worth
potentially creating an extra instruction for.
So, only do the transformation if `(xor/sub N-1, Y)` has one use.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D144985
Previously not inserting the `-1` in `not Y` (`xor Y, -1`) into the
DAG. Not inserting `-1` as a DAG node comes up as a bug when doing
`(xor (shl 1, A), B)` -> `(btc A, B)`. `btc` requires `B` (dst) to be
a register.
Differential Revision: https://reviews.llvm.org/D144984
Ignoring freeze(undef) if it has multiple uses in LowerAVXCONCAT_VECTORS
causes the custom INSERT_SUBVECTOR for vector widening to be ignored.
Differential Revision: https://reviews.llvm.org/D14490
This patch adds the LLVM IR intrinsics for the following:
* uunpk (2 and 4 vectors)
* sunpk (2 and 4 vectors)
I have named the tests sve2p1-intrinsics-* because although
the instructions are added as part of the SME2 feature they
only operate on SVE vectors.
NOTE: These intrinsics are still in development and are subject to future changes.
Differential Revision: https://reviews.llvm.org/D142964
In AVRFrameLowering::spillCalleeSavedRegisters(), when a 16-bit
livein register is spilled, two PUSH instructions are generated
for the higher and lower 8-bit registers. But these two 8-bit
registers are marked as killed in the two PUSH instructions, so
any future use of them will cause a crash.
This patch fixes the above issue by adding the two sub 8-bit
registers to the livein list.
Fixes https://github.com/llvm/llvm-project/issues/56423
Reviewed By: jacquesguan
Differential Revision: https://reviews.llvm.org/D144720
`emitPrologue` may insert stack pointer adjustment in tail call optimized functions where the callee argument stack size is bigger than the caller's. In such a case, the adjustment must be taken into account when generating CFA directives.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D143618
There are a variety of cases where we want more control over the exact
instruction emitted. This commit creates a new pass to fixup
instructions after the DAG has been lowered. The pass is only meant to
replace instructions that are guranteed to be interchangable, not to
do analysis for special cases.
Handling these instruction changes in in X86ISelLowering of
X86ISelDAGToDAG isn't ideal, as its liable to either break existing
patterns that expected a certain instruction or generate infinite
loops.
As well, operating as the MachineInstruction level allows us to access
scheduling/code size information for making the decisions.
Currently only implements `{v}permilps` -> `{v}shufps/{v}shufd` but
more transforms can be added.
Differential Revision: https://reviews.llvm.org/D143787
This patch adds support for TargetExtType/target(...) representing
SPIR-V builtin types. After D135202, target(...) is the preferred way
for representing SPIR-V builtin types in LLVM IR and the only working
in the opaque pointer mode.
In order to maintain compatibility with LLVM IR generated by older
versions of Clang and LLVM/SPIR-V Translator, pointers-to-opaque-structs
denoting SPIR-V/OpenCL builtin types will be translated to equivalent
SPIR-V target extension types. This translation is only available in the
typed pointer mode (-opaque-pointers=0).
The relevant LIT tests with SPIR-V builtins were converted to use the
new target(...) notation.
Differential Revision: https://reviews.llvm.org/D144494
In post-isel code, there are cases where there were redundant copies from a source FPR to an intermediate GPR in order to copy to a destination FPR. In this patch, we identify these patterns in post-isel peephole optimization and replace them with a direct FPR-to-FPR copy.
One example for this will be the insertion of the scalar result of 'uaddlv' neon intrinsic function into a destination vector. During instruction selection phase, 'uaddlv' result is copied to a GPR, & a vector insert instruction is matched separately to copy the previous result to a destination SIMD&FP register.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D142594
This change reorders the stack up-adjustment and return value copying phases of
machine-ir generation on Aarch64. Doing so prevents a bug observed for fastcc
calls with >8 arguments, where the up-adjustment required from making that call
is placed in the wrong place relative to spill and reloading code.
See: https://github.com/llvm/llvm-project/issues/60972 for full issue
reproduction and context.
Patch contributed by Bruce Collie
Differential Revision: https://reviews.llvm.org/D144791
If we have sext_inreg(vector_extract(x)) but the top bits are not used, DAG
will try to remove the sext_inreg, using vector_extract(x) directly. This can
lead to multiple uses of both sext_inreg(vector_extract(x)) and
vector_extract(x), leading to the generation of both umov and smov extracts.
This adds a target hook to prevent that under AArch64 where the sext_inreg can
be considered free if there are multiple uses of the sext and no uses of the
vector_extract. This helps fix a small regression from D144550.
Differential Revision: https://reviews.llvm.org/D144850
