For IR like this:
%icmp = icmp ult <4 x i32> %a, splat (i32 5)
%res = extractelement <4 x i1> %icmp, i32 1
where there is only one use of %icmp we can take a similar approach
to what we already do for binary ops such add, sub, etc. and convert
this into
%ext = extractelement <4 x i32> %a, i32 1
%res = icmp ult i32 %ext, 5
For AArch64 targets at least the scalar boolean result will almost
certainly need to be in a GPR anyway, since it will probably be
used by branches for control flow. I've tried to reuse existing code
in scalarizeExtractedBinop to also work for setcc.
NOTE: The optimisations don't apply for tests such as
extract_icmp_v4i32_splat_rhs in the file
CodeGen/AArch64/extract-vector-cmp.ll
because scalarizeExtractedBinOp only works if one of the input
operands is a constant.
Use a `DCI` object to actually check the DAG combine level instead of
using the type `i1` because this assumption fails on AVX512 where we
have types like `v8i1` after legalization.
Closes#117684
Initial support is just for UNPCKL(CVTPH2PS(X),CVTPH2PS(Y)) -> CVTPH2PS(UNPCKL(X,Y))
Making this more general for other shuffles/conversions will have to be done carefully as we have to handle changes in src/dst element width, so I just handled the CVTPH2PS regression case.
Fixes#83414
Fix all the places I could find that did't do this. We were already
mostly correct for FP_ROUND after
9a976f36615dbe15e76c12b22f711b2e597a8e51, but not STRICT_FP_ROUND.
Add patterns to select 16b imulzu with -mapx-feature=zu, including
folding of zero-extends of the result. IsDesirableToPromoteOp is changed
to leave 16b multiplies by constant un-promoted, as imulzu will not
cause partial-write stalls.
For IR like this:
%icmp = icmp ult <4 x i32> %a, splat (i32 5)
%res = extractelement <4 x i1> %icmp, i32 1
where there is only one use of %icmp we can take a similar approach
to what we already do for binary ops such add, sub, etc. and convert
this into
%ext = extractelement <4 x i32> %a, i32 1
%res = icmp ult i32 %ext, 5
For AArch64 targets at least the scalar boolean result will almost
certainly need to be in a GPR anyway, since it will probably be
used by branches for control flow. I've tried to reuse existing code
in scalarizeExtractedBinop to also work for setcc.
NOTE: The optimisations don't apply for tests such as
extract_icmp_v4i32_splat_rhs in the file
CodeGen/AArch64/extract-vector-cmp.ll
because scalarizeExtractedBinOp only works if one of the input
operands is a constant.
This helps ensure we lower to VPERMI2/T2 instructions that we can commute the index arg to VPERMT2/I2.
Similar to 1e31a4529244ead9f12abed524f33a48515abee2 to handle cases where the one use load appears after further folding (keep the lowerShuffleWithPERMV version as this can handle the non-VLX widening case as well).
The relevant bit from the Intel SDM for vinsertps semantics:
```
IF (SRC = REG) THEN COUNT_S := imm8[7:6] ELSE COUNT_S := 0
```
This is now taken into account.
#116419 assumed that getSHUFPDImm incorrectly hardcoded the mask size to 4 (cut+pasta typo from getV4X86ShuffleImm).
Waiting on reduced test case from @metaflow
Currently when creating a SHUFPD immediate mask, any undef shuffle elements are set to 0, which can limit options for further shuffle combining.
This patch attempts to canonicalize the mask to improve folding: first by detecting a per-lane broadcast style mask (which can allow us to fold to UNPCK instead), and second ensure any undef elements are set to an 'inplace' value to improve chances of the SHUFPD later folding to a BLENDPD (or be bypassed in a SimplifyMultipleUseDemandedVectorElts call).
This is very similar to canonicalization we already attempt in getV4X86ShuffleImm for vXi32/vXf32 SHUFPS/SHUFD shuffles.
These nodes should appear between CALLSEQ_START / CALLSEQ_END.
Previously, they could be scheduled after CALLSEQ_END because the nodes
didn't update the chain.
The change in a test is due to X86 call frame optimizer pass bailing out
for a particular call when CALLSEQ_START / CALLSEQ_END are not in the
same basic block. This happens because SEG_ALLOCA is expanded into a
sequence of basic blocks early. It didn't bail out before because the
closing CALLSEQ_END was scheduled before SEG_ALLOCA, in the same basic
block as CALLSEQ_START.
While here, simplify creation of these nodes: allocating a virtual
register and copying `Size` into it were unnecessary.
On AVX2+, support broadcasting of any element if it occurs in the bottom 128-bit subvector by shuffling the element down to element 0 and then broadcasting.
Fixes#113396
Don't demand every element of each demanded sublane - set the undemanded mask elements to UNDEF to allow simplification (usually to a VBROADCAST).
Fixes#66150
Make use of AVX2 broadcasts to splat the source integer across all lanes to simplify the per-lane byte shuffles.
Prep work to avoid a regression in the fix for #66150
This patch extends the existing fold "select(mask, extract_subvector(shuffle(x)), zero) --> extract_subvector(select(insert_subvector(mask), shuffle(x), zero))", to also handle the non-zero case.
I've put in a restriction for VPERMV3 3 vector operands shuffles to only work with the zero select as in most circumstances we are not selecting with either of the source vectors (the only case the mask instructions match).
We should be able to generalize this in the future to work with other maskable instructions, but this is a good initial improvement.
Fixes#113400
For the "select(mask, extract_subvector(shuffle(x)), zero) --> extract_subvector(select(insert_subvector(mask), shuffle(x), zero))" fold, match the zero operand inside the SelectableOp helper.
Prep work for #113400 - we will be able to relax the zero operand requirement for some target shuffles.
This is currently mostly the same as the VSHLI/VSRLI handling below, although I've kept them separate as I'm investigating adding non-uniform shift amount handling as a followup
When lowering a TLS address for an ELF target, we introduce a call to
obtain the TLS base address. So far, we do not insert CALLSEQ_START/END
markers around this call when it is generated, but use a custom inserter
to insert them in a later phase. This is problematic, since the TLS
address call can land in a CALLSEQ for another call before it is
wrapped in its own CALLSEQ. That results in nested CALLSEQs, which are
illegal and cause errors when expensive checks are enabled, e.g., in
issues #45574 and #98042.
This patch instead wraps each TLS address call in a CALLSEQ when it is
generated so that instruction selection can avoid nested CALLSEQs. This
is an alternative to PR #106965, which instead changes the custom
inserter to avoid generating CALLSEQs when the TLS address call is
already in a CALLSEQ.
This patch also effectively reverts commit
[228978c](228978c0dc),
which introduced the CustomInserter that so far added the CALLSEQ around
TLSAddrs.
Fixes#45574 and #98042.
