When eliding argument copies, the memory layout between a plain
store of the type and the layout of the argument lowering on the
stack must match. For multi-part argument lowerings, this is not
necessarily the case.
The code already tried to prevent this optimization for "scalarized
and extended" vectors, but the check for "extends" was incomplete.
While a scalarized vector of i32s stores i32 values on the stack,
these are stored in 8 byte stack slots (on x86_64), so effectively
have padding.
Rather than trying to add more special cases to handle this (which
is not straightforward), I'm going in the other direction and
exclude scalarized vectors from this optimization entirely. This
seems like a rare case that is not worth the hassle -- the complete
lack of test coverage is not reassuring either.
Fixes https://github.com/llvm/llvm-project/issues/63475.
Differential Revision: https://reviews.llvm.org/D154078
In certain cases, lowering negations can cause an infinite loop in SelectionDAG on X86.
The following snippet shows that behaviour:
https://godbolt.org/z/5hP45T4hY
What happens is that ADD(XOR(..., -1), 1) is detected as the two's complement and transformed into SUB(0, ...)
However, immediates can not be encoded as the LHS of a SUB on X86.
Therefore it is transformed back into an ADD/XOR pair, which is then again transformed into a SUB and so on.
In that specific case, I still think it is valid to display this as a SUB(0,...) , because it should eventually be lowered as a NEG.
Which seems better than an ADD/XOR pair.
Adding an exception to the X86 specific handling for SUBs with 0 LHS operand fixes this infinite loop.
Differential Revision: https://reviews.llvm.org/D154575
During lowering, especially of smaller vector types, we can end up with
`add (extract_subvector(zext(x), extract_subvector(zext(y))`, which can
be turned into `extract_subvector(add(zext(y), zext(x)))`, which can use
the addl AArch64 instruction. This adds some tablegen patterns for it,
along with addw where only one operand is an extract/extend and subl/subw.
Differential Revision: https://reviews.llvm.org/D153632
We are already checking for fp exceptions if VL changes, but I believe we
should also be checking for them if the mask changes as well, since that also
affects the set of active elements. From the spec:
> A vector floating-point exception at any active floating-point element sets
> the standard FP exception flags in the fflags register. Inactive elements do
> not set FP exception flags.
Note that we don't change the mask if IsMasked is true, i.e. True is masked
already, since in that case we keep the existing mask.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D154980
The fadd in these test cases is constrained and may set fflags differently
depending on the active elements (the nofpexcept flag isn't set on the node).
Therefore to preserve semantics we shouldn't change its mask.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D154979
These aren't implemented. They could be at moderate implementation
complexity. Raising an error is better than silently miscompiling.
Patching now because the patch at D155125 is a step towards using this metadata
more extensively as part of the lowering path and that will interact badly with
input variables with this annotation.
Lowering user defined variables at specific addresses would drop this error,
put them at the requested position in the frame during this pass, and then
use the same codegen that will be used for the kernel specific struct shortly.
Reviewed By: jmmartinez
Differential Revision: https://reviews.llvm.org/D155132
Stop folding vector truncations to PACKSS/PACKUS patterns prematurely - another step towards Issue #63710. We still prematurely fold to PACKSS/PACKUS if there are sufficient signbits, that will be addressed in a later patch when we remove combineVectorSignBitsTruncation.
This required ReplaceNodeResults to extend handling of sub-128-bit results to SSSE3 (or later) cases, which has allowed us to improve vXi32->vXi16 truncations to use PSHUFB.
I also tweaked LowerTruncateVecPack to recognise widened truncation source operands so the upper elements remain UNDEF (otherwise truncateVectorWithPACK* will constant fold them to allzeros/allones values).
Depends on D154635
For the cover letter of the patch-set, please checkout D154628.
This is the 8th patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154636
Depends on D154634
For the cover letter of the patch-set, please checkout D154628.
This is the 7th patch of the patch-set. This patch includes change to
vfcvt_x_f, vfcvt_xu_f, vfwcvt_x_f, vfwcvt_xu_f, vfncvt_x_f, vfncvt_xu_f
vfcvt_f_x, vfcvt_f_xu, vfncvt_f_x vfncvt_f_xu, vfncvt_f_f
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154635
Depends on D154633
For the cover letter of the patch-set, please checkout D154628.
This is the 6th patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154634
Depends on D154632
For the cover letter of the patch-set, please checkout D154628.
This is the 5th patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154633
Depends on D154631
For the cover letter of the patch-set, please checkout D154628.
This is the 4th patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154632
Depends on D154629
For the cover letter of the patch-set, please checkout D154628.
This is the 3rd patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154631
Depends on D154628
For the cover letter of the patch-set, please checkout D154628.
This is the 2nd patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154629
Depends on D152996.
This patch-set aims to add a variant for the RVV floating-point
intrinsics that controls the rounding mode (`frm`). The rounding mode
variant appends `_rm` before the policy suffix to distinguish from
those without them.
Specification PR: riscv-non-isa/rvv-intrinsic-doc#226
This is the 1st patch of the patch-set.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D154628
Depends on D152879.
Specification PR: riscv-non-isa/rvv-intrinsic-doc#226
This patch adds variant of `vfadd` that models the rounding mode control.
The added variant has suffix `_rm` appended to differentiate from the
existing ones that does not alternate `frm` and uses whatever is inside.
The value `7` is used to indicate no rounding mode change. Reusing the
semantic from the rounding mode encoding for scalar floating-point
instructions.
Additional data member `HasFRMRoundModeOp` is added so we can append
`_rm` suffix for the fadd variants that models rounding mode control.
Additional data member `IsRVVFixedPoint` is added so we can define
pseudo instructions with rounding mode operand and distinguish the
instructions between fixed-point and floating-point.
Reviewed By: craig.topper, kito-cheng
Differential Revision: https://reviews.llvm.org/D152996
TargetLowering::SimplifySetCC wants to swap the operands of a SETCC to
canonicalize the constant to the RHS. The bug here was that it did so whether
or not the RHS was already a constant, leading to an infinite loop.
rdar://111847838
Divverential revision: https://reviews.llvm.org/D155095
This reverts commit cdc633e4bc93d4bf241ecd4c29691ae065749313.
his change continues with the line of work discussed in https://discourse.llvm.org/t/riscv-transition-in-vector-pseudo-structure-policy-variants/71295.
This is analogous to other patches in the series, but with one key difference - the resulting pseudo does *not* have a policy operand. We could add one for vmerge, but the some of the multiclasses are sufficiently entwined with the mask producing arithmetic instructions that the change delta becomes unmanageable. Note that these instructions are *not* in the RISCVMaskedPseudo table, and thus the difference doesn't complicate other code. The main value of working incrementally here is that we get to eagerly cleanup the IsTA logic flowing through the post-ISEL combines.
Differential Revision: https://reviews.llvm.org/D154645
This mostly copies cases that already exist in ValueTracking, although
it skips the more complex ones. Those can be filled in as needed.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D149199
Fix a miscompilation crash where swiftself on x20 gets corrupted due
to incorrect save/restore at prologue/epilogue.
Differential Revision: https://reviews.llvm.org/D155001
TargetLowering::SimplifySetCC wants to swap the operands of a SETCC to
canonicalize the constant to the RHS. The bug here was that it did so whether
or not the RHS was already a constant, leading to an infinite loop.
rdar://111847838
Differential revision: https://reviews.llvm.org/D155095
MachineLICM pass handles inner loops only when outmost loop does not have unique
predecessor. If the loop has preheader and there is loop invariant code, the
invariant code can be hoisted to the preheader in general. This patch makes the
pass handle inner loops in general.
Differential Revision: https://reviews.llvm.org/D154205
If the operands to the mul have other uses we may be extending their
live range past a kill flag.
Reviewed By: asb, asi-sc
Differential Revision: https://reviews.llvm.org/D155046
For RISC-V, getRegisterType for fp16 returns i16. i16->fp64 extload
is considered legal because the LoadExtActions defaults to Legal
for all entries. Only fp/fp and int/int entries are changed to
Expand fore RISC-V.
This patch detects the FP-ness has changed and won't try to call
isLoadExtLegal.
Alternatively, we could add Expand for int/fp and fp/int, but that
seemed a little silly.
Fixes#63816
Reviewed By: asb, wangpc
Differential Revision: https://reviews.llvm.org/D155040
Record the SP adjustment on entry to each basic block. This is almost
always zero except on targets like ARM which can split a basic block in
the middle of a call sequence.
This simplifies PEI::replaceFrameIndices which previously had to visit
basic blocks in a specific order and had special handling for
unreachable blocks. More importantly it paves the way for an equally
simple implementation of a backwards version of replaceFrameIndices,
which is required to fully convert PrologEpilogInserter to backwards
register scavenging, which is preferred because it does not rely on
accurate kill flags.
Differential Revision: https://reviews.llvm.org/D154281
https://reviews.llvm.org/D130883 introduced MIMetadata to simplify
metadata propagation (DebugLoc and PCSections).
However, we're currently still permitting implicit conversion of
DebugLoc to MIMetadata, to allow for a gradual transition and let the
old code work as-is.
This manifests in lost !pcsections metadata for X86-specific lowerings.
For example, 128-bit atomics.
Fix the situation for X86ISelLowering by converting all BuildMI() calls
to use an explicitly constructed MIMetadata.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D154986
Arm Performance Libraries contain math library which provides
vectorized versions of common math functions.
This patch allows to use it with clang and llvm via -fveclib=ArmPL or
-vector-library=ArmPL, so loops with such calls can be vectorized.
The executable needs to be linked with the amath library.
Arm Performance Libraries are available at:
https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Libraries
Reviewed by: paulwalker-arm
Differential Revision: https://reviews.llvm.org/D154508
Currently when compiling for an execute-only target without movt then
EmitStructByval will generate a constant pool load which isn't
compatible with execute-only. Handle this by emitting tMOVi32imm,
and also simplify the existing movt handling by emitting t2MOVi32imm
or MOVi32imm.
Differential Revision: https://reviews.llvm.org/D154944
The expansion of the various MOVi32imm pseudo-instructions works by
splitting the operand into components (either halfwords or bytes) and
emitting instructions to combine those components into the final
result. When the operand is an immediate with some components being
zero this can result in pointless instructions that just add zero.
Avoid this by restructuring things so that a separate function handles
splitting the operand into components, then don't emit the component
if it is a zero immediate. This is straightforward for movw/movt,
where we just don't emit the movt if it's zero, but the thumb1
expansion using mov/add/lsl is more complex, as even when we don't
emit a given byte we still need to get the shift correct.
Differential Revision: https://reviews.llvm.org/D154943
This is a preparatory patch for extending DWARFDebugLine to properly
parse line number programs with maximum_operations_per_instruction > 1
for VLIW targets.
Add some scaffolding for handling op-index in line number programs, and
add printouts for that in the table. As this affects a lot of tests,
this is done in a separate commit to get a cleaner review for the actual
op-index implementation.
Verbose printouts are not present in many tests, and adding op-index to
those will require a bit more code changes, so that is done in the
actual implementation patch.
Reviewed By: StephenTozer
Differential Revision: https://reviews.llvm.org/D152535
Instead of checking the pointer type, check the element type of
the GEP.
Previously we ended up reusing GEP increments that were not in
expanded form, thus not respecting LSRs choice of representation.
The change in 2011-10-06-ReusePhi.ll recovers a regression that
appeared when converting that test to opaque pointers.
Changes in various Thumb tests now compute the step outside the
loop instead of using add.w inside the loop, which is LSR's
preferred representation for this target.
Documentation for TargetLowering::getShiftAmountTy says that LegalTypes
should generally be true during type legalization, so this patch does
that.
On AMDGPU the effect is that we use i32 (a sane type) instead of i64
(pointer sized type) for more shift amounts, which in turn allows more
formation of rotates and funnel shifts pre-legalization.
Differential Revision: https://reviews.llvm.org/D154960
This CL adds a new discriminator pass. Also adds a new sample profile
loading pass when MFS is enabled.
Differential Revision: https://reviews.llvm.org/D152577
More robust association between the kernels and lds struct.
Use poison instead of value() for lookup table elements introduced by dynamic lds lowering.
Extracted from D154946, new test from there verbatim. Segv fixed.
Fixes issues/63338
Fixes SWDEV-404491
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D154972
