There were several issues with this test, the most obvious being
that %strided.vec wasn't used and therefore could be deadcoded.
I've verified that the updated test still covers the code-changes
from its original patch.
This fixes a bug in 'allocateLazySaveBuffer' that led to the
buffer pointer being stored to the wrong address.
Reviewed By: david-arm
Differential Revision: https://reviews.llvm.org/D137734
This makes the existing fabs_fabs fold redundant, which
I thought was using more tablegen matching, but apparently not.
I'm not sure how to make match work with multiple opcodes. There
are a few more this could handle, but these are the ones that
legalization are more likely to introduce.
D134260/D138107 exposed that the MachineCombiner was not copying
pcsections metadata where it should. This patch switches the MIBuild
methods to use MIMetadata that can copy the debug loc and pcsections at
the same time.
Differential Revision: https://reviews.llvm.org/D138112
This adds some more scalar instructions that are both associative and
commutative to isAssociativeAndCommutative, allowing the machine
combiner to reassociate them to reduce critical path length.
Differential Revision: https://reviews.llvm.org/D134260
All instructions that can raise fp exceptions also read FPCR, with the
only other instructions that interact with it being the MSR/MRS to
write/read FPCR.
Introducing an FPCR register also requires adjusting
invalidateWindowsRegisterPairing in AArch64FrameLowering.cpp to use
the encoded value of registers instead of their enum value, as the
enum value is based on the alphabetical order of register names and
now FPCR is placed between FP and LR.
This change unfortunately means a large number of mir tests need to
be adjusted due to instructions now requiring an implicit fpcr operand
to be present.
Differential Revision: https://reviews.llvm.org/D121929
This adds an extra case to check if a vector_shuffle for a fixed-length
vector that's being lowered to SVE, is just a splat. Doing this avoids
a round trip to the stack and back for a few cases.
Reviewed By: c-rhodes
Differential Revision: https://reviews.llvm.org/D137966
This adds a new canonicalization rule to replace concats of truncated
negations with a negation of the concatenated truncates, e.g.
(concat_vectors (v4i16 (truncate (not (v4i32)))),
(v4i16 (truncate (not (v4i32)))))
->
(not (concat_vectors (v4i16 (truncate (v4i32))),
(v4i16 (truncate (v4i32)))))
Doing this allows avoiding redundant negations being emitted in
certain cases.
Reviewed By: peterwaller-arm
Differential Revision: https://reviews.llvm.org/D137433
AMDGPU legalizes i64 loads to loads of <2 x i32>, leaving the
i64 MMO with attached range metadata alone. The known bit width
was using the scalar element type, and asserting on a mismatch.
Currently, there is an assertion that limits the MUBUF instruction
with voffset used for a VGPR spill inside kernel functions when
the frame pointer is really used. It seemed more like an unwanted
limitation and hence removing the assertion.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D137892
nearbyint has the property to execute without exception.
For not modifying fflags, the patch added new machine opcode
PseudoVFROUND_NOEXCEPT_V that expands vfcvt.x.f.v and vfcvt.f.x.v between a pair
of frflags and fsflags.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D137685
The patch also added function expandVPBSWAP to expand ISD::VP_BSWAP nodes.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D137928
AMX shape should be defined before AMX intrinsics. However for below
case, the shape a.row is defined after tile load of b. If we transform
`load b` to `@llvm.x86.tileloadd64 intrinsic`, the shape dependency
doesn't meet.
```
void test_tile_dpbsud(__tile1024i a, __tile1024i b, __tile1024i c) {
__tile_dpbsud(&c, a, b);
}
```
This patch is to store the tile b to stack and reloaded it after the
def of b.row. It would cause redundant store/load, but it is simple
to avoid generating invalid IR.
The better way may hoist `def b.row` before tile load instruction,
but it seems more complicated to recursively hoist its operands.
Differential Revision: https://reviews.llvm.org/D137923
We may form a zero-stride vector load when lowering gather to strided
load. As what D137699 has done, we use `load+splat` for this form if
there is no optimized implementation.
We restrict this to unmasked loads currently in consideration of the
complexity of hanlding all falses masks.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D137931
Direct-move instructions are usually more efficient than load then store
for conversion. But direct moves are not needed when the source register
was just loaded from some address.
The pattern has already been recognized, but the source value of strict
nodes are not the first (that's the chain), but the second.
Reviewed By: shchenz
Differential Revision: https://reviews.llvm.org/D138011
We can reuse constants if we use SRL followed by AND and AND followed by SHL.
Similar was done to bitreverse previously.
Differential Revision: https://reviews.llvm.org/D138045
This reverts commit e05ce03cfa0b36e9b99149e21afcb1fc039df813.
Caused asan use-after-poison to 4 DebugInfo/AMDGPU/ tests.
Triggered in PEI::replaceFrameIndicesBackward called llvm::MachineInstr::getNumOperands
Alignment of function arguments can be increased only if we can do
this for all call sites. Therefore we do not increase it for external
functions, and now we skip functions that have address taken, to avoid
any issues with functions pointers.
Differential Revision: https://reviews.llvm.org/D135708
When --nvptx-short-ptr is set, local pointers are stored as 32-bit on
nvptx64 target.
Before this patch, arguments for a function declaration were always
emitted as b64 regardless of their address space, but they were set as
b32 for the corresponding call instruction:
.extern .func test
(
.param .b64 test_param_0
)
[...]
.param .b32 param0;
st.param.b32 [param0+0], %r1;
call.uni test, (param0);
This is not supported:
ptxas: Type of argument does not match formal parameter
'test_param_0'
Now short pointers in a function declaration are emitted as b32 if
--nvptx-short-ptr is set.
Differential Revision: https://reviews.llvm.org/D135674
This allow for better optimization later in the backend.
This fixes the remaining missed optimizations in D137717.
Depends on D137930
Differential Revision: https://reviews.llvm.org/D137947
This allow recognition of when a ptest was emitted as an any condition
and allows for extra optimization to be done later.
This addresses missing optimizations from D137716 and D137718, and
partially D137717.
Depends on D137716, D137717, D137718
Differential Revision: https://reviews.llvm.org/D137930
The issue that caused the revert has been fixed in:
44bd80751274a81c870882968ecd478b03af292a
-----
This switches Hexagon intrinsics to use the default attributes
(nosync, nofree, nocallback and willreturn). Especially willreturn
is needed to prevent optimization regressions in the future.
The only intrinsics I've excluded here are the load/store locked
intrinsics, which presumably aren't nosync.
Differential Revision: https://reviews.llvm.org/D137623
Make the stack alignment register (AP) reserved in the given function. This
will make it available everywhere in the function, and allow aligned access
to vector register spill slots.
AArch64InstrInfo::optimizePTestInstr attempts to remove a PTEST of a
predicate generating operation that identically sets flags (implictly).
When the mask is an all active of matching element size the PTEST is
currently removed. For while instructions this is correct since they
perform an implicit PTEST with an all active mask. However, for other
instructions such as compares the mask could be different.
This patch fixes this bug by only removing the PTEST if the same all
active mask is used by the predicating-generating instruction.
Reviewed By: bsmith
Differential Revision: https://reviews.llvm.org/D137718
AArch64InstrInfo::optimizePTestInstr attempts to remove a PTEST of a
predicate generating operation that identically sets flags (implictly).
When the mask is the same as the input predicate the PTEST is currently
removed. This is incorrect since the mask for the implicit PTEST
performed by the flag-setting instruction differs from the mask
specified to the explicit PTEST and could set different flags.
For example, consider
PG=<1, 1, x, x>
Z0=<1, 2, x, x>
Z1=<2, 1, x, x>
X=CMPLE(PG, Z0, Z1)
=<0, 1, x, x> NZCV=0xxx
PTEST(X, X), NZCV=1xxx
where the first active flag (bit 'N' in NZCV) is set by the explicit
PTEST, but not by the implicit PTEST as part of the compare. Given the
PTEST mask and source are the same however, first is equivalent to any,
so the PTEST could be removed if the condition is changed. The same
applies to last active. It is safe to remove the PTEST for any active,
but this information isn't available in the current optimization.
This patch fixes the bad optimization, a later patch will implement the
optimization proposed above and fix the any active case.
Reviewed By: bsmith
Differential Revision: https://reviews.llvm.org/D137717
AArch64InstrInfo::optimizePTestInstr attempts to remove a PTEST of a
predicate generating operation that identically sets flags (implictly).
When the PTEST and the predicate-generating operation use the same mask
the PTEST is currently removed. This is incorrect since it doesn't
consider element size. PTEST operates on 8-bit predicates, but for
instructions like compare that also support 16/32/64-bit predicates, the
implicit PTEST performed by the instruction will consider fewer lanes
for these element sizes and could set different first or last active
flags.
For example, consider the following instruction sequence
ptrue p0.b ; P0=1111-1111-1111-1111
index z0.s, #0, #1 ; Z0=<0,1,2,3>
index z1.s, #1, #1 ; Z1=<1,2,3,4>
cmphi p1.s, p0/z, z1.s, z0.s ; P1=0001-0001-0001-0001
; ^ last active
ptest p0, p1.b ; P1=0001-0001-0001-0001
; ^ last active
where the compare generates a canonical all active 32-bit predicate (equivalent
to 'ptrue p1.s, all'). The implicit PTEST sets the last active flag, whereas
the PTEST instruction with the same mask doesn't.
This patch restricts the optimization to instructions operating on 8-bit
predicates. One caveat is the optimization is safe regardless of element
size for any active, this will be addressed in a later patch.
Reviewed By: bsmith
Differential Revision: https://reviews.llvm.org/D137716
Precommit test containing examples where PTEST is incorrectly optimized
away. Later patches will fix incorrect optimizations.
Reviewed By: bsmith
Differential Revision: https://reviews.llvm.org/D137715
This adds a RISCVISD::ABSW to remember that we started with an i32
abs. Previously we used a DAG combine of (sext_inreg (abs)) to
delay emitting a freeze from type legalization in order to make
ComputeNumSignBits optimizations work on other promoted nodes.
This new approach always uses negw+max even if the result doesn't
need to be sign extended. This helps the RISCVSExtWRemoval pass
if the sext.w is in another basic block.
mflr is kind of expensive on Power version smaller than 10, so we should
schedule the store for the mflr's def away from mflr.
In epilogue, the expensive mtlr has no user for its def, so it doesn't
matter that the load and the mtlr are back-to-back.
Reviewed By: RolandF
Differential Revision: https://reviews.llvm.org/D137423
When the range of the unconditional branch is overflow, the indirect
branch way is used. The case when there is no scavenged register for
indirect branch needs to spill register to stack.
Reviewed By: SixWeining, wangleiat
Differential Revision: https://reviews.llvm.org/D137821
