If an instruction is sunk into a loop then any kill flags on
operands declared outside the loop must be cleared as these
will be live for all loop iterations.
Fixes#46827
Reviewed By: MatzeB
Differential Revision: https://reviews.llvm.org/D126754
A forward abstract state can be in the special SEWLMULRatioOnly state which means we're not allowed to inspect its fields. The scalar to vector move case was mising a guard, and we'd crash on an assert. Test cases included.
According to the vector spec, mf8 is not supported for i8 if ELEN
is 32. Similarily mf4 is not suported for i16/f16 or mf2 for i32/f32.
Since RVVBitsPerBlock is 64 and LMUL is calculated as
((MinNumElements * ElementSize) / RVVBitsPerBlock) this means we
need to disable any type with MinNumElements==1.
For generic IR, these types will now be widened in type legalization.
For RVV intrinsics, we'll probably hit a fatal error somewhere. I plan
to work on disabling the intrinsics in the riscv_vector.h header.
Reviewed By: arcbbb
Differential Revision: https://reviews.llvm.org/D128286
This adds the macrofusion plumbing and support fusing LUI+ADDI(W).
This is similar to D73643, but handles a different case. Other cases
can be added in the future.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D128393
waitcnt vmcnt instructions are currently generated in loop bodies before using
values loaded outside of the loop. In some cases, it is better to flush the
vmcnt counter in a loop preheader before entering the loop body. This patch
detects these cases and generates waitcnt instructions to flush the counter.
Reviewed By: foad
Differential Revision: https://reviews.llvm.org/D115747
This reverts commit 719658d078c4093d1ee716fb65ae94673df7b22b.
Breaks a few things, see comments on https://reviews.llvm.org/D128437
There's disagreement about the best fix.
So let's keep HEAD green while discussions are happening.
In GFX11 ShaderType is determined by the hardware and should no longer
be written into bits[3:2] of the ds_ordered_count offset field.
Differential Revision: https://reviews.llvm.org/D128196
There are some codegen differences here, because presence of
bitcasts affects AMX codegen in minor ways (the bitcasts are not
always in the input IR, but may be added by X86PreAMXConfig
for example).
Differential Revision: https://reviews.llvm.org/D128424
This is an improvement to LoongArch codegen. In D127206 we combined
((x >> shamt) & shifted_mask) to bstrpick and here we do a similar
combination when certain conditions are met.
Thanks to @xen0n for reminding me.
Differential Revision: https://reviews.llvm.org/D128191
The base RA support infrastructure that only allow a specific register
class be allocated in RA pss. Since greedy RA, basic RA derived from
base RA, they all allow allocating specific register class. Fast RA
doesn't support allocating register for specific register class. This
patch is to enable ShouldAllocateClass in fast RA, so that it can
support allocating register for specific register class.
Differential Revision: https://reviews.llvm.org/D126771
The reachability queries default to "reachable" after exploring too many
basic blocks. LoopInfo helps it skip over the whole loop.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D127917
StructurizeCFG linearizes the successors of branching basic block
by adding Flow blocks to record the true/false path for branches
and back edges. This patch reduces the number of Phi values needed
to capture the control flow path by improving the basic block
ordering.
Previously, StructurizeCFG adds loop exit blocks outside of the
loop. StructurizeCFG sets a boolean value to indicate the path
taken, and all exit block live values extend to after the loop.
For loops with a large number of exits blocks, this creates a
huge number of values that are maintained, which increases
compilation time and register pressure. This is problem
especially with ASAN, which adds early exits to blocks with
unreachable instructions for each instrumented check in the loop.
In specific cases, this patch reduces the number of values needed
after the loop by moving the exit block into the loop. This is
done for blocks that have a single predecessor and single successor
by moving the block to appear just after the predecessor.
Differential Revision: https://reviews.llvm.org/D123231
We were overly conservative and required a ret statement to be dominated
completely be a single lifetime.end marker. This is quite restrictive
and leads to two problems:
* limits coverage of use-after-scope, as we degenerate to
use-after-return;
* increases stack usage in programs, as we have to remove all lifetime
markers if we degenerate to use-after-return, which prevents
reuse of stack slots by the stack coloring algorithm.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D127905
Use an IRBuilder to insert instructions in preWriteTileCfg().
While here, also remove some unnecessary bool return values.
There are some test changes because the IRBuilder folds
"trunc i16 8 to i8" to "i8 8", and that has knock-on effects on
instruction naming.
I ran into this when converting tests to opaque pointers and
noticed that this pass introduces unnecessary "bitcast ptr to ptr"
instructions.
This runs the test through -instnamer and generates test checks
using update_test_checks.py. (The previous comment indicated that
update_llc_test_checks.py was used, but I rather doubt that.)
This relies on the non-determinism fix from
fbb72530fe80a95678a7d643d7a3f5ee8d693c93,
the previous check lines have apparently been written to accomodate
that non-determinism.
Migrate some more tests that required manual fixups (e.g. due to
phi nodes), but have no codegen changes.
Also tweak some more tests that had leftover no-op constant
expressions.
Test updates were performed using:
https://gist.github.com/nikic/98357b71fd67756b0f064c9517b62a34
These are only the test updates where the test passed without
further modification (which is almost all of them, as the backend
is largely pointer-type agnostic).
This fixes the combining of constant vector GEP operands in the
optimization of MVE gather/scatter addresses, when opaque pointers are
enabled. As opaque pointers reduce the number of bitcasts between geps,
more can be folded than before. This can cause problems if the index
types are now different between the two geps.
This fixes that by making sure each constant is scaled appropriately,
which has the effect of transforming the geps to have a scale of 1,
changing [r0, q0, uxtw #1] gathers to [r0, q0] with a larger q0. This
helps use a simpler instruction that doesn't need the extra uxtw.
Differential Revision: https://reviews.llvm.org/D127733
This patch adds support for:
* Querying the PSTATE.SM state with @llvm.aarch64.sme.get.pstatesm
* Reading/writing the TPIDR2 register with new
@llvm.aarch64.sme.get.tpidr2 and @llvm.aarch64.sme.set.tpidr2
intrinsics.
Tests added here:
CodeGen/AArch64/sme-get-pstatesm.ll
CodeGen/AArch64/sme-read-write-tpidr2.ll
Differential Revision: https://reviews.llvm.org/D127957
This enables an existing transformation that when combined with an
add will emit saba/uaba instructions.
Differential Revision: https://reviews.llvm.org/D128198
This created a weird loop making the tested registers live out of the
block, which I don't think is relevant to the purpose of the
tests. This caused regressions when the validity queries are changed
to use tests based whether the use instruction was a kill. If the
register was live out for the loop, it was still live.
I guess we could still do this in a narrow case where the value loops
back, but that's most a pointlessly complex case to handle.
Remove the known limitation of the library function call folders to only
work with top-level arrays of characters (as per the TODO comment in
the code) and allows them to also fold calls involving subobjects of
constant aggregates such as member arrays.
Noticed on D128216 - if we're zeroing out vector elements of a mul/mulh result then see if we can merge the and-mask into the mul by just multiplying by zero.
Ideally we'd make this generic (similar to the existing foldSelectWithIdentityConstant?), but these cases are appearing very late, after the constants have been lowered to constant-pool loads.
The granularity of SPI_SHADER_PGM_RSRC2_PS.EXTRA_LDS_SIZE changed
in GFX11. It is now in units of 256 dwords instead of 128 dwords.
COMPUTE_PGM_RSRC2.LDS_SIZE is unaffected. It is still in units of
128 dwords.
Differential Revision: https://reviews.llvm.org/D128179
This requires us to override the isTargetCanonicalConstantNode callback introduced in D128144, so we can recognise the various cases where a VBROADCAST_LOAD constant is being reused at different vector widths to prevent infinite loops.
As a followup to D128144, this adds extract(DUP(C)) as a canonical
constant to prevent it being transformed back into a BUILD_VECTOR,
leading to an infinite loop.
