When checking to see if our index expressions can be converted into strided
operations, we previously gave up if the index type wasn't an exact match for
the intptrty for the address. Per gep semantics, this mismatch implies a sext
or trunc cast to the respective index type. For constants, go ahead and
evaluate that cast instead of giving up.
Note that the motivation of this is mostly test cleanup. We canonicalize at IR
such that the gep index will match the intptrty. This is mostly useful so that
we can write both RV32 and RV64 tests from the same source. Its also helpful in
preventing confusion - I've stumbled across this at least four times now and
wasted time each one.
Note: The test change for scatters unit stride cases contains a minor
regression for rv32 and 64 bit indices. This is an artifact of order in which
changes are landing. This will be addressed in a near future change for all
configurations.
The Z#_HI register definitions were created during the very early SVE
enablement work and before the SVE calling convention was locked in.
As they look entirely unused, they need to go.
The intrinsic uses ImmArg so TargetConstant would be consistent
with how other intrinsics are handled.
This hides the constants from type legalization so we can remove
the promotion support.
isel patterns are updated accordingly.
This is a fix for a subset of legalization problems around 64 bit indices on
rv32 targets. For RV32+V, we were using the wrong mask type for the manual
truncation lowering for fixed length vectors. Instead, just use the generic
TRUNCATE node, and let it be lowered as needed.
Note that legalization is still broken for rv32+zve32. That appears to be
a different issue.
It maybe emit the .option directive without any follow up. Only emit the .option push/pop when there are supported extension difference between function and module.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D159399
We can get `BlockAddress` in user code via `Labels as Values` so
we should be able to merge the access to `BlockAddress`.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D159429
For RVV, If we want to perform an i8 or i16 element-wise vector
arithmetic right shift in the upper C/C++ program, the value to be
shifted would be first sign extended to i32, and the shift amount would
also be zero_extended to i32 to perform the vsra.vv instruction, and
followed by a truncate to get the final calculation result, such pattern
will later expanded to a series of "vsetvli" and "vnsrl" instructions
later, this is because the RVV spec only support 2 * SEW -> SEW
truncate. But for vector, the shift amount can also be determined by
smin (Y, ScalarSizeInBits(Y) - 1)). Also, for the vsra instruction, we
only care about the low lg2(SEW) bits as the shift amount.
- Alive2: https://alive2.llvm.org/ce/z/u3-Zdr
- C++ Test cases : https://gcc.godbolt.org/z/q1qE7fbha
The non-constant bit/bif/bsp already work through tablegen patterns, this
patch handles the constant case, mirroring the basic support for
`or(and(X, C), and(Y, ~C))` from ISel tryCombineToBSL. BSP gets expanded
to either BIT, BIF or BSL depending on the best register allocation.
G_BIT can be replaced with G_BSP as a more general alternative.
This patch adjusts the legality check for riscv to use `cpop/cpopw` since `isOperationLegal(ISD::CTPOP, MVT::i32)` returns false on rv64gc_zbb.
Clang vs gcc: https://godbolt.org/z/rc3s4hjPh
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D156390
If the data type is larger than e16, and the requires more than LMUL1 register
class, prefer the use of vrgatherei16. This has three major benefits:
1) Less work needed to evaluate the constant for e.g. vid sequences. Remember
that arithmetic generally scales lineary with LMUL.
2) Less register pressure. In particular, the source and indices registers
*can* overlap so using a smaller index can significantly help at m8.
3) Smaller constants. We've got a bunch of tricks for materializing small
constants, and if needed, can use a EEW=16 load.
If we have a gather or a scatter whose index describes a permutation of the
lanes, we can lower this as a shuffle + a unit strided memory operation. For
RISCV, this replaces a indexed load/store with a unit strided memory operation
and a vrgather (at worst).
I did not bother to implement the vp.scatter and vp.gather variants of these
transforms because they'd only be legal when EVL was VLMAX. Given that, they
should have been transformed to the non-vp variants anyways. I haven't checked
to see if they actually are.
Some instruction selection patterns required for ALU GPR instructions have
already been automatically imported from existing TableGen descriptions -
this patch simply adds testing for them. The first of the GIComplexPatternEquiv
definitions required to select the shiftMaskXLen ComplexPattern has been added.
Some instructions require special handling due to i32 not being a legal
type on RV64 in SelectionDAG so we can't reuse SelectionDAG patterns.
Co-authored-by: Lewis Revill <lewis.revill@embecosm.com>
Reviewed By: nitinjohnraj
Differential Revision: https://reviews.llvm.org/D76445
In D154687, we added a transform to narrow indexed load/store indices of the
form (shl (zext), C). We can move this into a generic transform over the
target independent nodes instead, and pick up the fixed vector cases with no
additional work required. This is an alternative to D158163.
Performing this transform points out that we weren't eliminating zero_extends
via the the generic DAG combine. Adjust the (existing) callbacks so that we
do.
This change *removes* the existing transform on the target specific intrinsic
nodes. If anyone has a use case this impacts, please speak up.
Note: Reviewed as part of a stack of changes in PR# 66405.
If the index type is greater or equal to XLEN, then signed and unsigned
are the same. Canonacalize towards unsigned to simplify upcoming transform.
Note: Reviewed as part of a stack of changes in PR# 66405.
We have the analogous case in the single insert path. The reasoning here is that if the original VL fits in LMUL1, we'd prefer to clobber a few extra dead lanes than to force two VL toggles. VTYPE toggles are generally cheaper than VL toggles.
The indirect lowering hinders the outliner's ability to see that
sequences are in fact common, since the sequence similarity is rendered
opaque by the register callee. The size savings from making them
indirect seems to be dwarfed by the outliner's savings from
de-duplication.
rdar://115178034
rdar://115459865
With both +sve and +dotprod, and a scalable vecreduce(sext) we could attempt to
access the number of elements of a scalable vector. Guard against this for now,
until scalable dotprod are properly supported.
LLVM fails to build on 32-bit Solaris/SPARC: several programs fail to
link due to undefined references to `__multi3`. This reference is from
`lib/libLLVMScalarOpts.a(LoopStrengthReduce.cpp.o)`. However, This
function exists neither in the 32-bit `libgcc.a` nor in
`libclang_rt.builtins-sparc.a`. It's only defined in their 64-bit
counterparts.
The same issue affects several 32-bit targets, e.g. 32-bit PowerPC as
described in Issue #54460. The fix is the same: inhibit the libcall for
32-bit compilations. This patch does just that, regenerating the
affected testcases. It allows the build to complete.
Tested on `sparc-sun-solaris2.11`.
Currently clang's medium code model treats all data as large, putting them in a large data section and using more expensive instruction sequences to access them.
Following gcc's -mlarge-data-threshold, which allows putting data under a certain size in a normal data section as opposed to a large data section. This allows using cheaper code sequences to access some portion of data in the binary (which will be implemented in LLVM in a future patch).
And under the medium codel mode, only put data above the large data threshold into large data sections, not all data.
Reviewed By: MaskRay, rnk
Differential Revision: https://reviews.llvm.org/D149288
This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.
This matches other nearby enums.
For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::
The ampere1 scheduling model uses IsCheapLSL predicates for ADDXri and
ADDWrr instructions, which only have 3 operands. In attempting to check
that the third is a shift, the predicate can attempt to access an out of
bounds operand, hitting an assert. This splits the rr/ri instructions
(which can never have shifts) from the rs/rx instructions to ensure they
both work correctly. Ampere1Write_1cyc_1AB was chosen for the rr/ir
instructions to match the cheap case.
This also sets CompleteModel = 0 for the ampere1 scheduling model, as at
runtime under debug it will attempt to check that as well as all
instructions having scheduling info, there is information for each
output operand.
DefIdx 1 exceeds machine model writes for
renamable $w9, renamable $w8 = LDPWi renamable $x8, 0
(Try with MCSchedModel.CompleteModel set to false)incomplete machine
model
On some AArch64 cores, including Ampere's ampere1 and ampere1a
architectures, load and store pair instructions are faster compared to
simple loads/stores only when the alignment of the pair is at least
twice that of the individual element being loaded.
Based on that, this patch introduces four new subtarget features, two
for controlling ldp and two for controlling stp, to cover the ampere1
and ampere1a alignment needs and to enable optional fine-grained control
over ldp and stp generation in general. The latter can be utilized by
another cpu, if there are possible benefits
with a different policy than the default provided by the compiler.
More specifically, for each of the ldp and stp respectively we have:
- disable-ldp/disable-stp: Do not emit ldp/stp.
- ldp-aligned-only/stp-aligned-only: Emit ldp/stp only if the source
pointer is aligned to at least double the alignment of the type.
Therefore, for -mcpu=ampere1 and -mcpu=ampere1a
ldp-aligned-only/stp-aligned-only become the defaults, because of the
benefit from the alignment, whereas for the rest of the cpus the default
behaviour of the compiler is maintained.
The stores created when passing operands via memory don't typically
maintain the chain, because they can be done in any order. Instead,
a new chain is created based on all collated stores. SVE parameters
passed via memory don't follow this idiom and try to maintain the
chain, which unfortunately can result in them being incorrectly
deadcoded when the chain is recreated.
This patch brings the SVE side in line with the non-SVE side to
ensure no stores become lost whilst also allowing greater flexibility
when ordering the stores.
We apparently somehow had lowering for the STRICT nodes without any handling
for the normal operations. This makes sure we support the LRINT and LROUND
intrinsics for fp16 when +fullfp16 is not present.
The instruction of ISD::FMINNUM/FMAXNUM should be legal if HasFPARMv8 &&
HasNEON.
For the combination of armv7+fp-armv8, armv7 imply the feature HasNEON
on, and fp-armv8 matchs the feature HasFPARMv8, so it is legal
Fixes https://github.com/llvm/llvm-project/issues/65820
reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)
This reverts commit ee643b706be2b6bef9980b25cc9cc988dab94bb5.
Fix up build failures in targets I missed in #66003
Kept as 3 commits for reviewers to see better what's changed. Will
squash when
merging.
- reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)
- fix all the targets I missed in #66003
- fix off by one found by llvm/test/CodeGen/SystemZ/inline-asm-addr.ll
This patch reverts commit 7614ba0a5db8 to optimize VPERM when one of its
vector operands is XXSWAPD, similar to XXPERM. It also reorganizes the
little-endian swap code on LE, swapping the vector operand after
adjusting the mask operand. This ensures that the vector operand is
swapped at the correct point in the code, resulting in a valid
constant pool for the mask operand.
Reviewed By: stefanp
Differential Revision: https://reviews.llvm.org/D149083
This reverts commit 2ca4d136124d151216aac77a0403dcb5c5835bcd.
Also revert the followup, "[InlineAsm] fix botched merge conflict resolution"
This reverts commit 8b9bf3a9f715ee5dce96eb1194441850c3663da1.
There were SystemZ and Mips build errors, too many to fix forward.
