47010 Commits

Author SHA1 Message Date
Piyou Chen
f1c4241fb6 [RISCV] Add new pass to transform undef to pseudo for vector values.
RISC-V vector instruction has register overlapping constraint for certain
instructions, and will cause illegal instruction trap if violated, we use
early clobber to model this constraint, but it can't prevent register allocator
allocated same or overlapped if the input register is undef value, so convert
IMPLICIT_DEF to temporary pseudo could prevent that happen, it's not best way
to resolve this. Ideally we should model the constraint right, but before we
model the constraint right, it's the approach to prevent that happen.

See also: https://github.com/llvm/llvm-project/issues/50157

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D129735
2023-02-14 19:42:44 -08:00
Piyou Chen
a17bfbad63 [RISCV] precommit test for D129735
Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D137763
2023-02-14 19:36:36 -08:00
WANG Xuerui
2b8cb7d87f [LoongArch] Make use of addu16i.d for adds with suitable immediates
Ideally `addu16i.d` could be paired with `{ld,st}ptr` for faster memory
accesses with 32-bit-aligned offsets (it was designed for this purpose),
but it would require more work and the original use case (GP-relative
accesses) does not exist any more with the current LoongArch psABI.

It could still be used for accelerating additions of certain constants
though, which is what this patch intends to do.

Reviewed By: SixWeining, gonglingqin

Differential Revision: https://reviews.llvm.org/D143710
2023-02-15 09:16:14 +08:00
WANG Xuerui
dcf9c60a53 [LoongArch] Add baseline tests for addu16i.d codegen. NFC
Reviewed By: gonglingqin

Differential Revision: https://reviews.llvm.org/D143846
2023-02-15 09:16:14 +08:00
Noah Goldstein
42e11a6ea3 Add transform (and/or (icmp eq/ne (A, C)), (icmp eq/ne (A, -C))) -> (icmp eq/ne (ABS A), ABS(C))
This can be beneficial if there is a fast `ABS` (For example with X86
`vpabs`) or if there is a dominating ABS(A) in the `DAG`.

Note `C` is constant so `ABS(C)` is just a constant.

Alive2 Links:
EQ: https://alive2.llvm.org/ce/z/829F-c
NE: https://alive2.llvm.org/ce/z/tsS8bU

Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D142601
2023-02-14 18:59:04 -06:00
Noah Goldstein
abf6692f95 Tests for (and/or (icmp eq/ne A, C), (icmp eq/ne A, -C)) <--> (icmp eq/ne (ABS A), ABS(C)); NFC
Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D142600
2023-02-14 18:59:04 -06:00
Noah Goldstein
8b5c390247 Transform (icmp eq/ne Abs(A), Pow2) -> (and/or (icmp eq/ne A,Pow2), (icmp eq/ne A,-Pow2))
Only if Abs(A) has one use, in which case the `(and/or (icmp eq/ne
A,Pow2), (icmp eq/ne A,-Pow2))` can be optimized in
`DAGCombiner::foldAndOrOfSETCC`.

Alive Links:
EQ: https://alive2.llvm.org/ce/z/gTxSgV
NE: https://alive2.llvm.org/ce/z/MUf57Y

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D142345
2023-02-14 18:59:04 -06:00
Noah Goldstein
54a9e992c8 Add Transform for (and/or (eq/ne A,Pow2),(eq/ne A,-Pow2))->(eq/ne (and (and A,Pow2),~(Pow2*2)), 0)
In many instances this can be preferable if the `icmp` -> `i1` cannot be
done in one instruction (such as X86 for scalars).

At the moment guarded behind `TLI.isDesirableToCombineLogicOpOfSETCC`.

alive2 links:
https://alive2.llvm.org/ce/z/nLm5sN
https://alive2.llvm.org/ce/z/moEcyE

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D142344
2023-02-14 18:59:04 -06:00
Noah Goldstein
e29c439323 Add tests for folding (and/or (icmp eq/ne A, Pow2), (icmp eq/ne A, -Pow2)); NFC
Differential Revision: https://reviews.llvm.org/D142343
2023-02-14 18:59:03 -06:00
Noah Goldstein
f3732c2b18 Transform vector SET{LE/ULT/ULE} -> SETLT and SET{GE/UGT/UGE} -> SETGT if possible
SETLT and SETGT can use `{v}pcmpgt` directly whereas the other SETCC
variants require some other instructions as well. On AVX512, which has
vector comparisons for all SETCC variants, this can still be
preferable if the destination is a vector. And if the destination is a
mask, its the same performance.

The transform for unsigned SETCC takes place if we know from
`KnownBits` that LHS/RHS have the same sign.

The transform for LE/GE -> LT/GT takes place if LHS/RHS is constant
and we can inc/dec all elements in the operand without overflowing
(both signed and unsigned).

Alive2 Links (on i8 so they don't timeout):
sge_s:   https://alive2.llvm.org/ce/z/rMPt9_
sge_s_2: https://alive2.llvm.org/ce/z/G74Mhs
sge_u:   https://alive2.llvm.org/ce/z/PTWARM
sge_u_2: https://alive2.llvm.org/ce/z/L9dsNn
sgt_s:   https://alive2.llvm.org/ce/z/q2CHEK
sgt_u:   https://alive2.llvm.org/ce/z/YPLnZ8
sle_s:   https://alive2.llvm.org/ce/z/HyYhQ_
sle_s_2: https://alive2.llvm.org/ce/z/ck6NkT
sle_u:   https://alive2.llvm.org/ce/z/tyF_wN
sle_u_2: https://alive2.llvm.org/ce/z/et8t98
slt_s:   https://alive2.llvm.org/ce/z/oCP43b
slt_u:   https://alive2.llvm.org/ce/z/EpLLPx
uge_s:   https://alive2.llvm.org/ce/z/rqSDwi
uge_s_2: https://alive2.llvm.org/ce/z/67UTXu
uge_u:   https://alive2.llvm.org/ce/z/yBNG9C
uge_u_2: https://alive2.llvm.org/ce/z/UhHYc_
ugt_s:   https://alive2.llvm.org/ce/z/tY9va4
ugt_u:   https://alive2.llvm.org/ce/z/F9zeAT
ule_s:   https://alive2.llvm.org/ce/z/1MNgka
ule_s_2: https://alive2.llvm.org/ce/z/oiS7Ls
ule_u:   https://alive2.llvm.org/ce/z/8DveC3
ule_u_2: https://alive2.llvm.org/ce/z/jGp2M7
ult_s:   https://alive2.llvm.org/ce/z/chzfwP
ult_u:   https://alive2.llvm.org/ce/z/Jj_JYu

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D142254
2023-02-14 18:59:03 -06:00
Jake Egan
08533f8b86 Revert "[CGP] Add generic TargetLowering::shouldAlignPointerArgs() implementation"
These commits are causing a test-suite build failure on AIX. Revert for now for time to investigate.
https://lab.llvm.org/buildbot/#/builders/214/builds/5779/steps/9/logs/stdio

This reverts commit bd87a2449da0c82e63cebdf9c131c54a5472e3a7 and 4c72266830ffa332ebb7cf1d3bbd6c56d001fa0f.
2023-02-14 15:20:06 -05:00
Stanislav Mekhanoshin
12b4f9e2af [AMDGPU] Do not apply schedule metric for regions with spilling
D139710 has added a metric to increase schedule's ILP while
staying within the same occupancy. Do not bother to apply this
metric to a region which is known to have spilling, it may result
in spilling to reappear after the previous stage and will do no
good if we already spilling anyway. It may also reduce compile
time a bit for such regions.

Fixes: SWDEV-377300

Differential Revision: https://reviews.llvm.org/D143934
2023-02-14 12:16:46 -08:00
Manolis Tsamis
d4012bc43f [RISCV] Add vendor-defined XTheadMAC (multiply-accumulate) extension
The vendor-defined XTHeadMAC (no comparable standard extension exists
at the time of writing) extension adds multiply accumulate instructions.

It is supported by the C9xx cores (e.g., found in the wild in the
Allwinner D1) by Alibaba T-Head.

The current (as of this commit) public documentation for this
extension is available at:
  https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf

Support for these instructions has already landed in GNU Binutils:
  https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=4041e11db3ec3611921d10150572a92689aa3154

Co-authored-by: Philipp Tomsich <philipp.tomsich@vrull.eu>

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D143847
2023-02-14 20:25:47 +01:00
Hassnaa Hamdi
bf987e108f [AArch64][SME]: Custom lower select and fp_extend for streaming SVE
Custom lower select/fp_extend and update related testing files

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D143434
2023-02-14 16:03:34 +00:00
Hassnaa Hamdi
fefe729f0f [AArch64][SME]: Custom-lower SIGN_EXTEND_INREG for streaming SVE
Custom lower SIGN_EXTEND_INREG and udpate related tests.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D143433
2023-02-14 15:35:37 +00:00
Hassnaa Hamdi
9c14132d7b [AArch64][SME]: Add missing Ops that need custom-lowering in streaming mode.
Add missing Ops and update related testing files.

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D141595
2023-02-14 15:00:25 +00:00
Matt Arsenault
09dd4d870e DAG: Remove hasBitPreservingFPLogic
This doesn't make sense as an option. fneg and fabs are bit
preserving by definition. If a target has some fneg or fabs
instruction that are not bitpreserving it's incorrect to lower
fneg/fabs to use it.
2023-02-14 10:25:24 -04:00
Matt Arsenault
f3c008ca77 DAG: Relax foldBitcastedFPLogic conditions
Requiring a bitcast to exist was unhelpful. The most basic cases
are always going to be a CopyFromReg or load, so they would need
a new cast inserted. Don't require a bitcast if it's a free
operation. I don't think this logic makes particularly much sense
(it seems to be imparting special interpretation of bitcast), but
this needs to be in sync with foldSignChangeInBitcast.

We should also get rid of this hasBitPreservingFPLogic hook. fabs/fneg
are bitpreserving or incorrectly implemented, so this should just be a
regular legality check.
2023-02-14 07:59:10 -04:00
David Spickett
94676cf8a1 [llvm][AArch64] Fix an interaction of SLS and BTI after a returns twice call
This fixes the combination of two things:
* Placing a BTI after calls to a returns twice function like setjmp.
  This allows the setjmp to return with a br instead of a ret.
* Straight line speculation mitigations that replace BLR with a BL
  to a thunk that does the mitigation, and then goes to the original
  target.

Originally I marked AArch64call_bti as requiring that SLS mitigation
be disabled. This caused a crash when you tried to codegen with both.
Since CALL_BTI tried to match with AArch64call_bti but could not.

This change does 2 things:
* Follow the pattern set by AArch64call and add 2 patterns for
  AArch64call_bti. One with no IP (interprocedural) registers,
  and one with. For SLS mitigation on and off respectively.
* Modify the sls hardening pass to iterate through bundled
  instructions, as the AArch64 KCFI pass does.

Since there is a 1:1 replacement of the BLR with a BL,
the bundle remains intact. This is checked with an MIR test.

The ir -> asm testing is updated to add runs with the sls
mitigation enabled.

Reviewed By: kristof.beyls, pzheng

Differential Revision: https://reviews.llvm.org/D143915
2023-02-14 11:25:30 +00:00
Matt Arsenault
4f0eb57222 AMDGPU: Teach getNegatedExpression about rcp 2023-02-14 04:02:39 -04:00
Matt Arsenault
ce4b719f33 AMDGPU: Add test for getNegatedExpression with rcp 2023-02-14 04:02:39 -04:00
Matt Arsenault
0a669bd894 AMDGPU: Add additional tests for combiner infinite loop 2023-02-14 04:02:38 -04:00
pvanhout
04f6934589 [DAG] Handle build_vector with all undefs in reduceBuildVecTruncToBitCast
While working on D143731 I hit a case where a build_vector with 2 undef operands could be generated (with one undef hidden behind a bitcast).
That made `reduceBuildVecTruncToBitCast` crash because it seems to assume there is at least one good operand.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D143886
2023-02-14 08:52:28 +01:00
Arthur Eubanks
7c6b46e87e Revert "[DAGCombiner] handle more store value forwarding"
This reverts commit f35a09daebd0a90daa536432e62a2476f708150d.

Causes miscompiles, see D138899
2023-02-13 19:07:28 -08:00
Chen Zheng
6ee2f770ef [PowerPC][GISel] add support for fpconstant
Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D133340
2023-02-14 02:39:22 +00:00
Alexander Shaposhnikov
8f5d81585a [Clang][LLVM] Enable __arithmetic_fence and fprotect-parens on AArch64
Enable __arithmetic_fence and fprotect-parens on AArch64.

Test plan: ninja check-clang check-clang-tools check-llvm

Differential revision: https://reviews.llvm.org/D143781
2023-02-13 19:26:11 +00:00
Craig Topper
7638409d43 [RISCV] Make vsetvli intrinsics default to MA.
The vsetvli insertion pass can replace it with MU if needed by
a using instruction. The vsetvli insertion pass will not convert
MU to MA so we need to start at MA.

Reviewed By: eopXD

Differential Revision: https://reviews.llvm.org/D143790
2023-02-13 10:39:55 -08:00
Christudasan Devadasan
1c9e6238fe [AMDGPU] Allow architected SGPRs for workgroup IDs
Some subtargets use architected SGPRs for workgroup
IDs instead of the regular SGPRs. This patch enables
the support for the same and is guarded under the
subtarget feature FeatureArchitectedSGPRs.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D143707
2023-02-13 22:11:35 +05:30
Mirko Müller
4198ff0cb9 [AArch64] Add NZCV Def for TLSDESC_CALLSEQ
The glibc and older musl handlers of tlsdesc_dynamic use a cmp instruction
which will clobber NZCV.

See glibc's _dl_tlsdesc_dynamic:
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/aarch64/dl-tlsdesc.S;hb=refs/heads/release/2.37/master

See v1.1.21 Musl's __tlsdesc_dynamic:
https://git.musl-libc.org/cgit/musl/tree/src/ldso/aarch64/tlsdesc.s?h=v1.1.21

Differential Revision: https://reviews.llvm.org/D143157
2023-02-13 16:14:23 +00:00
Dinar Temirbulatov
d44b31eca2 [DAGCombine] Allow DAGCombine to remove dead masked stores.
Remove a dead masked store if another one has the same base pointer and mask or
the following store has all true constant mask and size if equal or bigger to
the first store.

Differential Revision: https://reviews.llvm.org/D143069
2023-02-13 16:11:11 +00:00
Philipp Tomsich
fc02eeb24f [RISCV] Add vendor-defined XTheadBb (basic bit-manipulation) extension
The vendor-defined XTHeadBb (predating the standard Zbb extension)
extension adds some bit-manipulation extensions with somewhat similar
semantics as some of the Zbb instructions.

It is supported by the C9xx cores (e.g., found in the wild in the
Allwinner D1) by Alibaba T-Head.

The current (as of this commit) public documentation for XTHeadBb is
available from:
  https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf

Support for these instructions has already landed in GNU Binutils:
  https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8254c3d2c94ae5458095ea6c25446ba89134b9da

Depends on D143036

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D143439
2023-02-13 17:02:09 +01:00
Philipp Tomsich
04a2baf58f [RISCV] Add vendor-defined XTHeadBs (single-bit) extension
The vendor-defined XTHeadBs (predating the standard Zbs extension)
extension adds a bit-test instruction (th.tst) with similar semantics
as bexti from Zbs.  It is supported by the C9xx cores (e.g., found in
the wild in the Allwinner D1) by Alibaba T-Head.

The current (as of this commit) public documentation for XTHeadBs is
available from:
  https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf

Support for these instructions has already landed in GNU Binutils:
  https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8254c3d2c94ae5458095ea6c25446ba89134b9da

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D143036
2023-02-13 16:28:26 +01:00
Stefan Pintilie
2e47aafb02 [PowerPC] Fix float materialization patterns.
Two of the float materialization patterns use the VSSRC regsiter class. This
register class is not available before Power 8. The patterns will stay the same
for Power 8 and up but must use the class F4RC for Power 7 and earlier.

This patch fixes those patterns.

Reviewed By: nemanjai, amyk, #powerpc

Differential Revision: https://reviews.llvm.org/D142120
2023-02-13 10:18:53 -05:00
Phoebe Wang
6a6c527ee2 [X86][FP16] Combine two steps conversions into direct conversion
When both v8i64 and v4f16 are not legal in a v8i64->v8f16 conversion, legalizer will breaks it into v8i64->v4i64->v4f32->v8f32->v8f16.

Given we support v4i64->v8f16, we can combine them with a shuffle instruction.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D143872
2023-02-13 22:57:56 +08:00
David Green
e9eaee9da1 [AArch64] Reassociate sub(x, add(m1, m2)) to sub(sub(x, m1), m2)
The mid end will reassociate sub(sub(x, m1), m2) to sub(x, add(m1, m2)). This
reassociates it back to allow the creation of more mls instructions.

Differential Revision: https://reviews.llvm.org/D143143
2023-02-13 14:35:10 +00:00
Simon Pilgrim
d3b0fba608 Revert rG0b0a38a7a229b70d7261771ba0e702843bd34e97 : "[X86] combineX86ShufflesRecursively - don't widen shuffle subvector inputs"
Reports of miscompiles, that I'm still trying to triage - reverting for now
2023-02-13 13:28:48 +00:00
Tomas Matheson
9e3010ad99 [AArch64] Fix LSE2/LSE128/RCPC3 precedence
D142712 added tests for when both lse2 and lse128 are available, but
in practice there is no way to enable LSE128 without LSE2 from clang:
LSE128 is a v9 only feature and LSE2 has been mandatory since v8.4,
and +/-lse2 can not be specified on the clang command line.

Therefore it makes more sense that lse2+lse128 should emit lse128
instructions, otherwise they will not be emitted at all.

It also makes sense to remove the lse128-only backend tests if that set
of attributes is never set by the frontend.

Differential Revision: https://reviews.llvm.org/D143506
2023-02-13 12:31:38 +00:00
Samuel Parker
8f104a3f9a [ARM] O3-pipeline fix 2023-02-13 11:01:00 +00:00
Samuel Parker
2a58be4239 [HardwareLoops] NewPM support.
With the NPM, we're now defaulting to preserving LCSSA, so a couple
of tests have changed slightly.

Differential Revision: https://reviews.llvm.org/D140982
2023-02-13 09:46:31 +00:00
Kazu Hirata
e0e3296098 [X86] Precommit tests for D143838 2023-02-12 19:33:23 -08:00
Phoebe Wang
ff5c00d4ff [X86][FP16] Add tests to show suboptimal codegen, NFC 2023-02-13 10:06:20 +08:00
Simon Pilgrim
19c1682b6a [X86] combineConcatVectorOps - concatenate 512-bit VPERMILPS nodes. 2023-02-12 18:26:28 +00:00
Simon Pilgrim
6eb66890e4 [X86] Add some basic matrix multiplication test coverage
Based off the IR generated from matrix_type / -fenable-matrix - including 2x2, 3x3, 4x4 and 8x8 matrices
2023-02-12 18:26:27 +00:00
David Green
9f2014269a [AArch64] Regenerate and extend zip1 tests. NFC
This cleans up the existing tests and adds some extra cases that can be lowered
to zip instructions.
2023-02-12 16:52:38 +00:00
Simon Pilgrim
1bb95a3a99 [X86] combinePredicateReduction - attempt to fold subvector all_of(icmp_eq()) / any_of(icmp_ne()) to integers
Noticed while working on Issue #59867 and Issue #53419 - there's still more to do here, but for "all vector" comparisons, we should try to cast to a scalar integer for sub-128bit types
2023-02-12 15:23:47 +00:00
Simon Pilgrim
1300a4fdae Revert rG23cb32c6d5bda0919cc1ef129917ceb2dbf1b1b8 "[X86] combineX86ShufflesRecursively - treat ISD::TRUNCATE as faux shuffle"
This is causing a miscompile - waiting on a regression test from @bkramer
2023-02-12 14:46:08 +00:00
Martin Storsjö
7717e1114a Revert "[AArch64] Reassociate sub(x, add(m1, m2)) to sub(sub(x, m1), m2)"
This reverts commit c52255d26a23df6ecf09f60ca3e3615467f16bbe.

That commit caused certain files (in ffmpeg, libvpx and libaom) to hang
while compiling, see https://reviews.llvm.org/D143143 for repro.
2023-02-12 16:00:32 +02:00
Hsiangkai Wang
c9a7b92a23 [AArch64] Consider tiny code model in emitLoadFromConstantPool.
We should be able to use load(literal) to access constant pool under
the tiny code model.

Reviewed By: aemerson

Differential Revision: https://reviews.llvm.org/D132536
2023-02-12 06:02:47 +00:00
Mircea Trofin
4afb1ee7bc Specify target triple for bb-prefix-dump.ll
Differential Revision: https://reviews.llvm.org/D143815
2023-02-11 14:02:44 -08:00
Simon Pilgrim
23cb32c6d5 [X86] combineX86ShufflesRecursively - treat ISD::TRUNCATE as faux shuffle
getFauxShuffleMask can't handle ISD::TRUNCATE itself as it can't handle inputs that are larger than the output

Another step towards removing combineX86ShuffleChainWithExtract
2023-02-11 19:16:08 +00:00