19477 Commits

Author SHA1 Message Date
Nikita Popov
38c59b9f53 Revert "Reapply [Verifier] Sanity check alloca size against DILocalVariable fragment size"
This reverts commit 47324cfd7d8ca1a2a5cbb9f948ecff66a28ee6bc.

This exposed incorrect debuginfo in rustc. Revert the verification
until this has been fixed.
2023-09-18 17:24:53 +02:00
Sergei Barannikov
caaf61eb6e
[SDag] Fold saddo[_carry] with bitwise-not argument to ssubo[_carry] (#66571)
Fold `(saddo (not a), 1)` to `(ssubo 0, a)` and
`(saddo_carry (not a), b, c)` to `(ssubo_carry b, a, !c)`.

Proof: https://alive2.llvm.org/ce/z/Lj49YM

This is the same as https://reviews.llvm.org/D46505 and
https://reviews.llvm.org/D59208, but for signed opcodes.
2023-09-18 14:45:41 +03:00
Jay Foad
102838d3f6
update_mir_test_checks.py: match undef vreg subreg definitions (#66627)
Following on from D139466 which added support for dead vreg defs, this
patch adds support for "undef" defs of subregs.

Use this to regenerate checks for amx-greedy-ra-spill-shape.ll which
previously required manual tweaks to the autogenerated checks to fix an
EXPENSIVE_CHECKS failure; see commit
8b7c1fbd9647a5a6ef246a6b5b2543ea0f5a2337
2023-09-18 12:14:46 +01:00
Guozhi Wei
cbdccb30c2 [RA] Split a virtual register in cold blocks if it is not assigned preferred physical register
If a virtual register is not assigned preferred physical register, it means some
COPY instructions will be changed to real register move instructions. In this
case we can try to split the virtual register in colder blocks, if success, the
original COPY instructions can be deleted, and the new COPY instructions in
colder blocks will be generated as register move instructions. It results in
fewer dynamic register move instructions executed.

The new test case split-reg-with-hint.ll gives an example, the hot path contains
24 instructions without this patch, now it is only 4 instructions with this
patch.

Differential Revision: https://reviews.llvm.org/D156491
2023-09-15 19:52:50 +00:00
Nikita Popov
47324cfd7d Reapply [Verifier] Sanity check alloca size against DILocalVariable fragment size
Reapply after fixing a clang bug this exposed in D158972 and
adjusting a number of tests that failed for 32-bit targets.

-----

Add a check that the DILocalVariable fragment size in dbg.declare
does not exceed the size of the alloca.

This would have caught the invalid debuginfo regenerated by rustc
in https://github.com/llvm/llvm-project/issues/64149.

Differential Revision: https://reviews.llvm.org/D158743
2023-09-15 14:51:50 +02:00
Arthur Eubanks
1feb00a28c [X86] Introduce a large data threshold for the medium code model
Currently clang's medium code model treats all data as large, putting them in a large data section and using more expensive instruction sequences to access them.

Following gcc's -mlarge-data-threshold, which allows putting data under a certain size in a normal data section as opposed to a large data section. This allows using cheaper code sequences to access some portion of data in the binary (which will be implemented in LLVM in a future patch).

And under the medium codel mode, only put data above the large data threshold into large data sections, not all data.

Reviewed By: MaskRay, rnk

Differential Revision: https://reviews.llvm.org/D149288
2023-09-14 15:09:25 -07:00
Philip Reames
2f005df066
[DAG][X86] Fold mgather/mscatter/etc with splat index (#65980)
A splat index means the operation is reading from (writing to) the same
memory location. Generally, zero is the cheapest value to splat. As
such, we'd prefer to add the splatted value to the base, and use a
constant zero as the index operand.
2023-09-13 09:26:30 -07:00
Simon Pilgrim
8b479136c9 [X86] SimplifyMultipleUseDemandedBitsForTargetNode - add X86ISD::BLENDV handling
If we know the condition mask sign bit of the demanded elements then we can directly return the LHS/RHS selection
2023-09-13 17:23:11 +01:00
Simon Pilgrim
4a32c48280 [X86] LowerTRUNCATE - ensure we handle cases where we truncate to a sub-128bit type (PR66194)
Fixes #66194
2023-09-13 13:15:42 +01:00
Simon Pilgrim
e6b85c3027 [DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case (REAPPLIED)
Followup to D59363 which failed to handle the icmp(X,undef) -> isTrueWhenEqual case - similar to llvm::ConstantFoldCompareInstruction

As discussed on the review, this is affecting some previously reduced test cases, but will also prevent reductions from relying on this inconsistent behaviour in the future.

Reapplied after reversion at e1e3c75c7dad72 with a tweak to the pseudo-probe-peep.ll test

Differential Revision: https://reviews.llvm.org/D158068
2023-09-13 12:33:39 +01:00
Simon Pilgrim
e1e3c75c7d Revert rG6c56cf71ee82ec3a28e0dfc2b751bd10c16929da "[DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case"
Need to address a missed test change
2023-09-13 11:27:47 +01:00
Simon Pilgrim
6c56cf71ee [DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case
Followup to D59363 which failed to handle the icmp(X,undef) -> isTrueWhenEqual case - similar to llvm::ConstantFoldCompareInstruction

As discussed on the review, this is affecting some previously reduced test cases, but will also prevent reductions from relying on this inconsistent behaviour in the future.

Differential Revision: https://reviews.llvm.org/D158068
2023-09-13 11:01:58 +01:00
Jay Foad
0528dbfe5c
Add some -early-live-intervals RUN lines (#66058)
This adds test coverage for an upcoming change to
TwoAddressInstructionPass::processTiedPairs.
2023-09-12 13:06:10 +01:00
Matt Arsenault
cd4b906e18 RegisterCoalescer: Don't delete IMPLICIT_DEF if it's live into the same block
Live out implicit_defs need to be kept, but the check for this only
checked if the block parent was the same. This doesn't work if the
parent blocks are the same but the value is live. Fixes verifier error
"Instruction ending live segment doesn't read the register", which
would appear at the coalesced non-implicit_def def.

Fixes #38788

https://reviews.llvm.org/D158882
2023-09-12 09:28:33 +03:00
Matt Arsenault
de5585078e RegisterCoalescer: Correctly set valid lanes when keeping live out implicit defs
This fixes some verifier errors when live out implicit defs are
coalesced with identity copies. Fixes some reduced testcases from
issue #38788 but doesn't solve the original failure.

I was surprised this seems to obviate the special casing in
analyzeValue that's been there since the subregister liveness support
went in.

https://reviews.llvm.org/D158850
2023-09-12 09:28:33 +03:00
liqin.weng
1eec357494 [VP] IR expansion for maxnum/minnum
Add basic handling for VP ops that can expand to non-predicate ops

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D159494
2023-09-12 10:15:52 +08:00
Simon Pilgrim
f8b04eb6d0 [X86] matchIndexRecursively - add zext(add/addlike(x,c)) -> index: zext(x), disp + zext(c) pattern handling
More restricted alternative to a8cef6b58e2d41f
2023-09-11 15:36:13 +01:00
liqin.weng
3723ede3cf [VP] IR expansion for zext/sext/trunc/fptosi/fptosi/sitofp/uitofp/fptrunc/fpext
Add basic handling for VP ops that can expand to Cast intrinsics

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D159491
2023-09-11 21:14:38 +08:00
liqin.weng
28e74e6180 [VP] IR expansion for abs/smax/smin/umax/umin
Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D159495
2023-09-11 21:14:37 +08:00
Simon Pilgrim
ef87d43834 Revert rGa8cef6b58e2d41f04ed4fa63c3f628eac1a28925 "[X86] promoteExtBeforeAdd - add support for or/xor 'addlike' patterns"
Investigating reports of issues with second stage clang builds
2023-09-11 11:52:25 +01:00
Simon Pilgrim
a8cef6b58e [X86] promoteExtBeforeAdd - add support for or/xor 'addlike' patterns
Fold zext(addlike(x, C)) --> add(zext(x), C_zext) if its likely to help us create LEA instructions

Addresses some regressions exposed by D155472
2023-09-11 10:17:34 +01:00
Simon Pilgrim
79941c3a0d [X86] lea-2.ll - add test showing failure to fold shl(zext(or(x,c1)),c2) 'addlike' into LEA instruction 2023-09-11 10:17:33 +01:00
Shengchen Kan
503e3a4130
[X86] Remove _REV instructions from the EVEX2VEX tables (#65752)
_REV instruction should not appear before encoding optimization, so
there is no chance to compress it during MIR optimizations.
2023-09-11 09:54:05 +08:00
Fangrui Song
61c44f1822 [X86] FastISel -fno-pic: emit R_386_PC32 when calling an intrinsic
This matches how a SelectionDAG::getExternalSymbol node is lowered. On x86-32, a
function call in -fno-pic code should emit R_386_PC32 (since ebx is not set up).
When linked as -shared (problematic!), the generated text relocation will work.

Ideally, we should mark IR intrinsics created in
CodeGenFunction::EmitBuiltinExpr as dso_local, but the code structure makes it
not very feasible.

Fix #51078
2023-09-10 15:03:36 -07:00
Simon Pilgrim
76c09d9c5e [X86] matchIndexRecursively - don't peek through multiuse sext(add_nsw(x,c)) (PR65895)
Fixes #65895
2023-09-10 16:54:18 +01:00
liqin.weng
1b622fff44 [VP] IR expansion for inttoptr/ptrtoint
Add basic handling for VP ops that can expand to cast intrinsics

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D159478
2023-09-09 15:34:46 +08:00
Phoebe Wang
24194090e1 [X86][RFC] Add new option -m[no-]evex512 to disable ZMM and 64-bit mask instructions for AVX512 features
This is an alternative of D157485 and a pre-feature to support AVX10.

AVX10 Architecture Specification: https://cdrdv2.intel.com/v1/dl/getContent/784267
AVX10 Technical Paper: https://cdrdv2.intel.com/v1/dl/getContent/784343
RFC: https://discourse.llvm.org/t/rfc-design-for-avx10-feature-support/72661

Based on the feedbacks from LLVM and GCC community, we have agreed to
start from supporting `-m[no-]evex512` on existing AVX512 features.
The option `-mno-evex512` can be used with `-mavx512xxx` to build
binaries that can run on both legacy AVX512 targets and AVX10-256.

There're still arguments about what's the expected behavior when this
option as well as `-mavx512xxx` used together with `-mavx10.1-256`. We
decided to defer the support of `-mavx10.1` after we made consensus.
Or furthermore, we start from supporting AVX10.2 and not providing any
AVX10.1 options.

Reviewed By: RKSimon, skan

Differential Revision: https://reviews.llvm.org/D159250
2023-09-08 22:47:22 +08:00
Phoebe Wang
da1eb886c4
[X86] Do not check alignment for VINSERTPS (#65721)
We don't have alignment constraint in AVX instructions.
2023-09-08 19:23:43 +08:00
Phoebe Wang
2e44b07e24
[X86] Do not directly fold for VINSERTPS (#65718)
We have already customized folding for VINSERTPS by 7e6606f4f1, which do
the folding when alignment >= 4 bytes.

We cannot arbitrarily fold it like others because we need to calculate
the source offset.
2023-09-08 15:35:44 +08:00
Phoebe Wang
11c3b979e6 [X86][NFC] Add a test case to show wrong memory folding for vinsertps 2023-09-08 14:30:33 +08:00
Phoebe Wang
0856efbf88 Revert "[X86][RFC] Add new option -m[no-]evex512 to disable ZMM and 64-bit mask instructions for AVX512 features"
This reverts commit 7dd48cc24de2d54d40527432cbee8a9d97a8a4f7.

Causing buildbot failure.
2023-09-07 21:59:01 +08:00
Phoebe Wang
7dd48cc24d [X86][RFC] Add new option -m[no-]evex512 to disable ZMM and 64-bit mask instructions for AVX512 features
This is an alternative of D157485 and a pre-feature to support AVX10.

AVX10 Architecture Specification: https://cdrdv2.intel.com/v1/dl/getContent/784267
AVX10 Technical Paper: https://cdrdv2.intel.com/v1/dl/getContent/784343
RFC: https://discourse.llvm.org/t/rfc-design-for-avx10-feature-support/72661

Based on the feedbacks from LLVM and GCC community, we have agreed to
start from supporting `-m[no-]evex512` on existing AVX512 features.
The option `-mno-evex512` can be used with `-mavx512xxx` to build
binaries that can run on both legacy AVX512 targets and AVX10-256.

There're still arguments about what's the expected behavior when this
option as well as `-mavx512xxx` used together with `-mavx10.1-256`. We
decided to defer the support of `-mavx10.1` after we made consensus.
Or furthermore, we start from supporting AVX10.2 and not providing any
AVX10.1 options.

Reviewed By: RKSimon, skan

Differential Revision: https://reviews.llvm.org/D159250
2023-09-07 21:38:35 +08:00
Mohamed Atef
741c127817 [SelectionDAG] Add computeOverflowForSignedMul / computeOverflowForUnsignedMul overflow handlers
Support signed multiplication
Support unsigned multiplication

Differential Revision: https://reviews.llvm.org/D159406
2023-09-07 10:03:18 +01:00
Simon Pilgrim
e4d0e12099 [DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2) (REAPPLIED)
Assuming the ADD is nsw then it may be sign-extended to merge with a SHL op in a similar fold to the existing (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2) fold.

This is most useful for helping to expose address math for X86, but has also touched several aarch64 test cases as well.

Alive2: https://alive2.llvm.org/ce/z/2UpSbJ

Differential Revision: https://reviews.llvm.org/D159198
2023-09-06 13:19:42 +01:00
Dmitri Gribenko
97bf104d97 Revert "[DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2)"
This reverts commit b027ce0ab93060bc6cb79d5402d21520e8b93fb7.

This commit breaks Transforms/InferAddressSpaces/AMDGPU/flat_atomic.ll.
2023-09-06 11:28:55 +02:00
Simon Pilgrim
b027ce0ab9 [DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2)
Assuming the ADD is nsw then it may be sign-extended to merge with a SHL op in a similar fold to the existing (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2) fold.

This is most useful for helping to expose address math for X86, but has also touched several aarch64 test cases as well.

Alive2: https://alive2.llvm.org/ce/z/2UpSbJ

Differential Revision: https://reviews.llvm.org/D159198
2023-09-06 10:06:21 +01:00
Simon Pilgrim
e086e0aeef [X86] Add test coverage for new smulo folds added in D159406
Pulled from the InstCombine with_overflow.ll tests
2023-09-05 17:43:42 +01:00
Simon Pilgrim
e6971cbc06 [X86] combine-mulo.ll - add common CHECK prefix for SSE/AVX test runs 2023-09-04 16:42:48 +01:00
Simon Pilgrim
d9ffd3219e [X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero (REAPPLIED)
We can just bitcast the pre-shifted mask as an integer and use TEST/BT directly.

Reapplied with fix for 239ab16ec121 which didn't set the comparison type correctly
2023-09-02 17:45:17 +01:00
Simon Pilgrim
600b4634ac [X86] Add test to check that an extracted bool element comparison is correctly extended when the bool vector is bitcast instead
Thanks to @zequanwu for the reduced test case where 239ab16ec121 failed to correctly cast a compare-with-zero to the correct integer type
2023-09-02 17:34:12 +01:00
Matt Arsenault
b14e83d1a4 IR: Add llvm.exp10 intrinsic
We currently have log, log2, log10, exp and exp2 intrinsics. Add exp10
to fix this asymmetry. AMDGPU already has most of the code for f32
exp10 expansion implemented alongside exp, so the current
implementation is duplicating nearly identical effort between the
compiler and library which is inconvenient.

https://reviews.llvm.org/D157871
2023-09-01 19:45:03 -04:00
Zequan Wu
57595086db fix revert b0b3f82dd3c00cdba891f1ff6ba63abd419d0f18 2023-09-01 17:01:43 -04:00
Zequan Wu
b0b3f82dd3 Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero"
This reverts commit 239ab16ec1213749a2228368298519b377d336bb.

This causes crashes when compiling chromium with asan, attached reduced ir at: https://reviews.llvm.org/rG239ab16ec1213749a2228368298519b377d336bb#124559
2023-09-01 16:35:55 -04:00
Philip Reames
8347d7c152 Revert "Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero""
This reverts commit 460bba35211853b6278ddd6064f7228db02da132.  Change does not pass check-llvm.
2023-09-01 12:17:36 -07:00
Zequan Wu
460bba3521 Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero"
This reverts commit 239ab16ec1213749a2228368298519b377d336bb.

This causes crashes when compiling chromium with asan, attached reduced ir at: https://reviews.llvm.org/rG239ab16ec1213749a2228368298519b377d336bb#1245595
2023-09-01 15:06:04 -04:00
Simon Pilgrim
2a81396b1b [DAG] SimplifyDemandedBits - add SMIN/SMAX KnownBits comparison analysis
Followup to D158364

Also, final fix for Issue #59902 which noted that the snippet should just return 1
2023-09-01 12:42:30 +01:00
Simon Pilgrim
b3d454950c [X86] Add tests showing failure to use KnownBits known comparison results to remove SMIN/SMAX
Followup to D158364
2023-09-01 12:04:57 +01:00
Simon Pilgrim
aca8b9d0d5 [DAG] SimplifyDemandedBits - if we're only demanding the signbits, a MIN/MAX node can be simplified to a OR or AND node
Extension to the signbit case, if the signbits extend down through all the demanded bits then SMIN/SMAX/UMIN/UMAX nodes can be simplified to a OR/AND/AND/OR.

Alive2: https://alive2.llvm.org/ce/z/mFVFAn (general case)

Differential Revision: https://reviews.llvm.org/D158364
2023-09-01 10:56:32 +01:00
Rainer Orth
6ef767c075 [MC][ELF] Don't emit .note.GNU-stack sections on Solaris
LLVM currently emits `.note.GNU-stack` sections on all ELF targets.

However, Solaris ld doesn't know/care about them.  Even worse, with the
revised Solaris GNU ld patch (D85309 <https://reviews.llvm.org/D85309>),
there are hundreds of warnings:

  /usr/gnu/bin/ld: warning: /usr/lib/amd64/crtn.o: missing .note.GNU-stack
section implies executable stack
  /usr/gnu/bin/ld: NOTE: This behaviour is deprecated and will be removed
in a future version of the linker

The Solaris crts are not going to change here, and even if they were, GNU
ld would emit `PT_GNU_STACK` segments that Solaris `ld.so.1` ignores.

So the note sections are completely useless on Solaris and this patch
disables their creation.

Instead, Solaris has its own mechanisms to control stack executability:
`PT_SUNW_STACK`, `DT_SUNW_SX_NXSTACK` and the system-wide control via
`sxadm` where `nxstack` defaults to on.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11` with Solaris
ld and GNU ld, and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D159179
2023-09-01 11:20:42 +02:00
Arthur Eubanks
2a2f02e19f [X86] Use 64-bit jump table entries for large code model PIC
With the large code model, the label difference may not fit into 32 bits.
Even if we assume that any individual function is no larger than 2^32
and use a difference from the function entry to the target destination,
things like BOLT can rearrange blocks (even if BOLT doesn't necessarily
work with the large code model right now).

set directives avoid static relocations in some 32-bit entry cases, but
don't worry about set directives for 64-bit jump table entries (we can
do that later if somebody really cares about it).

check-llvm in a bootstrapped clang with the large code model passes.

Fixes #62894

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D159297
2023-08-31 14:13:38 -07:00