19444 Commits

Author SHA1 Message Date
Simon Pilgrim
e4d0e12099 [DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2) (REAPPLIED)
Assuming the ADD is nsw then it may be sign-extended to merge with a SHL op in a similar fold to the existing (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2) fold.

This is most useful for helping to expose address math for X86, but has also touched several aarch64 test cases as well.

Alive2: https://alive2.llvm.org/ce/z/2UpSbJ

Differential Revision: https://reviews.llvm.org/D159198
2023-09-06 13:19:42 +01:00
Dmitri Gribenko
97bf104d97 Revert "[DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2)"
This reverts commit b027ce0ab93060bc6cb79d5402d21520e8b93fb7.

This commit breaks Transforms/InferAddressSpaces/AMDGPU/flat_atomic.ll.
2023-09-06 11:28:55 +02:00
Simon Pilgrim
b027ce0ab9 [DAG] Fold (shl (sext (add_nsw x, c1)), c2) -> (add (shl (sext x), c2), c1 << c2)
Assuming the ADD is nsw then it may be sign-extended to merge with a SHL op in a similar fold to the existing (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2) fold.

This is most useful for helping to expose address math for X86, but has also touched several aarch64 test cases as well.

Alive2: https://alive2.llvm.org/ce/z/2UpSbJ

Differential Revision: https://reviews.llvm.org/D159198
2023-09-06 10:06:21 +01:00
Simon Pilgrim
e086e0aeef [X86] Add test coverage for new smulo folds added in D159406
Pulled from the InstCombine with_overflow.ll tests
2023-09-05 17:43:42 +01:00
Simon Pilgrim
e6971cbc06 [X86] combine-mulo.ll - add common CHECK prefix for SSE/AVX test runs 2023-09-04 16:42:48 +01:00
Simon Pilgrim
d9ffd3219e [X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero (REAPPLIED)
We can just bitcast the pre-shifted mask as an integer and use TEST/BT directly.

Reapplied with fix for 239ab16ec121 which didn't set the comparison type correctly
2023-09-02 17:45:17 +01:00
Simon Pilgrim
600b4634ac [X86] Add test to check that an extracted bool element comparison is correctly extended when the bool vector is bitcast instead
Thanks to @zequanwu for the reduced test case where 239ab16ec121 failed to correctly cast a compare-with-zero to the correct integer type
2023-09-02 17:34:12 +01:00
Matt Arsenault
b14e83d1a4 IR: Add llvm.exp10 intrinsic
We currently have log, log2, log10, exp and exp2 intrinsics. Add exp10
to fix this asymmetry. AMDGPU already has most of the code for f32
exp10 expansion implemented alongside exp, so the current
implementation is duplicating nearly identical effort between the
compiler and library which is inconvenient.

https://reviews.llvm.org/D157871
2023-09-01 19:45:03 -04:00
Zequan Wu
57595086db fix revert b0b3f82dd3c00cdba891f1ff6ba63abd419d0f18 2023-09-01 17:01:43 -04:00
Zequan Wu
b0b3f82dd3 Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero"
This reverts commit 239ab16ec1213749a2228368298519b377d336bb.

This causes crashes when compiling chromium with asan, attached reduced ir at: https://reviews.llvm.org/rG239ab16ec1213749a2228368298519b377d336bb#124559
2023-09-01 16:35:55 -04:00
Philip Reames
8347d7c152 Revert "Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero""
This reverts commit 460bba35211853b6278ddd6064f7228db02da132.  Change does not pass check-llvm.
2023-09-01 12:17:36 -07:00
Zequan Wu
460bba3521 Revert "[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero"
This reverts commit 239ab16ec1213749a2228368298519b377d336bb.

This causes crashes when compiling chromium with asan, attached reduced ir at: https://reviews.llvm.org/rG239ab16ec1213749a2228368298519b377d336bb#1245595
2023-09-01 15:06:04 -04:00
Simon Pilgrim
2a81396b1b [DAG] SimplifyDemandedBits - add SMIN/SMAX KnownBits comparison analysis
Followup to D158364

Also, final fix for Issue #59902 which noted that the snippet should just return 1
2023-09-01 12:42:30 +01:00
Simon Pilgrim
b3d454950c [X86] Add tests showing failure to use KnownBits known comparison results to remove SMIN/SMAX
Followup to D158364
2023-09-01 12:04:57 +01:00
Simon Pilgrim
aca8b9d0d5 [DAG] SimplifyDemandedBits - if we're only demanding the signbits, a MIN/MAX node can be simplified to a OR or AND node
Extension to the signbit case, if the signbits extend down through all the demanded bits then SMIN/SMAX/UMIN/UMAX nodes can be simplified to a OR/AND/AND/OR.

Alive2: https://alive2.llvm.org/ce/z/mFVFAn (general case)

Differential Revision: https://reviews.llvm.org/D158364
2023-09-01 10:56:32 +01:00
Rainer Orth
6ef767c075 [MC][ELF] Don't emit .note.GNU-stack sections on Solaris
LLVM currently emits `.note.GNU-stack` sections on all ELF targets.

However, Solaris ld doesn't know/care about them.  Even worse, with the
revised Solaris GNU ld patch (D85309 <https://reviews.llvm.org/D85309>),
there are hundreds of warnings:

  /usr/gnu/bin/ld: warning: /usr/lib/amd64/crtn.o: missing .note.GNU-stack
section implies executable stack
  /usr/gnu/bin/ld: NOTE: This behaviour is deprecated and will be removed
in a future version of the linker

The Solaris crts are not going to change here, and even if they were, GNU
ld would emit `PT_GNU_STACK` segments that Solaris `ld.so.1` ignores.

So the note sections are completely useless on Solaris and this patch
disables their creation.

Instead, Solaris has its own mechanisms to control stack executability:
`PT_SUNW_STACK`, `DT_SUNW_SX_NXSTACK` and the system-wide control via
`sxadm` where `nxstack` defaults to on.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11` with Solaris
ld and GNU ld, and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D159179
2023-09-01 11:20:42 +02:00
Arthur Eubanks
2a2f02e19f [X86] Use 64-bit jump table entries for large code model PIC
With the large code model, the label difference may not fit into 32 bits.
Even if we assume that any individual function is no larger than 2^32
and use a difference from the function entry to the target destination,
things like BOLT can rearrange blocks (even if BOLT doesn't necessarily
work with the large code model right now).

set directives avoid static relocations in some 32-bit entry cases, but
don't worry about set directives for 64-bit jump table entries (we can
do that later if somebody really cares about it).

check-llvm in a bootstrapped clang with the large code model passes.

Fixes #62894

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D159297
2023-08-31 14:13:38 -07:00
Simon Pilgrim
9734b2256d [X86] combineCMP - use widenMaskVector to allow us to handle sub-i8 mask cases when just comparing a bool element against zero 2023-08-31 16:57:22 +01:00
Simon Pilgrim
239ab16ec1 [X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero
We can just bitcast the pre-shifted mask as an integer and use TEST/BT directly.

This can be extended further to better handle sub-i8 mask cases, but just getting rid of KSHIFTR nodes makes a notable difference.
2023-08-31 13:14:01 +01:00
Simon Pilgrim
f33c64dd56 [X86] addr-mode-matcher-2.ll - add more sext/zext nsw/nuw permutations
As suggested by D159198
2023-08-31 12:18:44 +01:00
Simon Pilgrim
967d95382d [X86] lowerShuffleAsVALIGN - extend to recognize basic shifted element masks
Try to use VALIGN as a cross-lane version of VSHLDQ/VSRLDQ
2023-08-30 18:32:55 +01:00
Simon Pilgrim
d3d71b8d5b [X86] Add shuffle tests cases showing missed opportunity to use VALIGN 2023-08-30 18:32:55 +01:00
Simon Pilgrim
4b383107fa [X86] combineConcatVectorOps - support concatenation of 128->256-bit X86ISD::PCMPEQ/PCMPGT nodes on AVX2+ targets
Only bother if one of the ops is free to concat (most likely compare with constant).
2023-08-29 14:33:22 +01:00
Simon Pilgrim
2a82da8097 [X86] combineConcatVectorOps - handle splatting of a X86ISD::SUBV_BROADCAST_LOAD node
If we're splatting the original subvector width then just use the original X86ISD::SUBV_BROADCAST_LOAD node - similar to what we're already doing with X86ISD::VBROADCAST/VBROADCAST_LOAD
2023-08-29 14:21:13 +01:00
Simon Pilgrim
85cf2e8286 [X86] combineConcatVectorOps - concatenation of constant subvectors is free. 2023-08-29 14:15:24 +01:00
Phoebe Wang
b667e9c23d [X86][BF16] Lower FP_ROUND for vector types under AVX512BF16
Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D158952
2023-08-29 13:48:13 +08:00
Serguei Katkov
a701b7e368 [CGP] Remove dead PHI nodes before elimination of mostly empty blocks
Before elimination of mostly empty block it makes sense to remove dead PHI nodes.
It open more opportunity for elimination plus eliminates dead code itself.

It appeared that change results in failing many unit tests and some of
them I've updated and for another one I disable this optimization.
The pattern I observed in the tests is that there is a infinite loop
without side effects. As a result after elimination of dead phi node all other
related instruction are also removed and tests stops to check what it is expected.

Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D158503
2023-08-29 04:35:06 +00:00
Phoebe Wang
30ec9473c6 [X86][BF16] Add test coverage for AVX-NE-CONVERT
Split from D158952.
2023-08-29 09:08:01 +08:00
Phoebe Wang
6688701497 [X86][BF16] Lower FP_EXTEND for vector types under AVX512BF16
Fixes #64460

Reviewed By: RKSimon, skan

Differential Revision: https://reviews.llvm.org/D158950
2023-08-28 21:27:10 +08:00
Phoebe Wang
23e2a82446 Revert "[X86][BF16] Lower FP_EXTEND for vector types under AVX512BF16"
This reverts commit 4ae7ed6e19bab0d62c0f936bd6f555103cc3b197.

Sorry, missing the test update.
2023-08-28 21:06:22 +08:00
Phoebe Wang
4ae7ed6e19 [X86][BF16] Lower FP_EXTEND for vector types under AVX512BF16
Fixes #64460

Reviewed By: RKSimon, skan

Differential Revision: https://reviews.llvm.org/D158950
2023-08-28 20:52:06 +08:00
Phoebe Wang
3753ea8311 Revert "[X86][BF16] Lower FP_EXTEND for vector types under AVX512BF16"
This reverts commit 915139fc73fd34223caaec0c3b3525ad79540ec0.

The constant value is 16 rather than 8. Revert it and then reland.
2023-08-28 20:48:16 +08:00
Nikita Popov
98cf20f890 Revert "[Verifier] Sanity check alloca size against DILocalVariable fragment size"
This reverts commit 183f49c3e0f4a7facf237581f83ae07e7f4544ab.

The lang/cpp/trivial_abi/TestTrivialABI.py lldb test fails on
buildbots.
2023-08-28 09:44:51 +02:00
Nikita Popov
183f49c3e0 [Verifier] Sanity check alloca size against DILocalVariable fragment size
Add a check that the DILocalVariable fragment size in dbg.declare
does not exceed the size of the alloca.

This would have caught the invalid debuginfo regenerated by rustc
in https://github.com/llvm/llvm-project/issues/64149.

Differential Revision: https://reviews.llvm.org/D158743
2023-08-28 09:16:33 +02:00
Phoebe Wang
915139fc73 [X86][BF16] Lower FP_EXTEND for vector types under AVX512BF16
Fixes #64460

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D158950
2023-08-28 09:47:12 +08:00
Simon Pilgrim
54c20055da [X86] MergeConsecutiveStores.ll - add 32-bit i686 coverage 2023-08-27 11:35:16 +01:00
Felipe de Azevedo Piovezan
6be47fb8be [CodeGen] Separate X86 and Aarch entry_value test
Addresses the bot issues raised in D158636.
2023-08-24 17:07:21 -04:00
Felipe de Azevedo Piovezan
e070a5d230 [CodeGen] Separate X86 and Aarch tests
The directory this test used to live in is exclusive to Aarch.
Addresses the failure reported in D158636.
2023-08-24 16:33:13 -04:00
Simon Pilgrim
19777deba4 [X86] matchAddressRecursively - add foldMaskedShiftToBEXTR handling to ZERO_EXTEND nodes. 2023-08-24 13:14:41 +01:00
Simon Pilgrim
69a0f23598 [X86] extract-bits.ll - add test showing failure to match BEXTR through ZERO_EXTEND node 2023-08-24 13:14:41 +01:00
Simon Pilgrim
e283ef7e93 [X86] matchAddressRecursively - add foldMaskAndShiftToScale handling to ZERO_EXTEND nodes. 2023-08-24 11:47:07 +01:00
Simon Pilgrim
f84cd7e579 [X86] fold-and-shift-x86_64.ll - add zext test case where upper bits are known zero (and won't get simplified to any_extend)
Add test coverage showing failure to use foldMaskAndShiftToScale with zero_extend nodes
2023-08-24 11:47:07 +01:00
Simon Pilgrim
19cdd45b08 [X86] X86DAGToDAGISel::matchIndexRecursively - add SIGN_EXTEND(ADD_NSW(X,C)) handling
Split an index register from IndexReg = SIGN_EXTEND(ADD_NSW(X,C)) to IndexReg = SIGN_EXTEND(X), Offset = SIGN_EXTEND(C)
2023-08-24 10:19:37 +01:00
Serge Pavlov
6862f0fab1 [FPEnv] Intrinsics for access to FP control modes
The change introduces intrinsics 'get_fpmode', 'set_fpmode' and
'reset_fpmode'. They manage all target dynamic floating-point control
modes, which include, for instance, rounding direction, precision,
treatment of denormals and so on. The intrinsics do the same
operations as the C library functions 'fegetmode' and 'fesetmode'. By
default they are lowered to calls to these functions.

Two main use cases are supported by this implementation.

1. Local modification of the control modes. In this case the code
usually has a pattern (in pseudocode):

    saved_modes = get_fpmode()
    set_fpmode(<new_modes>)
    ...
    <do operations under the new modes>
    ...
    set_fpmode(saved_modes)

In the case when it is known that the current FP environment is default,
the code may be shorter:

    set_fpmode(<new_modes>)
    ...
    <do operations under the new modes>
    ...
    reset_fpmode()

Such patterns appear not only in user code but also in implementations
of various FP controlling pragmas. In particular, the implementation of
`#pragma STDC FENV_ROUND` requires similar code if the target does not
support static rounding mode.

2. Portable control of FP modes. Usually FP control modes are set by
writing to some control register. Different targets have different
layout of this register, the way the register is accessed also may be
different. Using set of target-specific definitions for the control
register bits together with these intrinsic functions provides enough
portable way to handle control modes across wide range of hardware.

This change defines only llvm intrinsic function, which implement the
access required for the aforementioned use cases.

Differential Revision: https://reviews.llvm.org/D82525
2023-08-24 15:52:19 +07:00
Simon Pilgrim
5d79a8d148 [X86] fold-and-shift.ll - add x86-64 test coverage
Although we already have fold-and-shift-x86_64.ll - this adds additional test coverage for various and-shift patterns split by sign/zero extensions from i32 index patterns to i64 pointers
2023-08-23 22:16:54 +01:00
Peter Rong
f58fbfc746 [X86][CodeGen] Add a dag pattern to fix #64323
After recent patch D30189, #64323's error message become a new one.
When DAGCombiner was optimizing `(vextract (scalar_to_vector val, 0) -> val`, it didn't
consider the possibility that the inserted value type has less bit than the dest type.
This patch fixes that.

Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D158355
2023-08-23 10:50:32 -07:00
Rahman Lavaee
d7e10df605 Remove checking stats from -gc-empty-basic-blocks test.
The test does not require asserts. So it can't check the stats.
2023-08-23 01:18:47 +00:00
Rahman Lavaee
d0ec03a384 Revert "[BasicBlockSections] avoid insertting redundant branch to fall through blocks"
This reverts commit ab53109166c0345a79cbd6939cf7bc764a982856 which was
commited by mistake.
2023-08-23 01:09:13 +00:00
Rahman Lavaee
ab53109166 [BasicBlockSections] avoid insertting redundant branch to fall through blocks 2023-08-22 23:32:02 +00:00
Rahman Lavaee
e280e406c2 Add a pass to garbage-collect empty basic blocks after code generation.
Propeller and pseudo-probes map profiles back to Machine IR via basic block addresses that are stored in metadata sections.
Empty basic blocks (basic blocks without real code) obfuscate the profile mapping because their addresses collide with their next basic blocks.
For instance, the fallthrough block of an empty block should always be adjacent to it. Otherwise, a completely unnecessary jump would be added.
This patch adds a MachineFunction pass named `GCEmptyBasicBlocks` which attempts to garbage-collect the empty blocks before the `BasicBlockSections` and pass.
This pass removes each empty basic block after redirecting its incoming edges to its fall-through block.
The garbage-collection is not complete. We keep the empty block in 4 cases:
      1. The empty block is an exception handling pad.
      2. The empty block has its address taken.
      3. The empty block is the last block of the function and it has
         predecessors.
      4. The empty block is the only block of the function.
The first three cases are extremely rare in normal code (no cases for the clang binary). Removing the blocks under the first two cases requires modifying exception handling structures and operands of non-terminator instructions -- which is doable but not worth the additional complexity in the pass.

Reviewed By: tmsriram

Differential Revision: https://reviews.llvm.org/D107534
2023-08-22 22:42:19 +00:00