17280 Commits

Author SHA1 Message Date
Simon Pilgrim
fb2539b9d8 [X86][SSE] Add X86ISD::AVG to isCommutativeBinOp to support folding shuffles through the binop 2021-10-13 10:54:37 +01:00
Simon Pilgrim
cceceb7242 [X86][SSE] Add tests showing missing shuffle(avg(shuffle(),shuffle())) -> avg(shuffle(),shuffle()) fold
X86ISD::AVG needs to be added to isCommutativeBinOp to use these folds.
2021-10-13 10:35:24 +01:00
Fangrui Song
c2d4fe51bb [X86] Remove little support we had for MPX
GCC 9.1 removed Intel MPX support. Linux kernel removed MPX in 2019.
glibc 2.35 will remove MPX.

Our support is limited: we support assembling of bndmov but not bnd.
Just remove it.

Reviewed By: pengfei, skan

Differential Revision: https://reviews.llvm.org/D111517
2021-10-12 16:18:51 -07:00
Roman Lebedev
958da6598f
[X86] detectAVGPattern(): don't require zext in the with-constant case 2021-10-12 20:24:17 +03:00
Roman Lebedev
0902451abe
[NFC][X86] Add another test case for PR52131 2021-10-12 20:19:35 +03:00
Roman Lebedev
5f4f5da634
[X86] detectAVGPattern(): support basic case of PAVG chaining (PR52131)
As noted in https://github.com/halide/Halide/pull/6302,
we hilariously fail to match PAVG if we even as much
as look at it the wrong way.

In this particular case, the problem stems from the fact that
`PAVG` root (def) is a `trunc`, and leafs (uses) are `zext`'s,
and InstCombine really loves to get rid of both of these,
for example replace them with a bit mask. So we may not have
said `zext`.

Instead of checking for that + type match,
i think we should rely on the actual active type,
as per the knownbits.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D111571
2021-10-12 19:51:43 +03:00
Freddy Ye
d57a87ea89 [X86][ISel] Lowering llvm.thread.pointer
Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D110681
2021-10-12 11:01:18 +08:00
Guozhi Wei
6599961c17 [TwoAddressInstructionPass] Improve the SrcRegMap and DstRegMap computation
This patch contains following enhancements to SrcRegMap and DstRegMap:

  1 In findOnlyInterestingUse not only check if the Reg is two address usage,
    but also check after commutation can it be two address usage.

  2 If a physical register is clobbered, remove SrcRegMap entries that are
    mapped to it.

  3 In processTiedPairs, when create a new COPY instruction, add a SrcRegMap
    entry only when the COPY instruction is coalescable. (The COPY src is
    killed)

With these enhancements isProfitableToCommute can do better commute decision,
and finally more register copies are removed.

Differential Revision: https://reviews.llvm.org/D108731
2021-10-11 15:28:31 -07:00
Jay Foad
edfdce2627 [PHIElimination] Fix accounting for undef uses when updating LiveVariables
PHI elimination updates LiveVariables info as described here:

    // We only need to update the LiveVariables kill of SrcReg if this was the
    // last PHI use of SrcReg to be lowered on this CFG edge and it is not live
    // out of the predecessor. We can also ignore undef sources.

Unfortunately if the last use also happened to be an undef use then it
would fail to update the LiveVariables at all. Fix this by not counting
undef uses in the VRegPHIUse map.

Thanks to Mikael Holmén for the test case!

Differential Revision: https://reviews.llvm.org/D111552
2021-10-11 20:22:47 +01:00
Roman Lebedev
7af6a44077
[NFC][X86][Codegen] Add semi-negative PAVG chain test (PR52131) 2021-10-11 22:13:38 +03:00
Roman Lebedev
76495ea317
[NFC][X86][Codegen] Add basic PAVG chain test (PR52131) 2021-10-11 21:42:32 +03:00
Simon Pilgrim
ad16c6e52f [X86][AVX] Ensure we retain zero elements in select(pshufb,pshufb) -> or(pshufb,pshufb) fold (PR52122)
The select(pshufb,pshufb) -> or(pshufb,pshufb) fold uses getConstVector to create the refreshed pshufb masks, which treats all negative indices as undef.

PR52122 shows that if we were selecting an element that the PSHUFB has set to zero we must set it back to 0x80 when we recreate the PSHUFB mask and not just leave it as SM_SentinelZero
2021-10-11 14:50:24 +01:00
Simon Pilgrim
84adaabf3e [X86][AVX] Add test case for PR52122 2021-10-11 13:46:05 +01:00
Dávid Bolvanský
e6ce86bb62 [NFC] Added tests for PR52056 2021-10-10 11:34:39 +02:00
Qiu Chaofan
573531fb1f Fix typo of colon to semicolon in lit tests 2021-10-09 10:03:50 +08:00
Itay Bookstein
faa0e2ae76 [SelectionDAG] Fix shift libcall ABI mismatch in shift-amount argument
The shift libcalls have a shift amount parameter of MVT::i32, but
sometimes ExpandIntRes_Shift may be called with a node whose
second operand is a type that is larger than that. This leads to
an ABI mismatch, and for example causes a spurious zeroing of
a register in RV32 for 64-bit shifts. Note that at present regular
shift intstructions already have their shift amount operand adapted
at SelectionDAGBuilder::visitShift time, and funnelled shifts bypass that.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D110508
2021-10-08 09:57:57 +08:00
Wang, Pengfei
c236883b6b [X86] Optimize fdiv with reciprocal instructions for half type
Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D110557
2021-10-08 09:41:13 +08:00
Adrian Prantl
9f93f2bfbd Do not emit prologue_end for line 0 locs if there is a non-zero loc present
This change fixes a bug where the compiler generates a prologue_end
for line 0 locs. That is because line 0 is not associated with any
source location, so there should not be a prolgoue_end at a location
that doesn't correspond to a source location.

There were some LLVM tests that were explicitly checking for line 0
prologue_end's as well since I believe that to be incorrect, I had to
change those tests as well.

Patch by Shubham Rastogi!

Differential Revision: https://reviews.llvm.org/D110740
2021-10-07 13:54:28 -07:00
Jay Foad
3c9dfba189 [PHIElimination] Account for INLINEASM_BR when inserting kills
When PHIElimination adds kills after lowering PHIs to COPYs it knows
that some instructions after the inserted COPY might use the same
SrcReg, but it was only looking at the terminator instructions at the
end of the block, not at other instructions like INLINEASM_BR that can
appear after the COPY insertion point.

Since we have already called findPHICopyInsertPoint, which knows about
INLINEASM_BR, we might as well reuse the insertion point that it
calculated when looking for instructions that might use SrcReg.

This fixes a machine verification failure if you force machine
verification to run after PHIElimination (currently it is disabled for
other reasons) when running
test/CodeGen/X86/callbr-asm-phi-placement.ll.

Differential Revision: https://reviews.llvm.org/D110834
2021-10-07 20:04:39 +01:00
Jay Foad
4109483013 [PHIElimination] Pre-commit a test case for D110834 2021-10-07 20:04:39 +01:00
Jay Foad
5b8befdd02 [X86] Special-case ADD of two identical registers in convertToThreeAddress
X86InstrInfo::convertToThreeAddress would convert this:

  %1:gr32 = ADD32rr killed %0:gr32(tied-def 0), %0:gr32, implicit-def dead $eflags

to this:

  undef %2.sub_32bit:gr64 = COPY killed %0:gr32
  undef %3.sub_32bit:gr64_nosp = COPY %0:gr32
  %1:gr32 = LEA64_32r killed %2:gr64, 1, killed %3:gr64_nosp, 0, $noreg

Note that in the ADD32rr, %0 was used twice and the first use had a kill
flag, which is what MachineInstr::addRegisterKilled does.

In the converted code, each use of %0 is copied to a new reg, and the
first COPY inherits the kill flag from the ADD32rr. This causes
machine verification to fail (if you force it to run after
TwoAddressInstructionPass) because the second COPY uses %0 after it is
killed. Note that machine verification is currently disabled after
TwoAddressInstructionPass but this is a step towards being able to
enable it.

Fix this by not inserting more than one COPY from the same source
register.

Differential Revision: https://reviews.llvm.org/D110829
2021-10-07 19:50:27 +01:00
Jay Foad
42d07bdc40 [X86] Pre-commit a test case for D110829 2021-10-07 19:50:27 +01:00
Jay Foad
548b01c7a6 [MIRParser] Add support for IsInlineAsmBrIndirectTarget
Print this basic block flag as inlineasm-br-indirect-target and parse
it. This allows you to write MIR test cases for INLINEASM_BR. The test
case I added is one that I wanted to precommit anyway for D110834.

Differential Revision: https://reviews.llvm.org/D111291
2021-10-07 19:08:01 +01:00
Craig Topper
58b68e70eb [X86] Don't use popcnt for parity if only bits 7:0 of the input can be non-zero.
Without popcnt we had a special case for using the parity flag from a single test i8 test instruction if only bits 7:0 could be non-zero. That special case is still useful when we have popcnt.

To reach this special case, we enable custom lowering of parity for i16/i32/i64 even when popcnt is enabled. The check for POPCNT being enabled is now after the special case in LowerPARITY.

Fixes PR52093

Differential Revision: https://reviews.llvm.org/D111249
2021-10-06 14:39:57 -07:00
Wolfgang Pieb
f53d05135e [UBSAN][PS4] For the PS4 target, emit the ud2 ocpode for ubsan traps.
Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D111172
2021-10-06 11:49:36 -07:00
Craig Topper
05de0ab431 [X86] Add X86 and X64 prefixes to parity.ll to reduce duplicate check lines. NFC 2021-10-06 11:01:03 -07:00
Craig Topper
fa7a1bea2d [X86] Add test cases for PR52093. NFC 2021-10-06 10:33:15 -07:00
Bjorn Pettersson
1896fb2cff [SelectionDAG] Assume that a GlobalAlias may alias other global values
This fixes a bug detected in DAGCombiner when using global alias
variables. Here is an example:
  @foo = global i16 0, align 1
  @aliasFoo = alias i16, i16 * @foo
  define i16 @bar() {
    ...
    store i16 7, i16 * @foo, align 1
    store i16 8, i16 * @aliasFoo, align 1
    ...
  }

BaseIndexOffset::computeAliasing would incorrectly derive NoAlias
for the two accesses in the example above, resulting in DAGCombiner
miscompiles.

This patch fixes the problem by a defensive approach letting
BaseIndexOffset::computeAliasing return false, i.e. that the aliasing
couldn't be determined, when comparing two global values and at least
one is a GlobalAlias. In the future we might improve this with a
deeper analysis to look at the aliasee for the GlobalAlias etc. But
that is a bit more complicated considering that we could have
'local_unnamed_addr' and situations with several 'alias' variables.

Fixes PR51878.

Differential Revision: https://reviews.llvm.org/D110064
2021-10-05 12:15:55 +02:00
Amara Emerson
8bde5e58c0 Delay outgoing register assignments to last.
The delayed stack protector feature which is currently used for SDAG (and thus
allows for more commonly generating tail calls) depends on being able to extract
the tail call into a separate return block. To do this it also has to extract
the vreg->physreg copies that set up the call's arguments, since if it doesn't
then the call inst ends up using undefined physregs in it's new spliced block.

SelectionDAG implementations can do this because they delay emitting register
copies until  *after* the stack arguments are set up. GISel however just
processes and emits the arguments in IR order, so stack arguments always end up
last, and thus this breaks the code that looks for any register arg copies that
precede the call instruction.

This patch adds a thunk argument to the assignValueToReg() and custom assignment
hooks. For outgoing arguments, register assignments use this return param to
return a thunk that does the actual generating of the copies. We collect these
until all the outgoing stack assignments have been done and then execute them,
so that the copies (and perhaps some artifacts like G_SEXTs) are placed after
any stores.

Differential Revision: https://reviews.llvm.org/D110610
2021-10-04 12:33:20 -07:00
Roman Lebedev
b4218a1451
[NFC][X86][Codegen] Add test coverage for interleaved i64 load/store stride=6 2021-10-04 20:57:36 +03:00
Roman Lebedev
1f4d364ed8
[NFC][X86][Codegen] Add test coverage for interleaved i32 load/store stride=6 2021-10-04 20:57:35 +03:00
Roman Lebedev
c63a9a79af
[NFC][X86][Codegen] Add test coverage for interleaved i64 load/store stride=4 2021-10-04 17:31:57 +03:00
Roman Lebedev
6bba2bd768
[NFC][X86][Codegen] Add test coverage for interleaved i32 load/store stride=4 2021-10-04 17:31:57 +03:00
Simon Pilgrim
bf30c48419 [X86] SimplifyDemandedVectorEltsForTargetNode - simplify PMADDWD for known zero elements
Noticed while investigating the regressions in D110995 - if the RHS element is already zero, then we don't need the corresponding LHS element.

Technically we could also recheck RHS once we have LHS's known zeros, but I haven't seen any missed opportunities from that yet.
2021-10-04 14:36:45 +01:00
Simon Pilgrim
14bcd8bfda [X86] Add tests for enabling slow-mulld on AVX2 targets
As discussed on D110588 - Haswell/Broadwell don't have a great PMULLD implementation, we might want to enable this for them in the future
2021-10-04 11:24:05 +01:00
Roman Lebedev
9afec88907
[NFC][X86][Codegen] Add test coverage for interleaved i64 load/store stride=3 2021-10-03 17:52:12 +03:00
Roman Lebedev
a834849aac
[NFC][X86][Codegen] Add test coverage for interleaved i32 load/store stride=3 2021-10-03 16:49:51 +03:00
Roman Lebedev
9505fe2911
[NFC][X86][Codegen] Add test coverage for interleaved i8 load/store stride=6 2021-10-03 14:35:17 +03:00
Simon Pilgrim
31d0c8f355 [X86] Add SSE2/AVX1/AVX512BW test coverage to interleaved load/store tests
Extension to PR51979 so codegen tests keep close to the costmodel tests
2021-10-03 12:31:30 +01:00
Takafumi Arakaki
e8806d7486 Re-apply the fix on DwarfEHPrepare and add a test
This patch re-introduces the fix in the commit https://github.com/llvm/llvm-project/commit/66b0cebf7f736 by @yrnkrn

> In DwarfEHPrepare, after all passes are run, RewindFunction may be a dangling
>
> pointer to a dead function. To make sure it's valid, doFinalization nullptrs
> RewindFunction just like the constructor and so it will be found on next run.
>
> llvm-svn: 217737

It seems that the fix was not migrated to `DwarfEHPrepareLegacyPass`.

This patch also updates `llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` to include `-run-twice` to exercise the cleanup. Without this patch `llvm-lit -v llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` fails with

```
-- Testing: 1 tests, 1 workers --
FAIL: LLVM :: CodeGen/X86/dwarf-eh-prepare.ll (1 of 1)
******************** TEST 'LLVM :: CodeGen/X86/dwarf-eh-prepare.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice < /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll -S | /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll
--
Exit Code: 2

Command Output (stderr):
--
Referencing function in another module!
  call void @_Unwind_Resume(i8* %ehptr) #1
; ModuleID = '<stdin>'
void (i8*)* @_Unwind_Resume
; ModuleID = '<stdin>'
in function simple_cleanup_catch
LLVM ERROR: Broken function found, compilation aborted!
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice -S
1.      Running pass 'Function Pass Manager' on module '<stdin>'.
2.      Running pass 'Module Verifier' on function '@simple_cleanup_catch'
 #0 0x000056121b570a2c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:569:0
 #1 0x000056121b56eb64 llvm::sys::RunSignalHandlers() /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Signals.cpp:97:0
 #2 0x000056121b56f28e SignalHandler(int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:397:0
 #3 0x00007fc7e9b22980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
 #4 0x00007fc7e87d3fb7 raise /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
 #5 0x00007fc7e87d5921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
 #6 0x000056121b4e1386 llvm::raw_svector_ostream::raw_svector_ostream(llvm::SmallVectorImpl<char>&) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:674:0
 #7 0x000056121b4e1386 llvm::report_fatal_error(llvm::Twine const&, bool) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/ErrorHandling.cpp:114:0
 #8 0x000056121b4e1528 (/home/arakaki/build/llvm-project/main/bin/opt+0x29e3528)
 #9 0x000056121adfd03f llvm::raw_ostream::operator<<(llvm::StringRef) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:218:0
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll

--

********************
********************
Failed Tests (1):
  LLVM :: CodeGen/X86/dwarf-eh-prepare.ll

Testing Time: 0.22s
  Failed: 1
```

Reviewed By: loladiro

Differential Revision: https://reviews.llvm.org/D110979
2021-10-02 21:50:35 -04:00
Simon Pilgrim
9452ec722c [X86][SSE] Fix typo + infinite-loop in HOP(HOP'(X,X),HOP'(Y,Y)) fold (PR52040)
PR52040 identified several issues with the HOP(HOP'(X,X),HOP'(Y,Y)) -> HOP(PERMUTE(HOP'(X,Y)),PERMUTE(HOP'(X,Y)) slow-HOP fold.

Not only was there a copy+paste typo when accessing the inner HOP operands, but the (unnecessary) ReplaceAllUsesOfValueWith call was missing one use checks.

Now that we have better shuffle combines of HOPs we can just return a new HOP() sequence and not use ReplaceAllUsesOfValueWith at all - this actually improved pair_sum_v8i32_v4i32 codegen as it kicks off further shuffle combines.
2021-10-02 15:31:12 +01:00
Simon Pilgrim
bb42cc2090 [X86] decomposeMulByConstant - decompose legal vXi32 multiplies on SlowPMULLD targets and all vXi64 multiplies
X86's decomposeMulByConstant never permits mul decomposition to shift+add/sub if the vector multiply is legal.

Unfortunately this isn't great for SSE41+ targets which have PMULLD for vXi32 multiplies, but is often quite slow. This patch proposes to allow decomposition if the target has the SlowPMULLD flag (i.e. Silvermont). We also always decompose legal vXi64 multiplies - even latest IceLake has really poor latencies for PMULLQ.

Differential Revision: https://reviews.llvm.org/D110588
2021-10-02 12:35:25 +01:00
Simon Pilgrim
df672f66b6 [DAG] scalarizeExtractedVectorLoad - replace getABITypeAlign with allowsMemoryAccess (PR45116)
One of the cases identified in PR45116 - we don't need to limit extracted loads to ABI alignment, we can use allowsMemoryAccess - which tests using getABITypeAlign, but also checks if a target permits (fast) misaligned memory loads by checking allowsMisalignedMemoryAccesses as a fallback.

I've also cleaned up the alignment calculation code - if we have a constant extraction index then the alignment can be based on an offset from the original vector load alignment, but for non-constant indices we should assume the worst (single element alignment only).

Differential Revision: https://reviews.llvm.org/D110486
2021-10-01 21:07:34 +01:00
Roman Lebedev
3c40719603
[NFC][X86][Codegen] Add test coverage for interleaved i8 load/store stride=4 2021-10-01 22:49:06 +03:00
Roman Lebedev
907d5da3da
[NFC][Codegen][X86] Drop unused check prefixes in newly added tests 2021-10-01 19:35:14 +03:00
Roman Lebedev
657e1e316b
[NFC][X86][Codegen] Add test coverage for interleaved i8 load/store stride=3 2021-10-01 18:47:25 +03:00
Sanjay Patel
97948620b1 [x86] add test for 3 fcmps and logic; NFC
This is a more complex pattern than we handled with the
initial patch for PR51245:
D110342 / 09e71c367af3

We could extend the logic matching to allow a setcc as
one operand and an extract of vector setcc (or even an
arbitrary bool?) as the other.
2021-09-30 11:02:59 -04:00
Roman Lebedev
3bd02ec977
[NFC][X86][Codegen] Add test coverage for interleaved i64 load/store stride=2 2021-09-30 17:31:18 +03:00
Roman Lebedev
7dffb8b4da
[NFC][X86][Codegen] Add test coverage for interleaved i32 load/store stride=2 2021-09-29 22:17:12 +03:00
Wesley Wiser
2dd883439c [Mangler] Calculate the argument list byte count suffix correctly when returning large values
`__stdcall`, `__fastcall` and `__vectorcall` return large values via a
hidden pointer argument. However, the size of that argument should not
be included in the argument list byte count suffix added to the
function's decorated name.

This patch fixes that issue so that LLVM generates the same decorated
name as MSVC does.

MSVC example: https://godbolt.org/z/nc35MKPhr

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D110719
2021-09-29 11:42:28 -07:00