20098 Commits

Author SHA1 Message Date
Simon Pilgrim
c7198e0af3
[DAG] Fold insert_subvector(N0, extract_subvector(N0, N2), N2) --> N0 (#86487)
Handle the case where we've ended up inserting back into the source vector we extracted the subvector from.
2024-03-26 10:03:42 +00:00
Evgenii Kudriashov
fb394562a3
[X86][GlobalISel] Fix referencing nonexistent operand in G_ICMP (#86221)
Fixes #86203
2024-03-25 16:46:12 +01:00
AtariDreams
f5a067bb90
[SelectionDAG]: Deduce KnownNeverZero from SMIN and SMAX (#85722) 2024-03-25 10:35:28 +00:00
Phoebe Wang
2e4e04c590
[X86][BF16] Do not lower to VCVTNEPS2BF16 without AVX512VL (#86395)
Fixes: #86305
2024-03-25 10:06:12 +08:00
Simon Pilgrim
6c6fe4b2ae [X86] known-never-zero.ll - add 32-bit test coverage
Enabled vector coverage as well: i686+SSE2 and x64_64+AVX

Should improve test quality for #85722
2024-03-24 11:33:51 +00:00
Evgenii Kudriashov
d365a45cb3
[GlobalISel] Introduce G_TRAP, G_DEBUGTRAP, G_UBSANTRAP (#84941)
Here we introduce three new GMIR instructions to cover a set of trap
intrinsics. The idea behind it is that generic intrinsics shouldn't be
used with G_INTRINSIC opcode.

These new instructions can match perfectly with existing trap ISD nodes.
It allows X86, AArch64, RISCV and Mips to reuse SelectionDAG patterns for
selection and avoid manual selection. However AMDGPU is an exception. It
selects traps during legalization regardless SelectionDAG or GlobalISel.

Since there are not many places where traps are used, this change
attempts to clean up all the usages of G_INTRINSIC with trap intrinsics. So,
there is no stage when both G_TRAP and
G_INTRINSIC_W_SIDE_EFFECTS(@llvm.trap) are allowed.
2024-03-23 13:12:44 +01:00
Simon Pilgrim
a277dd82d8 [X86] vector-half-conversions.ll - add v4f16->v4i32 fptosi/fptoui test coverage 2024-03-22 13:54:42 +00:00
Simon Pilgrim
74c3150ffc [X86] Add shuffle tests from Issue #86076
SLP should be doing a better job, but both shuffles lower to poorer codegen than necessary
2024-03-22 11:23:36 +00:00
XChy
cb4453dc69
[SelectionDAG] Prevent combination on inconsistent type in combineCarryDiamond (#84888)
Fixes #84831
When matching carry pattern with `getAsCarry`, it may produce different
type of carryout. This patch checks such case and does early exit.

I'm new to DAG, any suggestion is appreciated.
2024-03-22 16:05:20 +05:30
Freddy Ye
3e4caa9da4
[X86] Support DomainReassignment for APX NDD instructions (#85737) 2024-03-22 08:52:40 +08:00
Jonas Paulsson
7564566779 Reapply "Move assertion for AdjustsStack from PEI to MachineVerifier (#85698)"
- The check is now actually done in both PEI and the MachineVerifier.
- More .mir tests trivially updated with "adjustsStack: true" as needed.
2024-03-21 20:24:57 -04:00
Simon Pilgrim
3218570620 [X86] Add shuffle test case for Issue #86068 2024-03-21 17:56:58 +00:00
AtariDreams
7e72cafd68
[SelectionDAG] Add MaskedValueIsZero check to allow folding of zero extended variables we know are safe to extend (#85573)
Add ones for every high bit that will cleared.

This will allow us to evaluate variables that have their bits known to
see if they have no risk of overflow despite the shift amount being
greater than the difference between the two types.
2024-03-21 16:45:17 +05:30
Freddy Ye
07a5e31cb3
Move pre-commit test for #85737 (#86062) 2024-03-21 10:55:26 +08:00
Freddy Ye
35a66f965c
Precommit test for #85737 (#86056)
Copied from llvm/test/CodeGen/X86/domain-reassignment.mir
2024-03-21 10:19:28 +08:00
Jonas Paulsson
9ebd329ad8 Revert "Move assertion for AdjustsStack from PEI to MachineVerifier. (#85698)"
This reverts commit 05bde30585710a51592eee0a6cf6df8184d09c92.

Reverting due to verifier complaints with expensive checks on build-bot.
2024-03-20 11:48:30 -04:00
Jonas Paulsson
05bde30585
Move assertion for AdjustsStack from PEI to MachineVerifier. (#85698)
Have the verifier report a missing AdjustsStack flag rather than waiting until
PEI asserts.
2024-03-20 10:29:12 -04:00
Benjamin Kramer
5f5a64134b Revert "[DAGCombiner] Simplifying {si|ui}tofp when only signbit is needed"
This reverts commit 353fbeb0a294d2c7cef6d88607fa0fd50ee81462. It crashes
when it encounters an UINT_TO_FP.

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1618 in SDValue llvm::SelectionDAG::getConstant(const ConstantInt &, const SDLoc &, EVT, bool, bool): VT.isInteger() && "Cannot create FP integer constant!"
2024-03-20 15:08:37 +01:00
Noah Goldstein
353fbeb0a2 [DAGCombiner] Simplifying {si|ui}tofp when only signbit is needed
If we only need the signbit `uitofp` simplified to 0, and `sitofp`
simplifies to `bitcast`.

Closes #85138
2024-03-19 17:17:35 -05:00
Noah Goldstein
ebd1379663 [DAGCombiner] Add tests for simplifying {si|ui}tofp; NFC 2024-03-19 17:17:35 -05:00
Simon Pilgrim
2377b9773d [DAG] SimplifyShift - shift i1/vXi1 X, Y --> X (any non-zero shift amount is undefined).
Alive2: https://alive2.llvm.org/ce/z/SdESbg

Fixes #85681
2024-03-19 20:18:37 +00:00
Simon Pilgrim
66125ad8e9 [X86] Add test coverage for vector avgceils/avgceilu/avgfloors/avgflooru test patterns
SSE only has AVGCEILU vXi8/vXi16 support - but for other types we should be trying to use the fixed width expansion instead of extensions
2024-03-19 15:32:40 +00:00
Simon Pilgrim
9f433bf8ca [X86] Add PAVG(0,x) test coverage for PR #85581 2024-03-19 12:39:47 +00:00
Jonas Paulsson
09bc6abba6
[MachineFrameInfo] Refactoring around computeMaxcallFrameSize() (NFC) (#78001)
- Use computeMaxCallFrameSize() in PEI::calculateCallFrameInfo() instead of duplicating the code.

- Set AdjustsStack in FinalizeISel instead of in computeMaxCallFrameSize().
2024-03-18 10:37:59 -04:00
XinWang10
7b766a6f50
[X86] Support APX CMOV/CFCMOV instructions (#82592)
This patch support ND CMOV instructions and CFCMOV instructions.

RFC:
https://discourse.llvm.org/t/rfc-design-for-apx-feature-egpr-and-ndd-support/73031/4
2024-03-17 20:18:56 +08:00
Fangrui Song
f4335f075b
[X86,AsmPrinter] Set assembler dialect for module inline asm
`clang -c -masm=intel` compiling a source file with file scope basic asm
incorrectly uses the AT&T dialect.
```
% cat a.c
asm("mov rax, rax");
% clang a.c -c -masm=intel
<inline asm>:1:1: error: unknown use of instruction mnemonic without a size suffix
mov rax, rax
^
```

Fix this by setting the assembler dialect from the MCAsmInfo object.

Note: `clang -c -flto -masm=intel a.c` still fails because of
https://reviews.llvm.org/D82862 for #34830: it tried to support AT&T
syntax for clang-cl, but the forced AT&T syntax is not compatible with
intended Intel syntax.

Pull Request: https://github.com/llvm/llvm-project/pull/85367
2024-03-15 09:38:07 -07:00
SahilPatidar
186565513c
[X86][AVX] Fix handling of out-of-bounds SRA shift amounts in AVX2 vector shift nodes (#84426) 2024-03-15 16:34:33 +00:00
Phoebe Wang
f4676b6be6
[X86] Add Support for X86 TLSDESC Relocations (#83136) 2024-03-15 22:09:56 +08:00
Ganesh
61fadd0b09
[X86] Fast AVX-512-VNNI vpdpwssd tuning (#85375)
Adding a tuning feature to fix
https://github.com/llvm/llvm-project/issues/84182
Generates vpdpwssd (instead of vpmaddwd + vpaddd sequence)
2024-03-15 16:45:41 +05:30
Jonas Paulsson
6588ac3017
[MachineCombiner] Don't ignore PHI depths (#82025)
The depths of the Root and the NewRoot are to be compared in
MachineCombiner::improvesCriticalPathLen(), and while the call to
BlockTrace.getInstrCycles(*Root) includes the Depth of a PHI, for some
reason PHI nodes have been ignored in getOperandDef(). 

This patch removes the special handling of PHIs in getOperandDef() so that
Root and NewRoot get a fair comparison. This does not affect loop headers
as MachineTraceMetrics handles that case by ignoring incoming PHI edges.
2024-03-14 11:47:29 -04:00
Nikita Popov
20b15e645c [Tests] Drop inrange attribute from some tests (NFC)
These don't actually test anything related to inrange, so drop the
attribute.
2024-03-13 11:49:16 +01:00
Arthur Eubanks
6bbb73b4cb
[X86] Fix determining if globals with size <8 bits are large (#84975)
Previously any global under 8 bits would accidentally be considered 0
sized, which is considered a large global.
2024-03-12 12:43:29 -07:00
Arthur Eubanks
45219702e7 [test][X86] Precommit test for large data threshold and i1 global 2024-03-12 19:08:40 +00:00
Simon Pilgrim
c1af6ab505 [X86] getFauxShuffleMask - recognise CONCAT(SUB0, SUB1) style patterns
Handles the INSERT_SUBVECTOR(INSERT_SUBVECTOR(UNDEF,SUB0,0),SUB1,N) pattern

Currently limited to v8i64/v8f64 cases as only AVX512 has decent cross lane 2-input shuffles, the plan is to relax this as I deal with some regressions
2024-03-12 17:40:19 +00:00
Bjorn Pettersson
4d0f79e346 Pre commit test cases SRL/SRA support in canCreateUndefOrPoison. NFC
Add test cases to show that we can't push freeze through SRA/SRL with
'exact' flag when there are multiple uses.
2024-03-12 16:03:18 +01:00
Phoebe Wang
e89b4bcf32
[X86] Remove SlowDivide tuning from GRTTuning (#84676)
The DIV32/64 throughput was improved since Goldmont in the Atom
architecture. The Alder Lake-E shows similar number too. So we shouldn't
add such tunings to Gracemont and later products.

Checked from Agner Fog's table and uops.info.
2024-03-12 13:41:49 +08:00
Simon Pilgrim
6cd68c2f87 [X86] Add base SSE2 coverage to SRL/SRA combines tests 2024-03-11 16:25:05 +00:00
Simon Pilgrim
7dc4d5f6a0 [X86] Add AVX512 (x86-64-v4) coverage to generic shift combines tests 2024-03-11 16:22:47 +00:00
Simon Pilgrim
862c7e0218 [X86] combineAndShuffleNot - ensure the type is legal before create X86ISD::ANDNP target nodes
Fixes #84660
2024-03-10 16:23:51 +00:00
Simon Pilgrim
92d7aca441
[X86] Add missing immediate qualifier to the (V)CMPSS/D instructions (#84496)
Matches (V)CMPPS/D and makes it easier to algorithmically recreate the instruction name in various analysis scripts I'm working on
2024-03-09 16:21:25 +00:00
Jay Foad
fd3eaf76ba
[GISel] Enforce G_PTR_ADD RHS type matching index size for addr space (#84352) 2024-03-09 09:07:22 +00:00
Evgenii Kudriashov
10edabbcf3
[X86][GlobalISel] Enable G_SDIV/G_UDIV/G_SREM/G_UREM (#81615)
* Create a libcall for s64 type for 32 bit targets.
* Fix a bug in REM selection: SUBREG_TO_REG is not intended to produce a
value from super registers.
* Replace selector tests by end-to-end tests. Other passes
check the selected MIR better.
2024-03-08 00:10:53 +01:00
Noah Goldstein
9f96db8e31 [X86] Fold (icmp ult (add x,-C),2) -> (or (icmp eq X,C), (icmp eq X,C+1)) for Vectors
This is undoing a middle-end transform which does the opposite. Since
X86 doesn't have unsigned vector comparison instructions pre-AVX512,
the simplified form gets worse codegen.

Fixes #66479

Proofs: https://alive2.llvm.org/ce/z/UCz3wt

Closes #84104
Closes #66479
2024-03-07 13:12:09 -06:00
Noah Goldstein
3e73a080fa [X86] Add tests for folding (icmp ult (add x,-C),2) -> (or (icmp eq X,C), (icmp eq X,C+1)); NFC 2024-03-07 13:12:09 -06:00
SahilPatidar
9e0f5909d0
[DAG] Fix Failure to reassociate SMAX/SMIN/UMAX/UMIN (#82175)
Resolve #58110
2024-03-07 15:15:17 +00:00
Fangrui Song
e63ea9d6f7 [CommandFlags] Rename option -relax-elf-relocations to -x86-relax-relocations
relax-elf-relocations is misleading and there were AMDGPU/SystemZ tests
misusing this x86-specific option.
2024-03-06 23:03:11 -08:00
Simon Pilgrim
0bd9255f8a
[X86] Improve KnownBits for X86ISD::PSADBW nodes (#83830)
Don't just return the known zero upperbits, compute the absdiff Knownbits and perform the horizontal sum.

Add implementations that handle both the X86ISD::PSADBW nodes and the INTRINSIC_WO_CHAIN intrinsics (pre-legalization).
2024-03-06 17:23:15 +00:00
Benjamin Kramer
55c466da2f [X86][AVX512BF16] Add a few missing insert/extract patterns
These are really the same as the f16 (and i16) instructions, but we need
them for any type that can occur.
2024-03-06 00:52:29 +01:00
elhewaty
26058e68ea
[DAG] select (sext m), (add X, C), X --> (add X, (and C, (sext m)))) (#83640)
- [DAG][X86] Add tests for Folding select m, add(X, C), X --> add (X, and(C, m))(NFC)
- [DAG][X86] Fold select (sext m), (add X, C), X --> (add X, (and C, (sext m))))
- Fixes: https://github.com/llvm/llvm-project/issues/66101
2024-03-05 16:41:41 +00:00
Yeting Kuo
d95a0d7c0f
[DAG] Teach SelectionDAGBuilder to read parameter alignment of compressstore/expandload. (#83763)
Previously SelectionDAGBuilder used ABI alignment for
compressstore/expandload. This patch allows SelectionDAGBuilder to use
parameter alignment like vp intrinsics. This does not follow the
original code to default use vector type alignment, since it is possible
implemented to unaligned vector alignment.
2024-03-05 20:48:37 +08:00