20098 Commits

Author SHA1 Message Date
Simon Pilgrim
74ab7958bd [X86] printZeroUpperMove - add support for constant vectors.
Allows cases where movss/movsd etc. are loading constant (ConstantDataSequential) sub-vectors, ensuring we pad with the correct number of zero upper elements by making repeated printConstant calls to print zeroes in a matching int/fp format.
2024-01-22 15:40:46 +00:00
XinWang10
dd6fec5d4f
[X86][APX]Support lowering for APX promoted AMX-TILE instructions (#78689)
The enc/dec of promoted AMX-TILE instructions have been supported in
https://github.com/llvm/llvm-project/pull/76210.
This patch support lowering for promoted AMX-TILE instructions and
integrate test to existing tests.
2024-01-22 11:33:23 +08:00
XinWang10
d3cd1ce6ab
[X86] Add lowering tests for promoted CMPCCXADD and update CC representation (#78685)
https://github.com/llvm/llvm-project/pull/76125 supported the enc/dec
for CMPCCXADD instructions, this patch
1. Add lowering test for promoted CMPCCXADD
2. Update the representation of condition code for promoted CMPCCXADD to
align with the existing one
2024-01-22 11:32:03 +08:00
Arthur Eubanks
86eaf6083b
[X86] Refine X86DAGToDAGISel::isSExtAbsoluteSymbolRef() (#76191)
We just need to check if the global is large or not.

In the kernel code model, globals are in the negative 2GB of the address
space, so globals can be a sign extended 32-bit immediate.

In other code models, small globals are in the low 2GB of the address
space, so sign extending them is equivalent to zero extending them.
2024-01-19 14:11:18 -08:00
Simon Pilgrim
a2a0089ac3
[X86] movsd/movss/movd/movq - add support for constant comments (#78601)
If we're loading a constant value, print the constant (and the zero upper elements) instead of just the shuffle mask.

This did require me to move the shuffle mask handling into addConstantComments as we can't handle this in the MC layer.
2024-01-19 14:21:26 +00:00
Danila Malyutin
9ad7d8f0e4
[Statepoint] Optimize Location structure size (#78600)
Reduce its size from 24 to 12 bytes. Improves memory consumption when
dealing with statepoint-heavy code.
2024-01-19 17:15:36 +04:00
Haohai Wen
fb2c6bbf42
[BranchFolding] Use isSuccessor to confirm fall through (#77923)
When merging blocks, if the previous block has no any branch instruction
and has one successor, the successor may be SEH landing pad and the
block will always raise exception and nerver fall through to next block.
We can not merge them in such case. isSuccessor should be used to
confirm it can fall through to next block.
2024-01-18 23:26:22 +08:00
Simon Pilgrim
33287e35f2
[X86] Emit verbose (constant) comments before EVEX compression tag (#78585)
This helps ensure the encoding details are next to the EVEX tag

Noticed while preparing to add more constant commenting as part of #73783 and #71078
2024-01-18 15:13:42 +00:00
Florian Hahn
40d952b874
[CGP] Avoid replacing a free ext with multiple other exts. (#77094)
Replacing a free extension with 2 or more extensions unnecessarily
increases the number of IR instructions without providing any benefits.
It also unnecessarily causes operations to be performed on wider types
than necessary.

In some cases, the extra extensions also pessimize codegen (see
bfis-in-loop.ll).

The changes in arm64-codegen-prepare-extload.ll also show that we avoid
promotions that should only be performed in stress mode.

PR: https://github.com/llvm/llvm-project/pull/77094
2024-01-18 10:48:10 +00:00
XinWang10
2d92f7de80
[X86] Support lowering for APX promoted BMI instructions. (#77433)
R16-R31 was added into GPRs in
https://github.com/llvm/llvm-project/pull/70958,
This patch supports the lowering for promoted BMI instructions in EVEX
space, enc/dec has been supported in
https://github.com/llvm/llvm-project/pull/73899.

RFC:
https://discourse.llvm.org/t/rfc-design-for-apx-feature-egpr-and-ndd-support/73031/4
2024-01-18 10:15:54 +08:00
XinWang10
f6617091a9
[X86][test] Add --show-mc-encoding for lowering tests of NDD arithmetic instructions (#78406)
#77564 added lowering tests for NDD arithmetic instructions.
It would be great to add `--show-mc-encoding` to check the NDD variant
is selected first.
2024-01-18 09:29:02 +08:00
Arthur Eubanks
00647a18ce
[X86] Don't respect large data threshold for globals with an explicit section (#78348)
If multiple globals are placed in an explicit section, there's a chance
that the large data threshold will cause the different globals to be
inconsistent in whether they're large or small. Mixing sections with
mismatched large section flags can cause undesirable issues like
increased relocation pressure because there may be 32-bit references to
the section in some TUs, but the section is considered large since input
section flags are unioned and other TUs added the large section flag.

An explicit code model on the global still overrides the decision. We
can do this for globals without any references to them, like what we did
with asan_globals in #74514. If we have some precompiled small code
model files where asan_globals is not considered large mixed with
medium/large code model files, that's ok because the section is
considered large and placed farther. However, overriding the code model
for globals in some TUs but not others and having references to them
from code will still result in the above undesired behavior.

This mitigates a whole class of mismatched large section flag issues
like what #77986 was trying to fix.

This ends up not adding the SHF_X86_64_LARGE section flag on explicit
sections in the medium/large code model. This is ok for the large code
model since all references from large text must use 64-bit relocations
anyway.
2024-01-17 15:38:32 -08:00
Simon Pilgrim
d92ce344bf Revert faecc736e2ac3cd8c77 #74443 [DAG] isSplatValue - node is a splat if all demanded elts have the same whole constant value (#74443)
Relying on ComputeKnownBits to find a splat is causing miscompilations where a shift of zero is being assumed to give zero, but further simplification leads to a shift of zero by undef, resulting in an unexpected undef value.

Fixes #78109
2024-01-17 15:59:33 +00:00
Nikita Popov
cde780c18f
[DAGCombine] Add debug counter (#78259)
Add a debug counter for DAGCombine. This can help with bisecting which
DAG combine introduced a miscompile.
2024-01-17 09:31:56 +01:00
Fangrui Song
d4cb5d9f2b
[X86] Add "Ws" constraint and "p" modifier for symbolic address/label reference (#77886)
Printing the raw symbol is useful in inline asm (e.g. getting the C++
mangled name, referencing a symbol in a custom way while ensuring it is
not optimized out even if internal). Similar constraints are available
in other targets (e.g. "S" for aarch64/riscv, "Cs" for m68k).

```
namespace ns { extern int var, a[4]; }
void foo() {
  asm(".pushsection .xxx,\"aw\"; .dc.a %p0; .popsection" :: "Ws"(&ns::var));
  asm(".reloc ., BFD_RELOC_NONE, %p0" :: "Ws"(&ns::a[3]));
}
```

Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105576
2024-01-16 23:57:42 -08:00
Danila Malyutin
46a929f0a0
[SelectionDAG] Fix isKnownNeverZeroFloat for vectors (#78308)
Return true iff all of vector elements are constant AND not zero

Fixes #77805

Previously, it'd return `true` (as in - the value is known to be never
zero) for any build_vector/splat_vector with non-constant elements.
2024-01-17 12:55:57 +07:00
Phoebe Wang
8d6e82d501
[X86] Use vXi1 for k constraint in inline asm (#77733)
Fixes #77172
2024-01-17 11:40:32 +08:00
Phoebe Wang
9745c13ca8
[X86][BF16] Improve float -> bfloat lowering under AVX512BF16 and AVXNECONVERT (#78042) 2024-01-17 10:09:26 +08:00
Rahman Lavaee
e1616ef9d7
[BasicBlockSections] Always keep the entry block in the beginning of the function. (#74696)
BasicBlockSections must enforce placing the entry block at the beginning
of the function regardless of the basic block sections profile.
2024-01-16 14:15:33 -08:00
Simon Pilgrim
d6ee91b110 [X86] Add test case for Issue #77805 2024-01-16 17:23:45 +00:00
Simon Pilgrim
662d1cb86b [X86] Add test case for Issue #78109 2024-01-16 16:33:21 +00:00
Shengchen Kan
b1eaffd389 [X86][test] Add test for lowering NDD AND
We supported encoding/decoding for APX AND in #76319

This test should be added in #77564 but was missing.
2024-01-16 20:09:40 +08:00
Nicholas Mosier
49138d97c0
[X86] Fix SLH crash on llvm.eh.sjlh.longjmp (#77959)
Fix #60081.
2024-01-14 12:03:18 +08:00
Fangrui Song
7e604485e1 [test] Improve x86 inline asm tests
Reorganize *asm-modifier* and make other cleanups.
2024-01-11 23:35:46 -08:00
Shengchen Kan
4f71068b72 [X86] Correct the asm comment for compression NF_ND -> NF 2024-01-12 12:55:11 +08:00
Shengchen Kan
9095eec052
[X86][CodeGen] Support EVEX compression: NDD to nonNDD (#77731) 2024-01-12 10:03:30 +08:00
HaohaiWen
b6fc463d4c
[SEH] Redirect test output to /dev/null (#77784) 2024-01-11 23:31:57 +08:00
HaohaiWen
f892cc36fd
[BranchFolding] Fix missing predecessors of landing-pad (#77608)
When removing an empty machine basic block, all of its successors should
be inherited by its fall through MBB. This keeps CFG as only have one
entry which is required by LiveDebugValues.

Reland #77441 as LiveDebugValues test.
2024-01-11 22:09:41 +08:00
Shengchen Kan
e4e0b65838 [X86][test] Pre-commit test for #77731 2024-01-11 18:51:34 +08:00
Simon Pilgrim
7bf13fe812
[DAG] Fold (sext (sext_inreg x)) -> (sext (trunc x)) if the trunc is free (#77616) 2024-01-11 09:39:30 +00:00
Shengchen Kan
1fe7bdb87b
[X86][CodeGen] Support lowering for NDD ADD/SUB/ADC/SBB/OR/XOR/NEG/NOT/INC/DEC/IMUL (#77564)
We supported encoding/decoding for these instructions in

https://github.com/llvm/llvm-project/pull/76319
https://github.com/llvm/llvm-project/pull/76721
https://github.com/llvm/llvm-project/pull/76919
2024-01-11 12:15:17 +08:00
HaohaiWen
9bde5becb4
[BranchFolding][SEH] Add test to track SEH CFG optimization (#77598)
This test tracks BranchFolding pass which removes fall through jump and
leaves landing-pad to be machine basic block of no predecessors. It
would raise bug as introduced in #77441.
2024-01-10 22:34:18 +08:00
Simon Pilgrim
cc21aa1922 [X86] lower1BitShuffle - fold permute(setcc(x,y)) -> setcc(permute(x),permute(y)) for 32/64-bit element vectors
Noticed in #77459 - for wider element types, its usually better to pre-shuffle the comparison arguments if we can, like we already for broadcasts
2024-01-10 12:35:50 +00:00
Simon Pilgrim
78cf2c041b [X86] pr77459.ll - add missing AVX512 check prefixes
Missed these in 3210ce276350a247220b193db12a9b45d1034724 for the #77459 fix
2024-01-10 12:09:38 +00:00
HaohaiWen
c9124adfd8
Revert "[SEH][CodeGen] Add test to track CFG optimization bug for SEH" (#77542)
Reverts llvm/llvm-project#77441
I'll land it with fix.
2024-01-10 09:25:45 +08:00
Zequan Wu
4e8986fc58
[Coverage] Mark coverage sections as metadata sections on COFF. (#76834)
Mark `.lcovmap$M`, `.lcovfun$M`, `.lcovd` and `.lcovn` as metadata
sections on COFF so they are not loaded into memory.
2024-01-09 16:58:28 -05:00
Simon Pilgrim
3210ce2763 [X86] Fold (iX bitreverse(bitcast(vXi1 X))) -> (iX bitcast(shuffle(X)))
X86 doesn't have a BITREVERSE instruction, so if we're working with a casted boolean vector, we're better off shuffling the vector instead if we have PSHUFB (SSSE3 or later)

Fixes #77459
2024-01-09 19:06:32 +00:00
Simon Pilgrim
417df8ee4a [X86] Add test coverage for #77459 2024-01-09 18:55:05 +00:00
Fangrui Song
f972e4d343 [MC,ELF] .section: unconditionally print section flag 'G' after 'o'
* Placing 'G' before 'M' (SHF_MERGE) can be misleading as the sh_entsize
  argument goes before the section group name, if a reader doesn't know
  that the order of extra arguments is not affected by the order of flags.
* 'a', 'w', and 'x' indicate basic permission-related flags. Separating
  them with 'G' is kinda ugly.

Simplify code and move 'G' after 'o'. The new output is more similar to
GCC.
2024-01-09 10:48:23 -08:00
Fangrui Song
7620f03ef7
[MC] Parse SHF_LINK_ORDER argument before section group name (#77407)
When both SHF_LINK_ORDER | SHF_GROUP flags are set, GNU assembler from
2.35 onwards (https://sourceware.org/PR25381
https://sourceware.org/binutils/docs/as/Section.html) parses the
SHF_LINK_ORDER argument before section group name, different from us.

This is unfortunate, but does not matter because the `.section` flag `o`
is a niche feature only used by compiler instrumentations, not adopted
by hand-written assembly, and using both flags is extremely rare. Let's
just match GNU assembler. There is another benefit: we now support
zero-flag section group with the SHF_LINK_ORDER flag, while previously
there isn't a syntax.

While here, print 'G' after 'o' to be clear that the 'G' argument is
parsed after the 'o' argument. To make the diff smaller, we don't print
'G' after 'w' in the absence of 'o' for now.
2024-01-09 10:42:34 -08:00
HaohaiWen
a2dba0c977
[SEH][CodeGen] Add test to track CFG optimization bug for SEH (#77441)
LiveDebugValues requires CFG only has one entry. BranchFolding and
MachineBlockPlacement may remove all predecessors of landing pad which
leaves it to be another entry.
2024-01-09 22:30:13 +08:00
Nick Anderson
f1ec0d12bb
Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#77182)
Port CodeGenPrepare to new pass manager and dependency
BasicBlockSectionsProfileReader
Fixes: #75380

Co-authored-by: Krishna-13-cyber <84722531+Krishna-13-cyber@users.noreply.github.com>
2024-01-09 13:32:59 +07:00
Shengchen Kan
38ce770ef1 [X86][test] Add test to check ah is not allocatable for register class gr8_norex2
This test should be added after #73529
2024-01-09 14:28:38 +08:00
Simon Pilgrim
d460c1de3b
[DAG] SimplifyDemandedBits - don't fold sext(x) -> aext(x) if we lose an 0/-1 allsignbits mask (#77296)
For targets that use 0/-1 boolean results, we want to keep this pattern through extensions/truncations as much as possible - so avoid simplifying to any_extend even if we don't demand the upper bits.

Noticed in triage for https://reviews.llvm.org/D152928
2024-01-08 18:01:41 +00:00
Simon Pilgrim
52ebf61bac [X86] ftrunc.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.

Add common AVX check prefix for 32/64 bit test coverage
2024-01-08 17:25:44 +00:00
Simon Pilgrim
fbfc9cb7ea [X86] vector-shuffle-mmx.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.

Add nounwind to remove cfi noise as well.
2024-01-08 17:25:44 +00:00
Simon Pilgrim
9632f98716 [X86] legalize-shl-vec.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.

Add nounwind to remove cfi noise as well.
2024-01-08 17:25:44 +00:00
Simon Pilgrim
635f6d3845 [X86] inline-sse.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.
2024-01-08 17:25:44 +00:00
Simon Pilgrim
8bd16789ff [X86] lea-2.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only (although in this case the gnux32 tests share the X64 checks)
2024-01-08 17:25:43 +00:00
Simon Pilgrim
61dcfaa745 [X86] i64-mem-copy.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.

Add nounwind to remove cfi noise as well.
2024-01-08 17:25:43 +00:00