19780 Commits

Author SHA1 Message Date
Simon Pilgrim
2cbf652615 [X86] avx512-pmovxrm.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.
2024-01-04 17:17:08 +00:00
Simon Pilgrim
63e3074781 [X86] aligned-variadic.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.
2024-01-04 17:17:07 +00:00
Simon Pilgrim
ce4459d590 [X86] 64-bit-shift-by-32-minus-y.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.
2024-01-04 17:17:07 +00:00
Simon Pilgrim
076dbc0272 [X86] SimplifyDemandedVectorEltsForTargetNode - add X86ISD::VZEXT_LOAD handling.
Simplify to a scalar_to_vector(load()) if we don't demand any of the upper vector elements.
2024-01-04 17:17:07 +00:00
Simon Pilgrim
5cd3cf1072 [X86] cvtv2f32.ll - replace X32 checks with X86. NFC.
We try to use X32 for gnux32 triples only.
2024-01-04 17:17:06 +00:00
HaohaiWen
e147dcbcbc
[SEH] Add test to track EHa register liveness verification (#76921)
This test tracks bug of MachineVerifier to check live range segment for
EHa. Async exception can happen at any place within seh scope, not only
the call instruction. Need to teach MachineVerifier to know that.
2024-01-04 20:49:11 +08:00
Phoebe Wang
176c341198 [X86][BF16] Add 32-bit tests to show ABI problem, NFC 2024-01-04 15:43:34 +08:00
Micah Weston
7df28fd61a
[SHT_LLVM_BB_ADDR_MAP][AsmPrinter] Implements PGOAnalysisMap emitting in AsmPrinter with tests. (#75202)
Uses machine analyses to emit PGOAnalysisMap into the bb-addr-map ELF
section. Implements filecheck tests to verify emitting new fields.

This patch emits optional PGO related analyses into the bb-addr-map ELF
section during AsmPrinter. This currently supports Function Entry Count,
Machine Block Frequencies. and Machine Branch Probabilities. Each is
independently enabled via the `feature` byte of `bb-addr-map` for the given
function.

A part of [RFC - PGO Accuracy Metrics: Emitting and Evaluating Branch and Block Analysis](https://discourse.llvm.org/t/rfc-pgo-accuracy-metrics-emitting-and-evaluating-branch-and-block-analysis/73902).
2024-01-03 19:17:44 -05:00
Craig Topper
47a1704ac9
[SelectionDAG][X86] Use disjoint flag in SelectionDAG::isADDLike. (#76847)
Keep the haveNoCommonBitsSet check because we haven't started inferring
the flag yet.

I've added tests for two transforms, but these are not the only
transforms that use isADDLike.
2024-01-03 11:54:29 -08:00
Arthur Eubanks
c4146121e9 Revert "Reapply "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG""
This reverts commit 0e46b49de43349f8cbb2a7d4c6badef6d16e31ae.

Causes crashes, see repro on 0e46b49de4.
2024-01-03 17:09:46 +00:00
Simon Pilgrim
1d27669e8a [X86] combineConcatVectorOps - fold 512-bit concat(blendi(x,y,c0),blendi(z,w,c1)) to AVX512BW mask select
Yet another yak shaving regression fix for #73509
2024-01-03 12:38:26 +00:00
Simon Pilgrim
39be138cb7 [X86] combineTargetShuffle - fold SHUF128(CONCAT(),CONCAT()) to peek through upper subvectors
If SHUF128 is accessing only the upper half of a vector source that is a concatenation/insert_subvector then try to access the subvector directly and adjust the element mask accordingly.
2024-01-03 11:09:14 +00:00
Simon Pilgrim
58a335a3f1 [X86] Fold concat_vectors(permq(x),permq(x)) -> permq(concat_vectors(x,x))
Handle a common subvector shuffle pattern in combineConcatVectorOps
2024-01-03 11:09:14 +00:00
Phoebe Wang
a384cd5012
[X86][BF16] Add subvec_zero_lowering patterns (#76507) 2023-12-31 11:14:41 +08:00
Phoebe Wang
6c87f46795 [X86][NFC] Remove meaningless FIXME
Solved by #76485.
2023-12-29 10:45:42 +08:00
Phoebe Wang
e499ae53b3
[X86][BF16] Support INSERT_SUBVECTOR and CONCAT_VECTORS (#76485) 2023-12-28 13:29:01 +08:00
Phoebe Wang
3081bacb60
[X86][BF16] Add X86SubVBroadcastld patterns (#76479) 2023-12-28 10:08:27 +08:00
Freddy Ye
8ddb0fcff9
[X86] Correct operand order of UWRMSR. (#76389) 2023-12-27 09:01:55 +08:00
Vettel
dc1fadef23
[MCP] Enhance MCP copy Instruction removal for special case(reapply) (#74239)
Machine Copy Propagation Pass may lose some opportunities to further
remove the redundant copy instructions during the ForwardCopyPropagateBlock
procedure. When we Clobber a "Def" register, we also need to remove the record 
from the copy maps that indicates "Src" defined "Def" to ensure the correct semantics
of the ClobberRegister function.  This patch reapplies #70778 and addresses the corner 
case bug  #73512 specific to the AMDGPU backend. Additionally, it refines the criteria 
for removing empty records from the copy maps, thereby enhancing overall safety.

For more information, please see the C++ test case generated code in 
"vector.body" after the MCP Pass: https://gcc.godbolt.org/z/nK4oMaWv5.
2023-12-26 16:22:42 +08:00
Matt Arsenault
0e46b49de4 Reapply "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG"
This reverts commit c398fa009a47eb24f88383d5e911e59e70f8db86.

PPC backend was fixed in 2f82662ce901c6666fceb9c6c5e0de216a1c9667
2023-12-22 16:46:22 +07:00
Arthur Eubanks
7433b1ca3e Reapply "[X86] Set SHF_X86_64_LARGE for globals with explicit well-known large section name (#74381)"
This reverts commit 19fff858931bf575b63a0078cc553f8f93cced20.

Now that explicit large globals are handled properly in the small code model.
2023-12-21 10:51:30 -08:00
Arthur Eubanks
2366d53d8d
[X86] Fix more medium code model addressing modes (#75641)
By looking at whether a global is large instead of looking at the code
model.

This also fixes references to large data in the small code model.

We now always fold any 32-bit offset into the addressing mode with the
large code model since it uses 64-bit relocations.
2023-12-21 10:40:56 -08:00
David Li
f44079db22
[ISel] Add pattern matching for depositing subreg value (#75978)
Depositing value into the lowest byte/word is a common code pattern.
This patch improves the code generation for it to avoid redundant AND
and OR operations.
2023-12-21 10:18:57 -08:00
Simon Pilgrim
6ec350b483 [X86] SimplifyDemandedVectorEltsForTargetShuffle - don't simplify constant mask if it has multiple uses
Avoid generating extra constant vectors
2023-12-20 15:22:48 +00:00
Jonas Paulsson
e32e147d6c
[DAGCombiner] Don't drop alignment info of original load. (#75626)
Pass the original MMO instead of different individual values.

getAlign() was used before where actually getOriginalAlign() would have been
better, and this patch has the same effect.
2023-12-19 16:30:47 +01:00
Arthur Eubanks
68c976bf64 [X86] Fix referencing local tagged globals
We should treat the medium code model like the small code model.
Classifying non-local references already properly handled this.
2023-12-17 13:49:50 -08:00
Stefan Pintilie
c398fa009a Revert "Reapply "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG""
This reverts commit f4b5be1ecdc85ca4257b739afb8d57e23c7a8030.

The above change was breaking the clang-ppc64le-linux-test-suite bot.
2023-12-16 07:30:53 -06:00
Arthur Eubanks
b3e353d263
[X86] Don't use rip-relative lea to get a function address in medium static mode (#75656)
This essentially reverts https://reviews.llvm.org/D140593. Somewhere
along the line we properly fixed the medium code model to assume
functions are small, so now we get a 32-bit movl as desired.
2023-12-15 15:15:18 -08:00
Paul Kirth
9a578a9f60
Revert "[StackColoring] Delete dead stack slots (#75351)" (#75655)
This reverts commit 08b306dc8e7c0b2498f4f194a3c51686d56dbd20.

it causes the following assertion failure:
llvm/include/llvm/CodeGen/MachineFrameInfo.h:530: int64_t
llvm::MachineFrameInfo::getObjectOffset(int) const: Assertion
`!isDeadObjectIndex(ObjectIdx) && "Getting frame offset for a dead
object?"' failed.
2023-12-15 13:32:39 -08:00
Arthur Eubanks
809ee6cfcf [X86][test] Update tagged-globals*.ll tests
Use update_llc_test_checks.py.

Split out jump table tests into separate file since we don't want to check the exact instruction sequence for it.
2023-12-15 12:54:55 -08:00
mohammed-nurulhoque
08b306dc8e
[StackColoring] Delete dead stack slots (#75351)
deletes slots that have lifetime markers and the lifetime ranges are empty.
2023-12-15 09:58:19 +00:00
Craig Topper
2a21260ea8 [SelectionDAG] Use getVectorElementPointer in DAGCombiner::replaceStoreOfInsertLoad. (#74249)
This ensures we clip the index to be in bounds of the vector we are
inserting into. If the index is out of bounds the results of the insert
element is poison. If we don't clip the index we can write memory that
was not part of the original store.

Fixes #74248 #75557.
2023-12-14 20:25:16 -08:00
Matt Arsenault
f4b5be1ecd Reapply "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG"
This reverts commit 69c4930aad9659ec6ab846c8e7124d6afe044b1e.

See if this sticks after a few more coalescer assertions are fixed.
2023-12-15 10:51:47 +07:00
Arthur Eubanks
239a41e8f2 Re-Reland [X86] Respect code models more when determining if a global reference can fit in 32 bits (#75386)
For non-GlobalValue references, the small and medium code models can use
32 bit constants.

For GlobalValue references, use TargetMachine::isLargeGlobalObject().
Look through aliases for determining if a GlobalValue is small or large.
Even the large code model can reference small objects with 32 bit
constants as long as we're in no-pic mode, or if the reference is offset
from the GOT.

Original commit broke the build...

First reland broke large PIC builds referencing small data since it was using GOTOFF as a 32-bit constant.
2023-12-14 14:12:37 -08:00
Jon Roelofs
640c1d3dd1
[llvm] Support IFuncs on Darwin platforms (#73686)
... by lowering them as lazy resolve-on-first-use symbol resolvers. Note that this is subtly different timing than on ELF platforms, where ifunc resolution happens at load time.

Since ld64 and ld-prime don't support all the cases we need for these, we lower them manually in the AsmPrinter.
2023-12-14 14:40:52 -07:00
Arthur Eubanks
15617d14f7 Revert "Reland [X86] Respect code models more when determining if a global reference can fit in 32 bits (#75386)"
This reverts commit ec92d74a0ef89b9dd46aee6ec8aca6bfd3c66a54.

Breaks some compiler-rt tests, e.g. https://lab.llvm.org/buildbot/#/builders/37/builds/28834
2023-12-14 12:28:50 -08:00
Arthur Eubanks
ec92d74a0e Reland [X86] Respect code models more when determining if a global reference can fit in 32 bits (#75386)
For non-GlobalValue references, the small and medium code models can use
32 bit constants.

For GlobalValue references, use TargetMachine::isLargeGlobalObject().
Look through aliases for determining if a GlobalValue is small or large.
Even the large code model can reference small objects with 32 bit
constants as long as we're in no-pic mode, or if the reference is offset
from the GOT.

Original commit broke the build...
2023-12-14 09:49:35 -08:00
Arthur Eubanks
f0c03da63c
Revert "[X86] Respect code models more when determining if a global reference can fit in 32 bits" (#75500)
Reverts llvm/llvm-project#75386

Breaks build.
2023-12-14 09:32:55 -08:00
Simon Pilgrim
88f1a2c50d [X86] combineLoad - allow constant loads to share matching 'lower constant bits' with larger VBROADCAST_LOAD/SUBV_BROADCAST_LOAD nodes
We already had separate support for VBROADCAST_LOAD - merge this with the generic load handling and add SUBV_BROADCAST_LOAD support as well.
2023-12-14 17:31:07 +00:00
Arthur Eubanks
5e38ba26d2
[X86] Respect code models more when determining if a global reference can fit in 32 bits (#75386)
For non-GlobalValue references, the small and medium code models can use
32 bit constants.

For GlobalValue references, use TargetMachine::isLargeGlobalObject().
Look through aliases for determining if a GlobalValue is small or large.
Even the large code model can reference small objects with 32 bit
constants as long as we're in no-pic mode, or if the reference is offset
from the GOT.
2023-12-14 09:28:27 -08:00
Simon Pilgrim
3c423722cf [X86] combineLoad - improve constant pool matches by ignoring undef elements
When trying to share constant pool entries, we can ignore the undef elements of the entry that is being removed
2023-12-14 15:13:39 +00:00
Simon Pilgrim
2141a51be1 [X86] broadcast-elm-cross-splat-vec.ll - drop constant pool check
This is handled in the assembly comments.
2023-12-14 12:10:33 +00:00
Arthur Eubanks
c64334fb30
[X86][FastISel] Support medium code model in more places (#75375)
The medium code model is basically identical to the small code model
except that large objects cannot be referenced with 32-bit offsets.
2023-12-13 13:44:37 -08:00
Arthur Eubanks
e8f43883a0 [X86][test] Use separate check prefix in code-model-elf.ll
Since these will produce different results in upcoming changes.
2023-12-13 13:05:59 -08:00
Matt Arsenault
300a55003c
RegisterCoalescer: Fix implicit operand handling during rematerialize (#75271)
If the rematerialize was placing a subregister into a super register,
and implicit operands referenced the original register, we need to add
undef flags to the now-subregister indexed implicit operands.

Depends #75152
2023-12-13 15:15:36 +07:00
Matt Arsenault
538a83e4b9
RegisterCoalescer: Add undef flags in removePartialRedundancy (#75152)
If the copy being hoisted was undef, we have the same problems that
eliminateUndefCopy needs to solve. We would effectively be introducing a
new live out implicit_def. We need to add an undef flag to avoid
artificially introducing a live through undef value. Previously, the
verifier would fail due to the dead def inside the loop providing the
live in value for the %1 use.
2023-12-13 15:02:53 +07:00
Arthur Eubanks
19fff85893 Revert "[X86] Set SHF_X86_64_LARGE for globals with explicit well-known large section name (#74381)"
This reverts commit 323451ab88866c42c87971cbc670771bd0d48692.

Code with these section names in the wild doesn't compile because
support for large globals in the small code model is not complete yet.
2023-12-12 16:31:41 -08:00
Arthur Eubanks
3959231695 [X86][FastISel] Bail out on large objects when materializing a GlobalValue
To avoid crashes with explicitly large objects.

I will clean up fast-isel with large objects/medium code model soon.
2023-12-12 12:45:20 -08:00
Simon Pilgrim
06613095c4 [X86] avx512-vbroadcast.ll - fix orphan check prefixes
The AVX512F/AVX512BW checks had been removed despite still being used
2023-12-12 18:34:53 +00:00
Evgenii Kudriashov
ef35da825f
[X86][GlobalISel] Add instruction selection for G_SELECT (#70753) 2023-12-12 16:08:08 +01:00