35575 Commits

Author SHA1 Message Date
Jay Foad
a6170d5b7e
[SelectionDAG] Dump convergencectrl_glue DAG node (#87487) 2024-04-03 16:21:57 +01:00
Simon Pilgrim
39eedfded4 [DAG] visitADDLikeCommutative - convert (add x, shl(0 - y, n)) fold to SDPatternMatch. NFC. 2024-04-03 15:37:38 +01:00
aniplcc
d650fcd6bf
[DAG] SimplifyDemandedVectorElts - add ISD::AVGCEILS/AVGCEILU/AVGFLOORS/AVGFLOORU nodes (#86284)
Fixes #84768
2024-04-03 15:00:50 +01:00
AinsleySnow
52b18430ae
[VP][DAGCombine] Use simplifySelect when combining vp.select. (#87342)
Hi all,

This patch is a follow-up of #79101. It migrates logic from
`visitVSELECT` to `visitVP_SELECT` to simplify `vp.select`. With this
patch we can do the following combinations:

```
vp.select undef, T, F --> T (if T is a constant), F otherwise
vp.select <condition>, undef, F --> F
vp.select <condition>, T, undef --> T
vp.select false, T, F --> F
vp.select <condition>, T, T --> T
```

I'm a total newbie to llvm and I'm sure there's room for improvements in
this patch. Please let me know if you have any advice. Thank you in
advance!
2024-04-03 07:45:50 -04:00
Gleb Popov
0356d0cfdc
Print more descriptive error message when trying to link a global with appending linkage (#69613)
This is a proper fix for https://github.com/llvm/llvm-project/issues/40308
2024-04-03 12:26:12 +01:00
Elizaveta Noskova
4dd103e9c6
[CodeGen][ShrinkWrap] Clarify StackAddressUsedBlockInfo meaning (#80679) 2024-04-03 11:22:43 +03:00
Bevin Hansson
7edddee2aa
[ExpandLargeFpConvert] Scalarize vector types. (#86954)
expand-large-fp-convert cannot handle vector types.
If overly large vector element types survive into
isel, they will likely be scalarized there, but since
isel cannot handle scalar integer types of that size,
it will assert.

Handle vector types in expand-large-fp-convert by
scalarizing them and then expanding the scalar type
operation. For large vectors, this results in a
*massive* code expansion, but it's better than
asserting.
2024-04-03 08:45:59 +02:00
Ryotaro KASUGA
ea4a11926b
Reapply "[CodeGen] Fix register pressure computation in MachinePipeli… (#87312)
…ner (#87030)"

Fix broken test.

This reverts commit b8ead2198f27924f91b90b6c104c1234ccc8972e.
2024-04-03 09:28:09 +09:00
Matt Arsenault
7f2a41b643 MachineScheduler: Simplify usage of TargetInstrInfo 2024-04-02 16:24:47 -04:00
Prabhuk
212b1a84a6
[CallSiteInfo][NFC] CallSiteInfo -> CallSiteInfo.ArgRegPairs (#86842)
CallSiteInfo is originally used only for argument - register pairs. Make
it struct, in which we can store additional data for call sites.

Also, the variables/methods used for CallSiteInfo are named for its
original use case, e.g., CallFwdRegsInfo. Refactor these for the
upcoming
use, e.g. addCallArgsForwardingRegs() -> addCallSiteInfo().

An upcoming patch will add type ids for indirect calls to propogate them
from
middle-end to the back-end. The type ids will be then used to emit the
call
graph section.

Original RFC:
https://lists.llvm.org/pipermail/llvm-dev/2021-June/151044.html
Updated RFC:
https://lists.llvm.org/pipermail/llvm-dev/2021-July/151739.html

Differential Revision: https://reviews.llvm.org/D107109?id=362888

Co-authored-by: Necip Fazil Yildiran <necip@google.com>
2024-04-02 13:05:16 -07:00
Atousa Duprat
4aba595f09
[ADT] Add signed and unsigned mulh to APInt (#84719)
Fixes #84207
2024-04-02 17:07:56 +01:00
Bevin Hansson
cd6434f9ec
[ExpandLargeDivRem] Scalarize vector types. (#86959)
expand-large-divrem cannot handle vector types.
If overly large vector element types survive into
isel, they will likely be scalarized there, but since
isel cannot handle scalar integer types of that size,
it will assert.

Handle vector types in expand-large-divrem by
scalarizing them and then expanding the scalar type
operation. For large vectors, this results in a
*massive* code expansion, but it's better than
asserting.
2024-04-02 16:37:36 +02:00
Il-Capitano
0ef7437780
[SelectionDAG][Statepoint] Fix truncation of gc.statepoint ID argument (#85908)
The ID argument of `gc.statepoint` gets incorrectly truncated to 32 bits
during code generation.
This is fixed by using `uint64_t` instead of `unsigned` for the `ID`
member in `SelectionDAGBuilder::StatepointLoweringInfo`, and a
`patchpoint` test case is extended to check for 64 bit ID generation in
stackmaps.
2024-04-02 09:28:19 -04:00
Sizov Nikita
6654235594
[SelectionDAG] implement computeKnownBits for add AVG* instructions (#86754)
knownBits calculation for **AVGFLOORU** / **AVGFLOORS** / **AVGCEILU** / **AVGCEILS** instructions

Prerequisite for #76644
2024-04-02 10:39:49 +01:00
Thorsten Schütt
8bb9443333
[GlobalIsel] Combine G_EXTRACT_VECTOR_ELT (#85321)
preliminary steps
2024-04-02 09:01:24 +02:00
Gulfem Savrun Yeniceri
b8ead2198f Revert "[CodeGen] Fix register pressure computation in MachinePipeliner (#87030)"
This reverts commit a4dec9d6bc67c4d8fbd4a4f54ffaa0399def9627
because the test failed in the following builder:
https://luci-milo.appspot.com/ui/p/fuchsia/builders/prod/clang-linux-x64/b8751864477467126481/overview
2024-04-01 18:27:41 +00:00
Michael Maitland
da9f06c9b1
[GISEL] G_SPLAT_VECTOR can take a splat that is larger than the vector element (#86974)
This is what SelectionDAG does. We'd like to reuse SelectionDAG
patterns.
2024-04-01 08:46:22 -04:00
Ryotaro KASUGA
a4dec9d6bc
[CodeGen] Fix register pressure computation in MachinePipeliner (#87030)
`RegisterClassInfo::getRegPressureSetLimit` has been changed to return a
smaller value than before so the limit may become negative in later
calculations. As a workaround, change to use
`TargetRegisterInfo::getRegPressureSetLimit`.
Also improve tests.
2024-04-01 17:04:44 +09:00
Vitaly Buka
c4df57da1d [CodeGen] llvm.allow.{runtime,ubsan}.check() in FastISel
Follow up to #86049.
clang-armv8-quick build bot can trigger this branch.
2024-03-31 23:39:33 -07:00
Sameer Sahasrabuddhe
421557974a
[AMDGPU] Use glue for convergence tokens at call-like operations (#86766)
The earlier implementation on AMDGPU used explicit token operands at
SI_CALL and SI_CALL_ISEL. This is now replaced with CONVERGENCECTRL_GLUE
operands, with the following effects:

- The treatment of tokens at call-like operations is now consistent with
the treatment at intrinsics.
- Support for tail calls using implicit tokens at SI_TCRETURN "just
works".
- The extra parameter at call-like instructions is eliminated, thus
restoring those instructions and their handling to the original state.

The new glue node is placed after the existing glue node for the
outgoing call parameters, which seems to not interfere with selection of
the call-like nodes.
2024-04-01 10:51:13 +05:30
Vitaly Buka
20f56e1f8e
[CodeGen] Add default lowering for llvm.allow.{runtime,ubsan}.check() (#86049)
RFC:
https://discourse.llvm.org/t/rfc-add-llvm-experimental-hot-intrinsic-or-llvm-hot/77641
2024-03-31 22:19:33 -07:00
Shilei Tian
3a106e5b2c
[GlobalISel] Fold G_ICMP if possible (#86357)
This patch tries to fold `G_ICMP` if possible.
2024-03-29 15:59:50 -04:00
Shilei Tian
360f7f5674
[GlobalISel] Call setInstrAndDebugLoc before tryCombineAll (#86993)
This can remove all unnecessary redundant calls in each combiner.
2024-03-29 15:27:28 -04:00
Kevin P. Neal
fe893c93b7
[FPEnv][AtomicExpand] Correct strictfp attribute handling in AtomicExpandPass (#87082)
The AtomicExpand pass was lowering function calls with the strictfp
attribute to sequences that included function calls incorrectly lacking
the attribute. This patch corrects that.

The pass now also emits the correct constrained fp call instead of
normal FP instructions when in a function with the strictfp attribute.

Test changes verified with D146845.
2024-03-29 14:54:51 -04:00
Shilei Tian
661bb9daae
[GlobalISel] Handle div-by-pow2 (#83155)
This patch adds similar handling of div-by-pow2 as in `SelectionDAG`.
2024-03-29 12:41:47 -04:00
Thorsten Schütt
84299df301
[GlobalIsel] add trunc flags (#87045)
https://github.com/llvm/llvm-project/pull/85592
2024-03-29 13:38:08 +01:00
Wang Pengcheng
610b9e23c5
[SDAG] Use shifts if ISD::MUL is illegal when lowering ISD::CTPOP (#86505)
We can avoid libcalls.

Fixes #86205
2024-03-29 15:38:39 +08:00
Sudharsan Veeravalli
e005a09df5
[RISCV][TypePromotion] Dont generate truncs if PromotedType is greater than Source Type (#86941)
We currently check if the source and promoted types are not equal before
generating truncate instructions. This does not work for RV64 where the
promoted type is i64 and this lead to a crash due to the generation of
truncate instructions from i32 to i64.

Fixes #86400
2024-03-28 21:22:05 -07:00
Craig Topper
23d45e55ed
[MCP] Remove dead copies from basic blocks with successors. (#86973)
Previously we wouldn't remove dead copies from basic blocks with
successors. The comment said we didn't want to trust the live-in lists.
The comment is very old so I'm not sure if that's still a concern today.

This patch checks the live-in lists and removes copies from
MaybeDeadCopies if they are referenced by any live-ins in any
successors. We only do this if the tracksLiveness property is set. If
that property is not set, we retain the old behavior.
2024-03-28 14:43:49 -07:00
Craig Topper
f90813543b
[MCP] Use MachineInstr::all_defs instead of MachineInstr::defs in hasOverlappingMultipleDef. (#86889)
defs does not return the defs for inline assembly. We need to use
all_defs to find them.

Fixes #86880.
2024-03-28 08:37:19 -07:00
Jonas Paulsson
94b5c118b3
[ISel] Move handling of atomic loads from SystemZ to DAGCombiner (NFC). (#86484)
The folding of sign/zero extensions into an atomic load by specifying an
extension type is not target specific, and therefore belongs in the
DAGCombiner rather than in the SystemZ backend.

- Handle atomic loads similarly to regular loads by adding
AtomicLoadExtActions with set/get methods.
- Move SystemZ extendAtomicLoad() to DagCombiner.cpp.
2024-03-28 16:14:35 +01:00
Luke Lau
856e815ca1
[DAGCombiner] Set disjoint flag in add->or and xor->or combines (#86925)
We check DAG.haveNoCommonBitsSet so the operands will be known to be
disjoint.

I couldn't think of a codegen test case since most targets aren't
checking hasDisjoint yet, apart from RISCV in the or_is_add pattern, but
it also falls back to computeKnownBits.
2024-03-28 18:08:59 +08:00
Shilei Tian
a8b90c047d
[GlobalISel] Update MachineIRBuilder::buildAtomicRMW interface (#86851) 2024-03-27 17:41:30 -04:00
Craig Topper
acab142751 [LegalizeDAG] Freeze index when converting insert_elt/insert_subvector to load/store on stack.
We try clamp the index to be within the bounds of the stack object
we create, but if we don't freeze it, poison can propagate into the
clamp code. This can cause the access to leave the bounds of the
stack object.

We have other instances of this issue in type legalization and extract_elt/subvector,
but posting this patch first for direction check.

Fixes #86717
2024-03-27 13:01:23 -07:00
Craig Topper
baf66ec061
[Target][RISCV] Add HwMode support to subregister index size/offset. (#86368)
This is needed to provide proper size and offset for the GPRPair subreg
indices on RISC-V. The size of a GPR already uses HwMode. Previously we
said the subreg indices have unknown size and offset, but this stops
DwarfExpression::addMachineReg from being able to find the registers
that make up the pair.

I believe this fixes https://github.com/llvm/llvm-project/issues/85864
but need to verify.
2024-03-27 12:19:28 -07:00
Craig Topper
1c965801c4
[LegalizeDAG] Merge PerformInsertVectorEltInMemory into ExpandInsertToVectorThroughStack. NFC (#86755)
These functions are very similar. We can share them like we do for
EXTRACT_VECTOR_ELT and EXTRACT_SUBVECTOR.
2024-03-27 09:39:35 -07:00
Simon Pilgrim
78f0871bee Revert rG58de1e2c5eee548a9b365e3b1554d87317072ad9 "Fix stack layout for frames larger than 2gb (#84114)"
This is failing on some EXPENSIVE_CHECKS buildbots
2024-03-27 16:16:15 +00:00
Wesley Wiser
58de1e2c5e
Fix stack layout for frames larger than 2gb (#84114)
For very large stack frames, the offset from the stack pointer to a local can be more than 2^31 which overflows various `int` offsets in the frame lowering code.

This patch updates the frame lowering code to calculate the offsets as 64-bit values and resolves the overflows, resulting in the correct codegen for very large frames.

Fixes #48911
2024-03-27 15:05:58 +00:00
Justin Cady
26464f2662
[FreeBSD] Mark __stack_chk_guard dso_local except for PPC64 (#86665)
Adjust logic of 1cb9f37a17ab to match freebsd/freebsd-src@9a4d48a645.

D113443 is the original attempt to bring this FreeBSD patch to
llvm-project,
but it never landed. This change is required to build FreeBSD kernel
modules
with -fstack-protector using a standard LLVM toolchain. The FreeBSD
kernel
loader does not handle R_X86_64_REX_GOTPCRELX relocations.

Fixes #50932.
2024-03-27 09:03:46 -04:00
Simon Pilgrim
9247f3185c [DAG] foldAddSubOfSignBit - reuse existing SDLoc instead of regenerating it. NFC. 2024-03-27 12:22:31 +00:00
Simon Pilgrim
51388fbab1 [DAG] visitSub - reuse existing SDLoc instead of regenerating it. NFC. 2024-03-27 12:22:30 +00:00
Michael Maitland
d345599c28 [GISEL][NFC] Use getElementCount instead of getNumElements in more places
These cases in particular are  done as a precommit to support
legalization, regbank selection, and instruction selection for extends,
splat vectors, and integer compares in #85938.
2024-03-26 17:41:46 -07:00
Michael Maitland
54a9f0e441
[RISCV][GISEL] Legalize, regbankselect, and instruction-select G_VSCALE (#85967)
G_VSCALE should be lowered using VLENB. If the type is not sXLen it
should be lowered using a G_VSCALE on the narrow type and a G_MUL.
regbank select and instruction select are straightforward so we really
only need to add tests to show it works.
2024-03-26 20:17:22 -04:00
Craig Topper
09155ac290 [LegalizeDAG] Remove unneeded temporary SDValues from PerformInsertVectorEltInMemory. NFC
There were 3 temporaries that just renamed the 3 well name arguments to the
function to Tmp1-3. Looks like this was done when the code was extracted from
elsewhere into a separate function 15 years ago.
2024-03-26 16:25:24 -07:00
Emil Pedersen
0e5c504d3d
[DebugInfo] [SelectionDAG] Fix handling of duplicate dbg values (#86598)
Before this fix, a duplicate llvm.dbg.value intrinsic referring to an
argument, after an alloca, would be generated with `$noreg`, losing
debug information. Instead, we silently drop the second debug info, so
it doesn't break the first one.

rdar://125375717
2024-03-26 12:09:22 -07:00
Simon Pilgrim
1c9d5c25ae [DAG] foldAddSubBoolOfMaskedVal - reuse existing SDLoc instead of regenerating it. NFC. 2024-03-26 18:33:30 +00:00
Thorsten Schütt
da6cc4a24f
[CodeGen] Add nneg and disjoint flags (#86650)
MachineInstr learned the new flags.
2024-03-26 18:44:34 +01:00
David Green
47f4a07a2f
[GlobalISel] Add Knownbits for G_LOAD/ZEXTLOAD/SEXTLOAD with range metadata (#86431)
Similar to #80829 for GlobalISel.
2024-03-26 13:42:08 +00:00
Il-Capitano
308ed0233a
[Intrinsics] Make patchpoint.i64 generic on its return type (#85911)
Currently patchpoints can only have two result types, `void` and `i64`.
This limits the result to general purpose registers.
This patch makes `patchpoint.i64` an overloadable intrinsic, allowing
result values that can fit in a single register (e.g. integers,
pointers, floats).
2024-03-26 19:08:52 +05:30
Simon Pilgrim
5fc619b5ee [DAG] Update ISD::AVG folds to use hasOperation to allow Custom matching prior to legalization
Fixes issue where AVX1 targets weren't matching 256-bit AVGCEILU cases.
2024-03-26 10:41:07 +00:00