422 Commits

Author SHA1 Message Date
Fabian Ritter
96775e9229
[GISel] Handle Flags in G_PTR_ADD Combines (#152495)
So far, GlobalISel's G_PTR_ADD combines have ignored MIFlags like nuw, nusw,
and inbounds. That was in many cases unnecessarily conservative and in others
unsound, since reassociations re-used the existing G_PTR_ADD instructions
without invalidating their flags. This patch aims to improve that.

I've checked the transforms in this PR with Alive2 on corresponding middle-end
IR constructs.

A longer-term goal would be to encapsulate the logic that determines which
GEP/ISD::PTRADD/G_PTR_ADD flags can be preserved in which case, since this
occurs in similar forms in the middle end, the SelectionDAG combines, and the
GlobalISel combines here.

For SWDEV-516125.
2025-08-11 10:34:45 +02:00
paperchalice
ce86ff105b
[GlobalISel] Remove UnsafeFPMath references (#146319)
This is the GlobalISel part to remove `UnsafeFPMath` flag in CodeGen
pipeline.
2025-07-29 12:11:52 +08:00
jyli0116
fc5c5a934d
[GlobalISel] Allow expansion of srem by constant in prelegalizer (#148845)
This patch allows srem by a constant to be expanded more efficiently to
avoid the need for expensive sdiv instructions. This is the last part of
the patches which fixes #118090
2025-07-17 14:43:58 +01:00
jyli0116
806028add1
[GlobaISel] Allow expanding of sdiv -> mul by constant (#146504)
Allows expand of sdiv->mul by constant combine for the general case.
Previously this was only occurring in the exact case. This is part of
the resolution to issue #118090
2025-07-14 15:01:12 +01:00
jyli0116
9c0743fbc5
[GlobalISel] Allow expansion of urem by constant in prelegalizer (#145914)
This patch allows urem by a constant to be expanded more efficiently to
avoid the need for expensive udiv instructions. This is part of the
resolution to issue #118090
2025-07-02 13:46:36 +01:00
Daniel Man
045b827367
[GlobalISel] Use-Vector-Truncate Opt Needs Elt Type Check (#146003)
In the pre-legalizer combiner, there exists a bug with UseVectorTruncate
match-apply optimization. When the destinations' types do not match the
vector element type of the G_UNMERGE_VALUES instruction, the resulting
collapsed truncate does not preserve original functional behavior. This
commit introduces a simple type check to ensure that the destination
types match the vector element type.
2025-06-27 16:41:22 +09:00
Alan Li
ada2fbfe36
[GISel] Fix ShuffleVector assert (#139769)
Fixes issue: https://github.com/llvm/llvm-project/issues/139752

When G_SHUFFLE_VECTOR has only 1 element then it is possible the vector
is decayed into a scalar.
2025-05-20 21:25:31 -04:00
David Green
b9e32749d2
[GlobalISel] Clear nsw flags when converting sub to add. (#137288)
As shown in https://alive2.llvm.org/ce/z/PVwcTL we need to clear the nsw
flags too when converting a sub to a add if the constant is INT_MIN.

Fixes #137254
2025-04-26 11:00:53 +01:00
Kazu Hirata
58774f1b1f
[CodeGen] Construct SmallVector with iterator ranges (NFC) (#136258) 2025-04-18 10:26:48 -07:00
Alan Li
2795abb2f8
[GISel][AMDGPU] Expand ShuffleVector (#124527)
This patch dismantles G_SHUFFLE_VECTOR before lowering. The original
lowering would emit extract vector element ops. We found that by using
unmerged values the build vector op combine could find ways to fold.

Only enabled on AMDGPU.

This resolves #123631
2025-04-09 17:51:24 -07:00
Krisztian Rugasi
382962b4a8 [GlobalISel] Fix dangling reference in CombinerHelper::matchCombineExtractedVectorLoad 2025-04-07 15:04:36 +02:00
Tim Gymnich
1d0005a69a
[GlobalISel][NFC] Rename GISelKnownBits to GISelValueTracking (#133466)
- rename `GISelKnownBits` to `GISelValueTracking` to analyze more than
just `KnownBits` in the future
2025-03-29 11:51:29 +01:00
Pierre van Houtryve
7dcea28bf9
[AMDGPU] Add identity_combines to RegBankCombiner (#131305) 2025-03-17 10:11:28 +01:00
Shilei Tian
70fdd9f0a2
[GlobalISel] Check whether G_CTLZ is legal in matchUMulHToLShr (#126457)
We need to check `G_CTLZ` because the combine uses `G_CTLZ` to get log
base 2,
and it is not always legal for on a target.

Fixes SWDEV-512440.
2025-02-10 00:11:09 -05:00
David Green
5803335539 [GlobalISel] Add brackets around || in assert. NFC 2025-02-02 07:23:17 +00:00
lialan
5d9c717597
[GISel] Fold shifts to constant result. (#123510)
This resolves #123212
2025-01-21 05:10:45 -08:00
Min-Yih Hsu
3cac26f541
[GISel] Combine (neg (min/max x, (neg x))) into (max/min x, (neg x)) (#120998)
This is the GISel version of #120666. Also supports both unsigned and
signed version of min & max.
2025-01-02 16:29:34 -08:00
Vikash Gupta
283806695a
[GlobalIsel] Add combine for select with constants (#121088)
The SelectionDAG Isel supports the both version of combines mentioned
below :
```
select Cond, Pow2, 0 --> (zext Cond)  << log2(Pow2) 
select Cond, 0, Pow2 --> (zext !Cond) << log2(Pow2)
```
The GlobalIsel for now only supports the first one defined in it's
generic combinerHelper.cpp. This patch adds the missing second one.
2025-01-01 11:14:53 +05:30
Paul Bowen-Huggett
ee7ca0ddda
Make CombinerHelper methods const (#119529)
There are a number of backends (specifically AArch64, AMDGPU, Mips, and
RISCV) which contain a “TODO: make CombinerHelper methods const”
comment. This PR does just that and makes all of the CombinerHelper
methods const, removes the TODO comments and makes the associated
instances const. This change makes some sense because the CombinerHelper
class simply modifies the state of _other_ objects to which it holds
pointers or references.

Note that AMDGPU contains an identical comment for an instance of
AMDGPUCombinerHelper (a subclass of CombinerHelper). I deliberately
haven’t modified the methods of that class in order to limit the scope
of the change. I’m happy to do so either now or as a follow-up.
2024-12-20 08:29:18 +07:00
Craig Topper
0d9fc17433
[GISel] Remove unused DataLayout operand from getApproximateEVTForLLT (#119833) 2024-12-13 09:09:20 -08:00
David Green
94a77ebe24 [AArch64][GlobalISel] Guard against no operands in matchHoistLogicOpWithSameOpcodeHands
In case both LeftHandInst and RightHandInst are IMPLICIT_DEF with no input
operands, this patch protects against the post-legalizer-combiner
matchHoistLogicOpWithSameOpcodeHands with no operands. The
prelegalizercombiner-hoist-same-hands.mir test was cleaned up a little in the
process, and has a post-legalizer run line added so that the implicit_def do
not get folded awwy.
2024-12-13 11:02:55 +00:00
abhishek-kaushik22
d20731ce6b
[CGData][GlobalIsel][Legalizer][DAG][MC][AsmParser][X86][AMX] Use std::move to avoid copy (#118068) 2024-12-06 09:46:15 +08:00
Thorsten Schütt
f8d1905a24
[GlobalISel] Combine [S,U]SUBO (#116489)
We import the llvm.ssub.with.overflow.* Intrinsics, but the Legalizer
also builds them while legalizing other opcodes, see narrowScalarAddSub.
2024-11-18 22:39:23 +01:00
Konstantin Schwarz
0f0e2fe97b
[GlobalISel] Turn shuffle a, b, mask -> shuffle undef, b, mask iff mask does not reference a (#115377) 2024-11-14 15:13:41 -08:00
Nikita Popov
63fb980d50
[IR] Add helper for comparing KnownBits with IR predicate (NFC) (#115878)
Add `ICmpInst::compare()` overload accepting `KnownBits`, similar to the
existing one accepting `APInt`. This is not directly part of KnownBits
(or APInt) for layering reasons.
2024-11-12 17:41:08 +01:00
David Green
b242ae32f5 [AArch64][GlobalISel] Protect against undef first element in CombineShuffleConcat.
In case the first element is undef, we need to look through to find a valid
type for the inputs.
2024-11-11 19:37:51 +00:00
Craig Topper
17f3e00911 Recommit "[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)"
The increase in fallbacks that was previously reported were not caused
by this change.

Original description:

This matches InstCombine and DAGCombine.

RISC-V only has an ADDI instruction so without this we need additional
patterns to do the conversion.

Some of the AMDGPU tests look like possible regressions. Maybe some
patterns from isel aren't imported.
2024-11-08 10:21:46 -08:00
Konstantin Schwarz
cbfe87c253
[GlobalISel] Remove references to rhs of shufflevector if rhs is undef (#115076) 2024-11-06 16:36:13 -08:00
Craig Topper
cff2199e0f Revert "[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)"
This reverts commit 999dfb2067eb75609b735944af876279025ac171.

I received a report that his may have increased fallbacks on AArch64.
2024-11-06 10:45:23 -08:00
David Green
5dc9c39ac1
[GlobalISel] Check the correct register in sextload OneUse check. (#114763)
This fixes a bug that started triggering after #111730, where we could
remove a load with multiple uses. It looks like the match should be
checking the other register in a one-use check.

  %SrcReg = load..
  %DstReg = sign_extend_inreg %SrcReg
2024-11-05 10:18:07 +00:00
Craig Topper
999dfb2067
[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)
This matches InstCombine and DAGCombine.

RISC-V only has an ADDI instruction so without this we need additional
patterns to do the conversion.

Some of the AMDGPU tests look like possible regressions. Maybe some
patterns from isel aren't imported.
2024-11-04 17:20:11 -08:00
Matt Arsenault
db5bcb24c2
GlobalISel: Fix combine duplicating atomic loads (#111730)
The sext_inreg (load) combine was not deleting the old load instruction,
and it would never be deleted if volatile or atomic.
2024-10-31 07:55:12 -07:00
Thorsten Schütt
d8b17f2fb6
[GlobalISel] Combine G_UNMERGE_VALUES with anyext and build vector (#112370)
G_UNMERGE_VALUES (G_ANYEXT (G_BUILD_VECTOR))

ag G_UNMERGE_VALUES llvm/test/CodeGen/AArch64/GlobalISel | grep ANYEXT 
[ANYEXT] is build vector or shuffle vector

Prior art:
https://reviews.llvm.org/D87117
https://reviews.llvm.org/D87166
https://reviews.llvm.org/D87174
https://reviews.llvm.org/D87427

; CHECK-NEXT: [[BUILD_VECTOR2:%[0-9]+]]:_(<8 x s8>) = G_BUILD_VECTOR 
[[C2]](s8), [[C2]](s8), [[C2]](s8), [[C2]](s8), [[DEF1]](s8),
[[DEF1]](s8), [[DEF1]](s8), [[DEF1]](s8)

; CHECK-NEXT: [[ANYEXT1:%[0-9]+]]:_(<8 x s16>) = G_ANYEXT
[[BUILD_VECTOR2]](<8 x s8>)

; CHECK-NEXT: [[UV10:%[0-9]+]]:_(<4 x s16>), [[UV11:%[0-9]+]]:_(<4 x
s16>) = G_UNMERGE_VALUES
              [[ANYEXT1]](<8 x s16>)

Test:
llvm/test/CodeGen/AArch64/GlobalISel/combine-unmerge.mir
2024-10-19 09:41:43 +02:00
Petar Avramovic
14d006c53c
AMDGPU/GlobalISel: Run redundant_and combine in RegBankCombiner (#112353)
Combine is needed to clear redundant ANDs with 1 that will be
created by reg-bank-select to clean-up high bits in register.
Fix replaceRegWith from CombinerHelper:
If copy had to be inserted, first create copy then delete MI.
If MI is deleted first insert point is not valid.
2024-10-16 09:43:16 +02:00
Craig Topper
20e37f03c6
[GISel] Don't preserve NSW flag when converting G_MUL of INT_MIN to G_SHL. (#111230)
mul and shl have different meanings for the nsw flag. We need to drop it
when converting a multiply by the minimum negative value.
2024-10-05 10:27:09 -07:00
chuongg3
b0dc7b5b86
[AArch64][GlobalISel] Prefer to use Vector Truncate (#105692)
Tries to combine scalarised truncates into vector truncate operations

EXAMPLE:
`%a(i32), %b(i32) = G_UNMERGE %src(<2 x i32>)`
`%T_a(i16) = G_TRUNC %a(i32)`
`%T_b(i16) = G_TRUNC %b(i32)`
`%Imp(i16) = G_IMPLICIT_DEF(i16)`
`%dst(v8i16) = G_MERGE_VALUES %T_a(i16), %T_b(i16), %Imp(i16),
%Imp(i16)`

===>
`%Imp(<2 x i32>) = G_IMPLICIT_DEF(<2 x i32>)`
`%Mid(<4 x s16>) = G_CONCAT_VECTORS %src(<2 x i32>), %Imp(<2 x i32>)`
`%dst(<4 x s16>) = G_TRUNC %Mid(<4 x s16>)`
2024-09-23 13:52:37 +01:00
Craig Topper
4ca817d051
[GlobalISel] Add bail outs for scalable vectors to some combines. (#106496)
These combines call getNumElements() which isn't valid for scalable
vectors.
2024-08-29 14:02:53 -07:00
Thorsten Schütt
710a552ded
[GlobalIsel] Revisit ext of ext. (#102769)
Credits:
https://reviews.llvm.org/D86516

combine-ext.mir

Notable semantic changes:
InstCombine does not mix zero and sign extend,
  see CastInst::isEliminableCastPair.
New version has legality checks.
Folds sext/zext of anyext -> sext/zext
Support for nneg

Future work:
nneg zext of sext -> sext
sext of nneg zext -> sext
2024-08-16 15:38:56 +02:00
Thorsten Schütt
6b77531123
[GlobalIsel] Combine G_ADD and G_SUB with constants (#97771) 2024-08-09 20:14:32 +02:00
Thorsten Schütt
db8c84fc7a
[GlobalIsel] Push cast through select. (#100539) 2024-07-25 19:21:28 +02:00
Thorsten Schütt
bfcfb0fd2d
[GlobalIsel] Modernize truncate of ext. (#100338)
Credits:
https://github.com/llvm/llvm-project/pull/90964
https://reviews.llvm.org/D87050

combine-trunc.mir

Functional changes intended.
2024-07-24 22:13:52 +02:00
Thorsten Schütt
2ca300f914
[GlobalIsel][NFC] Move cast code (#100196)
Preparation for more cast combines
2024-07-24 06:33:11 +02:00
Craig Topper
d6ad4c2834
[GlobalISel] Reorder code in CombinerHelper::buildUDivUsingMul. NFC (#99565)
Group the code for handling Exact udiv together above the code for
non-Exact. Move the computeKnownBits call to after the Exact udiv
handling.
2024-07-22 15:27:55 -07:00
AtariDreams
a51f343b43
[CodeGen] Emit more efficient magic numbers for exact udivs (#87161)
Have simpler lowering for exact udivs in both SelectionDAG and
GlobalISel.

The algorithm is the same between unsigned exact divs and signed divs
save for arithmetic vs logical shift for even divisors, according to
Hacker's Delight, 2nd Edition, page 242.
2024-07-17 12:19:02 -07:00
David Green
3b73cb3bf1 [AArch64][GlobalISel] Create copy rather than single-element concat
The verifier does not accept single-element G_CONCAT_VECTORS, so if there is a
single Op generate a COPY instead.
2024-07-03 10:22:15 +01:00
Thorsten Schütt
c5b67dde98
[GlobalIsel][NFC] Modernize UBFX combine (#97513)
Credits: https://reviews.llvm.org/D99283
2024-07-03 09:19:40 +02:00
Kazu Hirata
58fd3bea6d
[CodeGen] Use range-based for loops (NFC) (#97467) 2024-07-02 16:36:13 -07:00
isuckatcs
937d79bc9d
[GlobalISel][AArch64][AMDGPU] Expand FPOWI into series of multiplication (#95217)
SelectionDAG already converts FPOWI into a series of optimized multiplications, 
this patch introduces the same optimization into GlobalISel.
2024-06-28 09:57:50 +02:00
Kazu Hirata
559ea40d9a
[CodeGen] Use range-based for loops (NFC) (#96855) 2024-06-27 11:00:27 -07:00
Nikita Popov
f2f18459d4 Revert "Intrinsic: introduce minimumnum and maximumnum (#93841)"
As far as I can tell, this pull request was not approved, and
did not go through an RFC on discourse.

This reverts commit 89881480030f48f83af668175b70a9798edca2fb.
This reverts commit 225d8fc8eb24fb797154c1ef6dcbe5ba033142da.
2024-06-21 08:34:04 +02:00