406 Commits

Author SHA1 Message Date
Min-Yih Hsu
3cac26f541
[GISel] Combine (neg (min/max x, (neg x))) into (max/min x, (neg x)) (#120998)
This is the GISel version of #120666. Also supports both unsigned and
signed version of min & max.
2025-01-02 16:29:34 -08:00
Vikash Gupta
283806695a
[GlobalIsel] Add combine for select with constants (#121088)
The SelectionDAG Isel supports the both version of combines mentioned
below :
```
select Cond, Pow2, 0 --> (zext Cond)  << log2(Pow2) 
select Cond, 0, Pow2 --> (zext !Cond) << log2(Pow2)
```
The GlobalIsel for now only supports the first one defined in it's
generic combinerHelper.cpp. This patch adds the missing second one.
2025-01-01 11:14:53 +05:30
Paul Bowen-Huggett
ee7ca0ddda
Make CombinerHelper methods const (#119529)
There are a number of backends (specifically AArch64, AMDGPU, Mips, and
RISCV) which contain a “TODO: make CombinerHelper methods const”
comment. This PR does just that and makes all of the CombinerHelper
methods const, removes the TODO comments and makes the associated
instances const. This change makes some sense because the CombinerHelper
class simply modifies the state of _other_ objects to which it holds
pointers or references.

Note that AMDGPU contains an identical comment for an instance of
AMDGPUCombinerHelper (a subclass of CombinerHelper). I deliberately
haven’t modified the methods of that class in order to limit the scope
of the change. I’m happy to do so either now or as a follow-up.
2024-12-20 08:29:18 +07:00
Craig Topper
0d9fc17433
[GISel] Remove unused DataLayout operand from getApproximateEVTForLLT (#119833) 2024-12-13 09:09:20 -08:00
David Green
94a77ebe24 [AArch64][GlobalISel] Guard against no operands in matchHoistLogicOpWithSameOpcodeHands
In case both LeftHandInst and RightHandInst are IMPLICIT_DEF with no input
operands, this patch protects against the post-legalizer-combiner
matchHoistLogicOpWithSameOpcodeHands with no operands. The
prelegalizercombiner-hoist-same-hands.mir test was cleaned up a little in the
process, and has a post-legalizer run line added so that the implicit_def do
not get folded awwy.
2024-12-13 11:02:55 +00:00
abhishek-kaushik22
d20731ce6b
[CGData][GlobalIsel][Legalizer][DAG][MC][AsmParser][X86][AMX] Use std::move to avoid copy (#118068) 2024-12-06 09:46:15 +08:00
Thorsten Schütt
f8d1905a24
[GlobalISel] Combine [S,U]SUBO (#116489)
We import the llvm.ssub.with.overflow.* Intrinsics, but the Legalizer
also builds them while legalizing other opcodes, see narrowScalarAddSub.
2024-11-18 22:39:23 +01:00
Konstantin Schwarz
0f0e2fe97b
[GlobalISel] Turn shuffle a, b, mask -> shuffle undef, b, mask iff mask does not reference a (#115377) 2024-11-14 15:13:41 -08:00
Nikita Popov
63fb980d50
[IR] Add helper for comparing KnownBits with IR predicate (NFC) (#115878)
Add `ICmpInst::compare()` overload accepting `KnownBits`, similar to the
existing one accepting `APInt`. This is not directly part of KnownBits
(or APInt) for layering reasons.
2024-11-12 17:41:08 +01:00
David Green
b242ae32f5 [AArch64][GlobalISel] Protect against undef first element in CombineShuffleConcat.
In case the first element is undef, we need to look through to find a valid
type for the inputs.
2024-11-11 19:37:51 +00:00
Craig Topper
17f3e00911 Recommit "[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)"
The increase in fallbacks that was previously reported were not caused
by this change.

Original description:

This matches InstCombine and DAGCombine.

RISC-V only has an ADDI instruction so without this we need additional
patterns to do the conversion.

Some of the AMDGPU tests look like possible regressions. Maybe some
patterns from isel aren't imported.
2024-11-08 10:21:46 -08:00
Konstantin Schwarz
cbfe87c253
[GlobalISel] Remove references to rhs of shufflevector if rhs is undef (#115076) 2024-11-06 16:36:13 -08:00
Craig Topper
cff2199e0f Revert "[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)"
This reverts commit 999dfb2067eb75609b735944af876279025ac171.

I received a report that his may have increased fallbacks on AArch64.
2024-11-06 10:45:23 -08:00
David Green
5dc9c39ac1
[GlobalISel] Check the correct register in sextload OneUse check. (#114763)
This fixes a bug that started triggering after #111730, where we could
remove a load with multiple uses. It looks like the match should be
checking the other register in a one-use check.

  %SrcReg = load..
  %DstReg = sign_extend_inreg %SrcReg
2024-11-05 10:18:07 +00:00
Craig Topper
999dfb2067
[GISel][AArch64][AMDGPU][RISCV] Canonicalize (sub X, C) -> (add X, -C) (#114309)
This matches InstCombine and DAGCombine.

RISC-V only has an ADDI instruction so without this we need additional
patterns to do the conversion.

Some of the AMDGPU tests look like possible regressions. Maybe some
patterns from isel aren't imported.
2024-11-04 17:20:11 -08:00
Matt Arsenault
db5bcb24c2
GlobalISel: Fix combine duplicating atomic loads (#111730)
The sext_inreg (load) combine was not deleting the old load instruction,
and it would never be deleted if volatile or atomic.
2024-10-31 07:55:12 -07:00
Thorsten Schütt
d8b17f2fb6
[GlobalISel] Combine G_UNMERGE_VALUES with anyext and build vector (#112370)
G_UNMERGE_VALUES (G_ANYEXT (G_BUILD_VECTOR))

ag G_UNMERGE_VALUES llvm/test/CodeGen/AArch64/GlobalISel | grep ANYEXT 
[ANYEXT] is build vector or shuffle vector

Prior art:
https://reviews.llvm.org/D87117
https://reviews.llvm.org/D87166
https://reviews.llvm.org/D87174
https://reviews.llvm.org/D87427

; CHECK-NEXT: [[BUILD_VECTOR2:%[0-9]+]]:_(<8 x s8>) = G_BUILD_VECTOR 
[[C2]](s8), [[C2]](s8), [[C2]](s8), [[C2]](s8), [[DEF1]](s8),
[[DEF1]](s8), [[DEF1]](s8), [[DEF1]](s8)

; CHECK-NEXT: [[ANYEXT1:%[0-9]+]]:_(<8 x s16>) = G_ANYEXT
[[BUILD_VECTOR2]](<8 x s8>)

; CHECK-NEXT: [[UV10:%[0-9]+]]:_(<4 x s16>), [[UV11:%[0-9]+]]:_(<4 x
s16>) = G_UNMERGE_VALUES
              [[ANYEXT1]](<8 x s16>)

Test:
llvm/test/CodeGen/AArch64/GlobalISel/combine-unmerge.mir
2024-10-19 09:41:43 +02:00
Petar Avramovic
14d006c53c
AMDGPU/GlobalISel: Run redundant_and combine in RegBankCombiner (#112353)
Combine is needed to clear redundant ANDs with 1 that will be
created by reg-bank-select to clean-up high bits in register.
Fix replaceRegWith from CombinerHelper:
If copy had to be inserted, first create copy then delete MI.
If MI is deleted first insert point is not valid.
2024-10-16 09:43:16 +02:00
Craig Topper
20e37f03c6
[GISel] Don't preserve NSW flag when converting G_MUL of INT_MIN to G_SHL. (#111230)
mul and shl have different meanings for the nsw flag. We need to drop it
when converting a multiply by the minimum negative value.
2024-10-05 10:27:09 -07:00
chuongg3
b0dc7b5b86
[AArch64][GlobalISel] Prefer to use Vector Truncate (#105692)
Tries to combine scalarised truncates into vector truncate operations

EXAMPLE:
`%a(i32), %b(i32) = G_UNMERGE %src(<2 x i32>)`
`%T_a(i16) = G_TRUNC %a(i32)`
`%T_b(i16) = G_TRUNC %b(i32)`
`%Imp(i16) = G_IMPLICIT_DEF(i16)`
`%dst(v8i16) = G_MERGE_VALUES %T_a(i16), %T_b(i16), %Imp(i16),
%Imp(i16)`

===>
`%Imp(<2 x i32>) = G_IMPLICIT_DEF(<2 x i32>)`
`%Mid(<4 x s16>) = G_CONCAT_VECTORS %src(<2 x i32>), %Imp(<2 x i32>)`
`%dst(<4 x s16>) = G_TRUNC %Mid(<4 x s16>)`
2024-09-23 13:52:37 +01:00
Craig Topper
4ca817d051
[GlobalISel] Add bail outs for scalable vectors to some combines. (#106496)
These combines call getNumElements() which isn't valid for scalable
vectors.
2024-08-29 14:02:53 -07:00
Thorsten Schütt
710a552ded
[GlobalIsel] Revisit ext of ext. (#102769)
Credits:
https://reviews.llvm.org/D86516

combine-ext.mir

Notable semantic changes:
InstCombine does not mix zero and sign extend,
  see CastInst::isEliminableCastPair.
New version has legality checks.
Folds sext/zext of anyext -> sext/zext
Support for nneg

Future work:
nneg zext of sext -> sext
sext of nneg zext -> sext
2024-08-16 15:38:56 +02:00
Thorsten Schütt
6b77531123
[GlobalIsel] Combine G_ADD and G_SUB with constants (#97771) 2024-08-09 20:14:32 +02:00
Thorsten Schütt
db8c84fc7a
[GlobalIsel] Push cast through select. (#100539) 2024-07-25 19:21:28 +02:00
Thorsten Schütt
bfcfb0fd2d
[GlobalIsel] Modernize truncate of ext. (#100338)
Credits:
https://github.com/llvm/llvm-project/pull/90964
https://reviews.llvm.org/D87050

combine-trunc.mir

Functional changes intended.
2024-07-24 22:13:52 +02:00
Thorsten Schütt
2ca300f914
[GlobalIsel][NFC] Move cast code (#100196)
Preparation for more cast combines
2024-07-24 06:33:11 +02:00
Craig Topper
d6ad4c2834
[GlobalISel] Reorder code in CombinerHelper::buildUDivUsingMul. NFC (#99565)
Group the code for handling Exact udiv together above the code for
non-Exact. Move the computeKnownBits call to after the Exact udiv
handling.
2024-07-22 15:27:55 -07:00
AtariDreams
a51f343b43
[CodeGen] Emit more efficient magic numbers for exact udivs (#87161)
Have simpler lowering for exact udivs in both SelectionDAG and
GlobalISel.

The algorithm is the same between unsigned exact divs and signed divs
save for arithmetic vs logical shift for even divisors, according to
Hacker's Delight, 2nd Edition, page 242.
2024-07-17 12:19:02 -07:00
David Green
3b73cb3bf1 [AArch64][GlobalISel] Create copy rather than single-element concat
The verifier does not accept single-element G_CONCAT_VECTORS, so if there is a
single Op generate a COPY instead.
2024-07-03 10:22:15 +01:00
Thorsten Schütt
c5b67dde98
[GlobalIsel][NFC] Modernize UBFX combine (#97513)
Credits: https://reviews.llvm.org/D99283
2024-07-03 09:19:40 +02:00
Kazu Hirata
58fd3bea6d
[CodeGen] Use range-based for loops (NFC) (#97467) 2024-07-02 16:36:13 -07:00
isuckatcs
937d79bc9d
[GlobalISel][AArch64][AMDGPU] Expand FPOWI into series of multiplication (#95217)
SelectionDAG already converts FPOWI into a series of optimized multiplications, 
this patch introduces the same optimization into GlobalISel.
2024-06-28 09:57:50 +02:00
Kazu Hirata
559ea40d9a
[CodeGen] Use range-based for loops (NFC) (#96855) 2024-06-27 11:00:27 -07:00
Nikita Popov
f2f18459d4 Revert "Intrinsic: introduce minimumnum and maximumnum (#93841)"
As far as I can tell, this pull request was not approved, and
did not go through an RFC on discourse.

This reverts commit 89881480030f48f83af668175b70a9798edca2fb.
This reverts commit 225d8fc8eb24fb797154c1ef6dcbe5ba033142da.
2024-06-21 08:34:04 +02:00
YunQiang Su
8988148003
Intrinsic: introduce minimumnum and maximumnum (#93841)
Currently, on different platform, the behaivor of llvm.minnum is
different if one operand is sNaN:

When we compare sNaN vs NUM:

ARM/AArch64/PowerPC: follow the IEEE754-2008's minNUM: return qNaN.
RISC-V/Hexagon follow the IEEE754-2019's minimumNumber: return NUM. X86:
Returns NUM but not same with IEEE754-2019's minimumNumber as
     +0.0 is not always greater than -0.0.
MIPS/LoongArch/Generic: return NUM.
LIBCALL: returns qNaN.

So, let's introduce llvm.minmumnum/llvm.maximumnum, which always follow
IEEE754-2019's minimumNumber/maximumNumber.

Half-fix: #93033
2024-06-21 11:53:08 +08:00
Christudasan Devadasan
27bebc1161
[GISel] Unify multiple instances of getTypeForLLT (NFC) (#95577)
Multiple static instances of this utility function have been found in
different GlobalISel files.
Unifying them by adding an instance in utils.cpp.
2024-06-15 18:11:32 +05:30
Dhruv Chawla
e12bf36d23
[GISel][CombinerHelper] Combine op(trunc(x), trunc(y)) -> trunc(op(x, y)) (#89023) 2024-06-03 10:42:10 +05:30
Yingwei Zheng
0864501b97
[GISel] Convert zext nneg to sext if it is cheaper (#93856)
This patch converts `zext nneg` to `sext` on RISCV to use free sext.

---------

Co-authored-by: Thorsten Schütt <schuett@gmail.com>
2024-06-01 15:02:43 +08:00
Him188
8bce40b1eb
[AArch64][GISel] Support SVE with 128-bit min-size for G_LOAD and G_STORE (#92130)
This patch adds basic support for scalable vector types in load & store
instructions for AArch64 with GISel.

Only scalable vector types with a 128-bit base size are supported, e.g.
`<vscale x 4 x i32>`, `<vscale x 16 x i8>`.

This patch adapted some ideas from a similar abandoned patch
[https://github.com/llvm/llvm-project/pull/72976](https://github.com/llvm/llvm-project/pull/72976).
2024-05-30 09:10:43 +01:00
Thorsten Schütt
6d90ac1e06
[GlobalIsel] Combine freeze (#93239) 2024-05-29 18:05:33 +02:00
Dhruv Chawla
4c48b3cb5c
[GISel][CombinerHelper] Push freeze through non-poison-producing operands (#90618)
This combine matches the existing fold in InstCombine, i.e.
InstCombinerImpl::pushFreezeToPreventPoisonFromPropagating.

It tries to push freeze through an operand if the operand has only one
maybe-poison operand and all other operands are guaranteed non-poison,
and if the operation itself cannot generate poison (eg. add with nsw can
generate poison, even with non-poison operands).

This is beneficial because it can potentially enable other optimizations
to occur that would otherwise be blocked because of the freeze.
2024-05-23 13:45:52 +05:30
Thorsten Schütt
778826f0b8
[GlobalIsel] Combine select to integer min max more (#92570) 2024-05-18 13:43:10 +02:00
Thorsten Schütt
9bffe79049
[GlobalIsel] Speedup select to integer min/max (#92378)
https://github.com/llvm/llvm-project/issues/92309
2024-05-17 07:32:18 +02:00
Pierre van Houtryve
0bc1ec5cda
[GlobalISel] Reduce KnownBits usage in matcher combines (#92381)
Two icmp/and combines forced computation of KnownBits on all operands
everytime. We can avoid computing KnownBits on the LHS by exploiting a
couple of properties:
- Constants are always on the RHS for those instructions. If we have no
KnownBits on the RHS, we can bail out early and avoid computing LHS
knownbits.
- For icmp uge/ult 0, we don't need to know the KBs of the LHS to infer
the result

This allows to save some KnownBits calls, which are very expensive,
without affecting codegen.
2024-05-16 13:34:22 +02:00
Thorsten Schütt
737e0bcfe3
[GlobalIsel] combine ext of trunc with flags (#87115)
https://github.com/llvm/llvm-project/pull/85592

https://discourse.llvm.org/t/rfc-add-nowrap-flags-to-trunc/77453

https://github.com/llvm/llvm-project/pull/88609
2024-05-08 14:27:02 +02:00
AtariDreams
13188bcd9f
[GlobalISel]: Simplify udiv lowering by determining known zeros (#89678) 2024-04-24 22:14:02 +02:00
Xu Zhang
f6d431f208
[CodeGen] Make the parameter TRI required in some functions. (#85968)
Fixes #82659

There are some functions, such as `findRegisterDefOperandIdx` and  `findRegisterDefOperand`, that have too many default parameters. As a result, we have encountered some issues due to the lack of TRI  parameters, as shown in issue #82411.

Following @RKSimon 's suggestion, this patch refactors 9 functions, including `{reads, kills, defines, modifies}Register`,  `registerDefIsDead`, and `findRegister{UseOperandIdx, UseOperand, DefOperandIdx, DefOperand}`, adjusting the order of the TRI parameter and making it required. In addition, all the places that call these functions have also been updated correctly to ensure no additional impact.

After this, the caller of these functions should explicitly know whether to pass the `TargetRegisterInfo` or just a `nullptr`.
2024-04-24 14:24:14 +01:00
AtariDreams
5fef5e68dc
[GlobalISel] matchSDivByConst should use isNullValue() (#89666)
It has been using isZeroValue(), which is for floats, not integers.
2024-04-22 15:47:18 -07:00
chuongg3
821935bb55
[AArch64][GlobalISel] Combine Shuffles of G_CONCAT_VECTORS (#87489)
Combines G_SHUFFLE_VECTOR whose sources comes from G_CONCAT_VECTORS into
a single G_CONCAT_VECTORS instruction.

a = G_CONCAT_VECTORS x, y, undef, undef
b = G_CONCAT_VECTORS z, undef, undef, undef
c = G_SHUFFLE_VECTORS a, b, <0, 1, 4, undef>
===>
c = G_CONCAT_VECTORS x, y, z, undef`
2024-04-22 10:47:09 +01:00
darkbuck
e2c91091e5
[GlobalISel] Handle more commutable instructions in commute_constant_to_rhs
Reviewers:
rupprecht, aartbik, cyndyishida, Pierre-vh, aemerson, ftynse, hanhanW, banach-space, jayfoad, nicolasvasilache, daniel-grumberg, arsenm, PeimingLiu, JDevlieghere, matthias-springer

Reviewed By: arsenm, Pierre-vh

Pull Request: https://github.com/llvm/llvm-project/pull/87424
2024-04-16 02:36:12 -04:00