60885 Commits

Author SHA1 Message Date
Philip Reames
db746c65b1
[RISCV] Allow constants in tryFoldSelectIntoOp (#157376)
For simm12 constants this allows the use of e.g. addi/xori, and for
non-simm12 constants seems to still produce better code than the default
lowering.
2025-09-08 13:11:34 -07:00
Farzon Lotfi
1dbade85cb
[DirectX] Add emulation for fp16 types of llvm.is.fpclass (#157505)
fixes #157504

This changes adds the emulation we need for IsNaN, IsNormal, & IsFinite
This change only applies these emulations to the llvm.is.fpclass cases
of fp16.

Since there is no DX intrinsics yet for these cases, applying the
emulation to the necessary intrinsics is left for future implementers of
- https://github.com/llvm/llvm-project/issues/99132
- https://github.com/llvm/llvm-project/issues/156069
- https://github.com/llvm/llvm-project/issues/99131
2025-09-08 13:47:50 -04:00
Andrei Safronov
393bc0cc9f
[Xtensa] Fix lowering FP compare operations. (#156740)
Implement lowering of the SETONE/SETOGT/SETOGE/SETUGT/SETUGE operations.
This fixes f32 "copysign" and "ueq" tests.

This PR fixes
https://github.com/llvm/llvm-project/issues/149666#event-18712363149
2025-09-08 20:42:52 +03:00
Trevor Gross
79d2961626
[WebAssembly] Update the test for half (NFC) (#152832)
Replace the existing `f16` test with the version that is uses for other
architectures (typically as `half.ll`). This still covers the
conversions from the existing test, but also adds checks for most simple
ops.

Additionally, rename `half-precision.ll` to `fp-intrinsics.ll` to keep
the name similar to this test.
2025-09-08 15:43:51 +00:00
guan jian
83af24dd85
[DAG] Generalize fold (not (neg x)) -> (add X, -1) (#154348)
Generalize `fold (not (neg x)) -> (add X, -1)` to `fold (not (sub Y, X)) -> (add X, ~Y)`

---------

Co-authored-by: Yui5427 <785369607@qq.com>
Co-authored-by: Simon Pilgrim <llvm-dev@redking.me.uk>
2025-09-08 15:12:59 +00:00
Simon Pilgrim
ed33690d2b
[X86] PromoteMaskArithmetic - oneuse vector_extend_inreg nodes can be safely promoted on AVX2+ targets (#157425)
Allows us to extend a result back to the largest type after we've handled mask logic using vXi1 result types from different source vector widths (e.g. v8i32 and v8i8)

Fixes #157382
2025-09-08 13:47:08 +00:00
Hari Limaye
e38392b19b
[AArch64] Transform add(x, abs(y)) -> saba(x, y, 0) (#156615)
Add a DAGCombine to perform the following transformations: 
- add(x, abs(y)) -> saba(x, y, 0)
- add(x, zext(abs(y))) -> sabal(x, y, 0)

As well as being a useful generic transformation, this also fixes an
issue where LLVM de-optimises [US]ABA neon ACLE intrinsics into separate
ABD+ADD instructions when one of the operands is a zero vector.
2025-09-08 14:14:24 +01:00
Simon Pilgrim
75fe619b24
[X86] Add test coverage for #157382 (#157411) 2025-09-08 10:22:50 +00:00
Amina Chabane
3b19717fb4
[AArch64] Removed redundant FMOV instruction for truncstores of f64/f32 via bitcast to i64/i32/i8. (#149997)
Previously, storing the low bits of a double, which was bitcast to i64
and truncated to i32 or i16, would emit a redundant FMOV. This patch
introduces new TableGen patterns to avoid the unnecessary FMOV. Tests
added: bitcast_truncstore.ll
2025-09-08 10:35:04 +01:00
David Sherwood
dd0161f602
[AArch64] Improve lowering for scalable masked interleaving stores (#156718)
Similar to #154338, this PR aims to support lowering of certain IR
to SVE's st2 and st4 instructions. The typical IR scenario looks
like:

%mask = .. @llvm.vector.interleave2(<vscale x 16 x i1> %m, <vscale x 16
x i1> %m)
%val = .. @llvm.vector.interleave2(<vscale x 16 x i8> %v1, <vscale x 16
x i8> %v2)
.. @llvm.masked.store.nxv32i8.p0(<vscale x 32 x i8> %val, ..., <vscale x
32 x i1> %mask)

where we're interleaving both the value and the mask being passed
to the wide store. When the mask interleave parts are identical
we can lower this to st2b.

This PR adds a DAG combine for lowering this kind of IR pattern
to st2X and st4X SVE instructions.
2025-09-08 10:34:53 +01:00
Benjamin Maxwell
1bd3cc2668
[AArch64][SME] Support Windows/stack probes in MachineSMEABIPass (#149063)
On Windows or with stack probes on other targets, additional code needs
to be inserted after dynamic stack allocations to validate stack
accesses and/or ensure enough stack space has been allocated.

Rather than handle this case in the MachineSMEABIPass (like we do for
the standard case), we allocate the memory for the lazy save buffer in
SelectionDAG, which allows the existing expansions to emit the correct
code.

Note: This means in these cases, we may allocate a lazy save buffer when
there are no lazy saves present in the function (as we have to allocate
the buffer before the MachineSMEABIPass runs).
2025-09-08 10:17:07 +01:00
David Green
2308d7bd77
[AArch64][GlobalISel] Fix passing <3 x ptr> via the stack. (#157247)
A <3 x ptr> will become a <1 x ptr> after being split, which needs to be
represented as a s64 in a LLT.
2025-09-08 07:57:46 +01:00
Philip Reames
c2c7b7efab [RISCV] Add coverage for the tryFoldSelectIntoOp combine with constants
An upcoming change will relax the requirement that the operands be
non-constant.
2025-09-07 17:01:26 -07:00
David Green
6d6122eaff [AArch64][GlobalISel] Add regbank information to other sha1 intrinsics. 2025-09-07 13:50:25 +01:00
David Green
d4450bb8ec [AArch64][GlobalISel] Move special selection for aarch64_crypto_sha1h.
This appears to just need to select FPR registers.
2025-09-07 13:45:00 +01:00
Giuseppe Cesarano
9892dc1d34
[AArch64] Fix vectorToScalarBitmask BE (#156314)
Closes #156312
2025-09-07 13:33:13 +01:00
paperchalice
667f919214
[SelectionDAG][ARM] Propagate fast math flags in visitBRCOND (#156647)
Factor out from #151275.
2025-09-06 20:44:25 +08:00
David Green
6c0154ff01 [AArch64][GlobalISel] Ensure vector types match in shift instructions
This adds legalizations for shifts with mis-matching shift types, tha can be
created from trunc(shift patterns. This helps reduce the amount of fallbacks.
2025-09-06 12:38:28 +01:00
David Green
23c51f17f9 [AArch64] More trunc(shift) tests. NFC 2025-09-06 12:38:28 +01:00
Fangrui Song
0d28b92506
[AArch64] Don't run loop-idiom-vectorize pass in the O0 pipeline (#156802)
As noted in #156787
2025-09-06 00:12:36 -07:00
Craig Topper
151c6edd45
[RISCV] Check for legal type before calling getSimpleValueType() in matchSplatAsGather. (#157188)
This just reorders existing so we do the legal type check first.

In this particular test case we're also protected by the i1 check that I
also moved earlier.

Fixes #157177.
2025-09-05 22:00:38 -07:00
Craig Topper
77db18a1e1
[TargetInstrInfo][AArch64] Don't assume register came from operand 0 in canCombine (#157210)
We already have the register number from the user operand. Use it
instead of assuming it must be operand 0 of the producing instruction.

Fixes #157118
2025-09-05 22:00:12 -07:00
Luke Lau
5facf08193
[RISCV] Use non-vp select in RISCVGatherScatterLowering. NFCish (#157071)
Similar to #157070 and #157068, let RISCVVLOptimizer do the work.
2025-09-06 11:46:02 +08:00
Philip Reames
1455b3cabd
[RISCV] Reorganize select lowering to pull binop expansion early (#156974)
This is purely stylistic, but I think makes the code easier to follow.

It isn't quite NFC because it undoes the arithmetic lowering added yesterday in #156957 for the select c, simm12, 0 cases for a processor with both conditional move forwarding and zicond.  What the right code is for that combination of features is currently an open question.
2025-09-05 18:13:53 -07:00
Matt Arsenault
884130bf93
AMDGPU: Allow folding multiple uses of some immediates into copies (#154757)
In some cases this will require an avoidable re-defining of
a register, but it works out better most of the time. Also allow
folding 64-bit immediates into subregister extracts, unless it would
break an inline constant.

We could be more aggressive here, but this set of conditions seems
to do a reasonable job without introducing too many regressions.
2025-09-06 08:22:09 +09:00
Matt Arsenault
7a0155b320
AMDGPU: Add agpr variants of flat atomic return instructions (#156990) 2025-09-06 07:41:53 +09:00
Stanislav Mekhanoshin
287a5a9efb
[AMDGPU] whole wave CSR tests on gfx1250. NFC. (#157166) 2025-09-05 13:57:17 -07:00
Stanislav Mekhanoshin
5dfb9649cb
[AMDGPU] Prevent VOPD combining of VGPRs with different MSBs (#157168) 2025-09-05 13:34:53 -07:00
Craig Topper
7506900b88
[RISCV] Add th.ext(u) to isSignExtendingOpW in RISCVOptWInstrs. (#157147)
These instructions extract and range of bits, move them to the LSB
then sign or zero extend the result to fill the upper bits. If
the extracted bits is small enough then the result will have 33
sign bits.
2025-09-05 12:05:28 -07:00
Craig Topper
faab8f76d5
[RISCV] Add th.ext(u) to RISCVDAGToDAGISel::hasAllNBitUsers. (#157133)
These instructions are being used by the scalar efficiency SIG to
determine if we should have a standard instruction. Improving our
support for these instructions may give better data for their analysis.

I'll follow up by adding this to RISCVOptWInstrs too which will remove
the 'w' suffixes added here.
2025-09-05 11:57:26 -07:00
Farzon Lotfi
16661b5d6c
[DirectX] Add isinf f16 emulation for SM6.8 and lower (#156932)
fixes #156068

- We needed to add a new sub arch to the target tripple so we can test
that emulation does not happen when targeting SM6.9
- The HLSL toolchain needed to be updated to handle the conversion of
strings to enums for the new sub arch.
- The emulation is done in DXILIntrinsicExpansion.cpp and needs to be
able to convert both llvm.is.fpclass and lvm.dx.isinf to the proper
emulation
- test updates in TargetParser/TripleTest.cpp, isinf.ll, is_fpclass.ll,
and DXCModeTest.cpp
2025-09-05 14:02:48 -04:00
Craig Topper
85dbe18bee
[RISCV] Add -print-pipeline-passes test as follow up to #156798. NFC (#157011) 2025-09-05 08:07:39 -07:00
Frederik Harwath
7f6098ed98
[X86] Fix expand-fp on optnone functions (#156900)
As observed by @mikaelholmen, PR #130988
"[AMDGPU] Implement IR expansion for frem instruction" introduced a
regression on x86. Its changes led to the pass being skipped on
functions with the optnone attribute. @bjope also noted that a check
concerning the optnone handling is wrong.

This patch fixes both issues which together fixes the regression. During
the review it was observed that, even before PR #130988, the pass would
not run on optnone functions with the new pass manager. This is also
fixed.
2025-09-05 16:22:22 +02:00
Phoebe Wang
94b164c218
[X86][AVX10] Remove EVEX512 and AVX10-256 implementations (#157034)
The 256-bit maximum vector register size control was removed from AVX10
whitepaper, ref: https://cdrdv2.intel.com/v1/dl/getContent/784343

We have warned these options in LLVM21 through #132542. This patch
removes underlying implementations in LLVM22.
2025-09-05 14:08:59 +00:00
Shaoce SUN
eb623e650b
[RISCV][GISel] Lower G_ABDS and G_ABDU (#155888)
Implementation follows the `ISD::ABDS` handling in
`RISCVTargetLowering`.
2025-09-05 21:16:35 +08:00
Simon Pilgrim
98f1e4a57f
[DAG] SelectionDAG::canCreateUndefOrPoison - AVGFLOOR/AVGCEIL don't create undef/poison (#157056)
AVGFLOORS: https://alive2.llvm.org/ce/z/6TdoQ_
AVGFLOORU: https://alive2.llvm.org/ce/z/4pfi4i
AVGCEILS: https://alive2.llvm.org/ce/z/nWu8WM
AVGCEILU: https://alive2.llvm.org/ce/z/CGvWiA

Fixes #147696
2025-09-05 11:11:15 +00:00
Luke Lau
95fabfd9a6
[RISCV][LoopIdiomVectorize] Replace vp.icmp -> regular icmp. NFCish (#157068)
To match the general direction of the loop vectorizer and reduce our
dependency on [trivial VP
intrinsics](https://discourse.llvm.org/t/rfc-remove-codegen-support-for-trivial-vp-intrinsics-in-the-risc-v-backend/87999),
replace the use of vp.icmp with a regular icmp. RISCVVLOptimizer can
take care of propagating VL.
2025-09-05 11:09:07 +00:00
Benjamin Maxwell
c71f03624e
[AArch64][SME] Use mrs %reg, SVCR rather than ABI routine with +sme (#156878)
Calling `__arm_sme_state` does more than we need in most cases and
clobbers both x0 and x1.

This also removes `(set|get)PStateSMRegUsed`, since we can directly
check the virtual register instead.
2025-09-05 10:19:05 +01:00
Florian Hahn
74ec38fad0
[SCEV] Fold (C * A /u C) -> A, if A is a multiple of C and C a pow-of-2. (#156730)
Alive2 Proof: https://alive2.llvm.org/ce/z/JoHJE9

PR: https://github.com/llvm/llvm-project/pull/156730
2025-09-05 08:45:13 +00:00
Rajat Bajpai
316107ffa7
[NVPTX] Add sm_88 and sm_110* arch support (#156857)
This change adds support of sm_88, sm_110, sm_110a, and sm_110f. These
are added with PTX ISA version 9.0.
2025-09-05 14:08:48 +05:30
Simon Pilgrim
6711099e69
[AArch64] canCreateUndefOrPoisonForTargetNode - AArch64ISD::VASHR\VLSHR\VSHL can't create undef/poison (#156445)
We can always fold freeze(VSHIFT(x,c)) -> VSHIFT(freeze(x),c) as VASHR\VLSHR\VSHL should always have an in-range constant shift amount.

Test coverage can be tricky, so I've hijacked some computeKnownBits/ComputeNumSignBits tests to show that value tracking can still analyze the shift node as the FREEZE will have been discarded by the canCreateUndefOrPoison/isGuaranteedNotToBeUndefOrPoison logic in getFreeze().
2025-09-05 08:26:47 +00:00
Cullen Rhodes
5a0d703fac
[AArch64][nfc] Remove llvm/test/CodeGen/AArch64/README (#156881)
It was added (02650ac03632) to document SVE-related warnings in tests,
but these were removed a while ago now (672f67300466).
2025-09-05 09:09:01 +01:00
Sudharsan Veeravalli
e71ec0f7a4
[RISCV] Use QC_E_LI to materialise addresses (#155819)
This is only used in the small code model and when `Xqcili` is enabled,
where addresses would otherwise use `LUI/ADDI`. Other code models need
to use pc-relative addressing. This patch does this for global/block
addresses, constant pools and jumptables.

Overall, this gives a better code size saving as` QC.E.LI` is easier to
relax to `QC.LI/LI` etc than` LUI/ADDI` (especially when the `LUI/ADDI`
might have sharing, or be split apart). `QC.E.LI` has the
`RISCV_QC_E_32` local relocation attached to it.
2025-09-05 12:58:28 +05:30
Stanislav Mekhanoshin
5901d896f4
[AMDGPU] Register amdgpu-lower-vgpr-encoding pass in npm (#156971) 2025-09-05 00:07:47 -07:00
Himadhith
ffbd616210
[NFC][PowerPC] adding the options for register names and VSR to VR (#157007)
NFC patch to add the flags -ppc-asm-full-reg-names --ppc-vsr-nums-as-vr
to the following test files
```
llvm/test/CodeGen/PowerPC/recipest.ll
llvm/test/CodeGen/PowerPC/setcc-logic.ll
llvm/test/CodeGen/PowerPC/vector-popcnt-128-ult-ugt.ll
```

Created this PR based on this discussion:
https://github.com/llvm/llvm-project/pull/151971#issuecomment-3234090675

Co-authored-by: himadhith <himadhith.v@ibm.com>
2025-09-05 10:27:02 +05:30
Brandon Wu
7fb1dc08d2
[RISCV] Support ZVqdot Codegen and C intrinsics (#154915)
spec: https://github.com/riscv/riscv-dot-product/tree/main

Node: we pack 4 int8/uint8 element in rs1 to a uint32.
2025-09-05 11:38:58 +08:00
Matt Arsenault
882575f382
AMDGPU: Fix using wrong type in i32 atomic cmpxchg test (#156988)
Copy paste error from i64 case
2025-09-05 00:56:11 +00:00
Luke Lau
dc2ed00432
[RISCV] Handle non uimm5 VL constants in isVLKnownLE (#156639)
If a VL operand is > 31 then it will be materialized into an ADDI $x0,
imm. We can reason about it by peeking at the virtual register
definition which allows RISCVVectorPeephole and RISCVVLOptimizer to
catch more cases.

There's a separate issue with RISCVVLOptimizer where the materialized
immediate may not always dominate the instruction we want to reduce the
VL of, but this is left to another patch.
2025-09-05 00:25:48 +00:00
Stanislav Mekhanoshin
a5f6db4b0d
[AMDGPU] Print high vgpr operand comments from objdump (#156966)
This followed the agreed convention: every basic block shall
start with all MSBs zero. Codegen does the same lowering.
2025-09-04 16:56:49 -07:00
Matt Arsenault
a7c2ce6009
AMDGPU: Add more tests for flat/global atomicrmw with agprs (#156874)
Add comprehensive tests for global atomics with return in
agpr / AV usage contexts.
2025-09-05 08:38:58 +09:00