79385 Commits

Author SHA1 Message Date
Luke Lau
11c2da8fb7
[RISCV] Narrow indices to e16 for LMUL > 1 when lowering vector_reverse (#104427)
The vector_shuffle lowering already does this to reduce register
pressure, so also do it here.
2024-08-15 23:18:03 +08:00
Lukacma
d7aeea626d
[AArch64] optimise SVE prefetch intrinsics with no active lanes (#103052)
This patch extends https://github.com/llvm/llvm-project/pull/73964 and
optimises away SVE prefetch intrinsics when predicate is zero.
2024-08-15 13:52:35 +01:00
Janek van Oirschot
bfce1aae76
[AMDGPU] MCExpr printing helper with KnownBits support (#95951)
Walks over the MCExpr and uses KnownBits to deduce whether an expression
is known and if so, prints said known value. Should support the most
common MCExpr cases for AMDGPU metadata.
2024-08-15 13:43:13 +01:00
Fraser Cormack
2e9f3f3b84
[AMDGPU][llvm-split] Fix another division by zero (#104421)
Somehow I missed this in #98888. It requires a log file, or the debug
flag to be passed.
2024-08-15 12:54:26 +01:00
zhongyunde 00443407
43ffe2eed0 [AArch64] Fold more load.x into load.i with large offset
The list of load.x is refer to canFoldIntoAddrMode on D152828.
Also support LDRSroX missed in canFoldIntoAddrMode
2024-08-15 18:22:52 +08:00
zhongyunde 00443407
33190490c6 [AArch64] merge index address with large offset into base address
A case for this transformation, https://gcc.godbolt.org/z/nhYcWq1WE
Fold
  mov     w8, #56952
  movk    w8, #15, lsl #16
  ldrb    w0, [x0, x8]
into
  add     x0, x0, 1036288
  ldrb    w0, [x0, 3704]

Only LDRBBroX is supported for the first time.
Fix https://github.com/llvm/llvm-project/issues/71917

Note: This PR is try relanding the commit 32878c2065 with fix crash for PR79756
  this crash is exposes when there is MOVKWi instruction in the head of a block,
but without MOVZWi
2024-08-15 18:22:52 +08:00
David Green
36231a5b55
[AArch64] Add verification for MemOp immediate ranges (#97561)
This adds an implementation of AArch64InstrInfo::verifyInstruction for
AArch64, and adds some basic verification of the range of immediate
ranges of memory operations using the information from getMemOpInfo.

Some extra memory operations have been added to getMemOpInfo, along with
the equivalent opcodes to getLoadStoreImmIdx to ensure we use the
correct index.

Please let us know if this starts reporting verification failures, Thanks.
2024-08-15 11:20:20 +01:00
Justin Bogner
8107810cad
[DirectX] Use a more consistent pass name for DXILTranslateMetadata
This updates the "dxil-metadata-emit" pass flag to be spelled
"dxil-translate-metadata" to better match the pass name.

Pull Request: https://github.com/llvm/llvm-project/pull/104249
2024-08-15 12:16:52 +03:00
Freddy Ye
372842b30f
[X86][MC] Remove CMPCCXADD's CondCode flavor. (#103898)
To align with gas's latest changes.
relate gas patch:
https://sourceware.org/pipermail/binutils/2024-May/134360.html
2024-08-15 14:18:59 +08:00
LiqinWeng
abaa53199e
[RISCV] Implement RISCVTTIImpl::shouldConsiderAddressTypePromotion for RISCV (#102560)
This optimization helps reduce repeated calculations of base addresses
by extracting type extensions when the same base address is accessed
multiple times but its offset is a constant.
2024-08-15 10:37:04 +08:00
Craig Topper
4f7ce107de [RISCV] Don't combine (sext_inreg (fmv_x_anyexth X), i16) with Zhinx.
With Zfh and Zfhmin this combine creates a fmv_x_signexth node so we can
remember that the result is sign extended. This become a fmv.x.h
instruction which sign extends its result.

With Zhinx, fmv_x_signexth becomes a COPY_TO_REGCLASS. In order for
this to guarantee the result is properly sign extended we need all
producers of a GPRF16 register class to guarantee the rest of the
GPR is sign extended. I don't think we've done that. bitcasts from i16
to f16 definitely don't do it.

The safest thing to do is to not do this combine so the sign_extend_inreg
will emit a shift pair. This is also consistent with the code generated
for Zfinx on RV64, we don't assume the upper 32 bits are sign extended.
2024-08-14 15:18:10 -07:00
Joshua Batista
48809fafbc
Remove unused variable, and unneeded extract element instruction (#103489)
This PR removes an unneeded extract element instruction from codegen,
along with the variable that captured that instruction's return value.
2024-08-14 15:11:34 -07:00
Shilei Tian
1ca9fe6db3 Reapply "[Attributor][AMDGPU] Enable AAIndirectCallInfo for AMDAttributor (#100952)"
This reverts commit 36467bfe89f231458eafda3edb916c028f1f0619.
2024-08-14 17:16:47 -04:00
Craig Topper
6cafebef4a
[RISCV][GISel] Remove support for s32 G_VAARG on RV64. (#102533)
Part of making s32 not legal for RV64. Unfortunately, generic
widening/narrowing is not implement for this operation so I had to
remove all tests.

I don't think clang uses G_VAARG on RISC-V so this shouldn't be a big
deal in practice.
2024-08-14 13:33:30 -07:00
Alex MacLean
099bf20c3f
[NVPTX] Add idp2a, idp4a intrinsics (#102763)
Add support for `llvm.nvvm.idp2a` and `llvm.nvvm.idp4a` which correspond
directly to `dp2a` and `dp4a` PTX instructions.
2024-08-14 13:31:16 -07:00
weiguozhi
2c12c1e755
[X86] Check if an invoked function clobbers fp or bp (#103446)
In most cases when an instruction or function call clobbers fp and/or bp
register, we can fix it by save/restore the clobbered register. But we
still can't handle it when an invoked function clobbers fp and/or bp
according to its calling convention. This patch detects this case and
reports error instead of silently generating wrong code.
2024-08-14 12:59:37 -07:00
Matt Arsenault
0f08aa43a6
AMDGPU: Avoid manually reconstructing atomicrmw (#103769)
When introducing the address space predicates, move and mutate
the original instruction, and clone for the shared case.
2024-08-14 22:01:15 +04:00
Matt Arsenault
21cea3f3be
AMDGPU: Stop promoting allocas with addrspacecast users (#104051)
We cannot promote this case unless we know the value is only
observed through flat operations. We cannot analyze this through
a call. PointerMayBeCaptured was an imprecise check for this.
A callee with a nocapture attribute may still cast to private and
observe the address space, so really we need a different notion
of nocapture.

I doubt this was of any use anyway. The promotable cases should
have optimized out addrspacecast to begin earlier.

Fixes #66669
Fixes #104035
2024-08-14 21:53:38 +04:00
gonzalobg
bdf4b9a00c
[NVPTX] Fix typo causing GCC warning (#103045)
Solves typo found by @mikaelholmen here:
https://github.com/llvm/llvm-project/pull/98551#discussion_r1714824883
2024-08-14 10:46:52 -07:00
Craig Topper
715c3033af
[RISCV][GISel] Support G_SEXT_INREG for Zbb. (#102682) 2024-08-14 10:34:49 -07:00
Abhina Sree
a0be7053d7
[SystemZ][z/OS] Continuation of __ptr32 support (#103393)
This is a continuation of the __ptr32 support added here
135fecd444
2024-08-14 13:26:30 -04:00
Simon Pilgrim
ea9df0982f [X86] concat(permv3(x0,m0,y0),permv3(x0,m1,y0)) -> permv3(concat(x0,u),m3,concat(y0,u))
Reference the lowest subvector if higher subvectors match - this often occurs in length changing shuffles.

Fixes #103564
2024-08-14 18:23:08 +01:00
Simon Pilgrim
8aacbfcb09 [X86] combineEXTRACT_SUBVECTOR - treat oneuse extractions from loads as free
Allows further reductions in instruction vector widths
2024-08-14 18:23:07 +01:00
Matt Arsenault
36a0f20ac3
AMDGPU/NewPM: Fill out addPreISelPasses (#102814)
This specific callback should now be at parity with the old
pass manager version. There are still some missing IR passes
before this point.

Also I don't understand the need for the RequiresAnalysisPass at the
end. SelectionDAG should just be using the uncached getResult?
2024-08-14 20:57:00 +04:00
Ivan Kosarev
f0fe6c66cb
[AMDGPU][NFC] Rename isHi() to isHi16Reg() for clarity. (#103888)
And declare it to take an MCRegister.

Also rename related entities and remove a comment for the function that
depending on its purpose is either irrelevant or misleading.
2024-08-14 17:04:15 +01:00
Craig Topper
294ed6a1eb [RISCV] Use if init statement to reduce scope of variable. NFC 2024-08-14 08:37:50 -07:00
Amy Kwan
5e990b0b7f
[PowerPC][GlobalMerge] Reduce TOC usage by merging internal and private global data (#101224)
This patch aims to reduce TOC usage by merging internal and private
global data.

Moreover, we also add the GlobalMerge pass within the PPCTargetMachine
pipeline, which is disabled by default. This transformation can be
enabled by -ppc-global-merge.
2024-08-14 10:14:33 -04:00
Craig Topper
1f995b5534
[M68k] Remove use of APInt::getRawData(). NFC (#103529)
getRawData exposes some internal details of APInt.

The code was iterating over the uint64_t pieces and then iterating
breaking them into 4 uint16_t pieces.

This patch changes the code to extract 16-bit pieces directly from the
APInt without using getRawData.
2024-08-14 07:12:27 -07:00
Matt Arsenault
0edd07770f
AMDGPU: Preserve alignment when custom expanding atomicrmw (#103768) 2024-08-14 17:16:59 +04:00
Simon Pilgrim
503ba6262e [X86] combineEXTRACT_SUBVECTOR - fold extractions from BLENDI nodes. 2024-08-14 13:50:26 +01:00
Simon Pilgrim
618366546d [X86] combineEXTRACT_SUBVECTOR - fold extractions from UNPCK nodes.
Allow subvector extraction as long as at least one operand extraction is free.

Refactor existing cases into a switch statement to allow easier reuse + future expansion.
2024-08-14 13:50:25 +01:00
Vyacheslav Levytskyy
2fc7a72733
[SPIR-V] Add implementation of the non-const G_BUILD_VECTOR and fix emission of the OpGroupBroadcast instruction (#103050)
This PR addresses a TODO in
lib/Target/SPIRV/SPIRVInstructionSelector.cpp by adding implementation
of the non-const G_BUILD_VECTOR, and fix emission of the
OpGroupBroadcast instruction for the case when the `..._group_broadcast`
builtin has more than one `local_id` argument and `OpGroupBroadcast`
requires a newly constructed vector with 2 or 3 components instead of
originally passed series of `local_id` arguments.

This PR may resolve https://github.com/llvm/llvm-project/issues/97310 if
the reason for the reported fail is an incorrectly generated
OpGroupBroadcast instruction that was definitely a case.

Existing test is hardened and a new test is added to cover this special
case of the OpGroupBroadcast instruction emission.
2024-08-14 11:42:48 +02:00
hanbeom
0d074ba197
[DAG] Support saturated truncate (#99418)
A truncate is considered saturated if no additional conversion is required between the target and return values. If the target is saturated when attempting to truncate from a vector, there is an opportunity to optimize it.

Previously, each architecture had its own attempt at optimization, leading to redundant code. This patch implements common logic by introducing three new ISDs:

`ISD::TRUNCATE_SSAT_S`: When the operand is a signed value and  the range of values matches the range of signed values of the  destination type.

`ISD::TRUNCATE_SSAT_U`: When the operand is a signed value and the range of values matches the range of unsigned values of the destination type.

`ISD::TRUNCATE_USAT_U`: When the operand is an unsigned value and the range of values matches the range of unsigned values of the destination type.

These ISDs indicate a saturated truncate.

Fixes https://github.com/llvm/llvm-project/issues/85903
2024-08-14 09:52:36 +01:00
Anton Sidorenko
5ab99bf1a7
[RISCV] Add scheduling model for Syntacore SCR4 and SCR5 (#102909)
Syntacore SCR4 is a microcontroller-class processor core that has much
in common with SCR3, but also supports F and D extensions.
Overview: https://syntacore.com/products/scr4

Syntacore SCR5 is an entry-level Linux-capable 32/64-bit RISC-V
processor core which scheduling model almost match SCR4.
Overview: https://syntacore.com/products/scr5

Co-authored-by: Dmitrii Petrov <dmitrii.petrov@syntacore.com>
Co-authored-by: Anton Afanasyev <anton.afanasyev@syntacore.com>
2024-08-14 11:42:31 +03:00
Kazu Hirata
e8f312de78
[X86] Use a range-based for loop (NFC) (#103519) 2024-08-13 21:56:11 -07:00
Madhur Amilkanthwar
b73771cf0f
[AArch64] Increase scatter overhead on Neoverse-V2 (#101296)
This patch increases scatter overhead on Neoverse-V2 to 13. This
benefits s128 kernel from TSVC_2 test suite.
SPEC 17, RAJAPerf, and Sptter are unaffected by this patch.

This patch boosts s128 kernel's performance from TSVC test suite by about
40% as this enables vectorization. Also, handle minor code refactoring
for gather related part.
2024-08-14 10:12:40 +05:30
Craig Topper
5e231ffe29 [PowerPC] Use APInt::getZExtValue() instead of getRawData(). NFC 2024-08-13 19:05:12 -07:00
Craig Topper
914a846e29
[RISCV][GISel] Don't custom legalize load/store of vector of pointers if ELEN < XLEN. (#101565)
We need to have elements than can hold a pointer sized element.
    
No test because it crashes in LowerLoad or LowerStore now which
needs to be addressed separately.

I also reordered things so all the vector load/store stuff is together.
2024-08-13 15:53:48 -07:00
Joshua Batista
1b2d11de93
Add normalize builtins and normalize HLSL function to DirectX and SPIR-V backend (#102683)
This PR adds the normalize intrinsic and an HLSL function that uses it.
The SPIRV backend is also implemented.

Used https://github.com/llvm/llvm-project/pull/101256 as a reference,
along with https://github.com/llvm/llvm-project/pull/102243
Fixes https://github.com/llvm/llvm-project/issues/99139
2024-08-13 15:15:19 -07:00
Philip Reames
ac6e1fd0c0
[RISCV][TTI] Cost non-power-of-two size changing casts (#101047)
For a cast with src and destination size being unequal, we were costing
the cast as if it were being scalarized, when in fact we can often
promote such cases to a wider legal type.

Note that for casts with equal size (i.e. bitcast, some fp<->i, and
ptrtoint) the generic logic in BasicTTI already assumed promotion. It
just doesn't handle the cast where source and destination are both
promoted to non-equal types.

This is analogous to d3fd28a, but with the same reasoning applied to
casts instead.
2024-08-13 14:58:16 -07:00
Craig Topper
7ec9fa0373 [RISCV][GISel] Group the vector load/store legalizer actions together. 2024-08-13 13:49:19 -07:00
Craig Topper
51bad732dc [SelectionDAG] Replace EVTToAPFloatSemantics with MVT/EVT::getFltSemantics. (#103001) 2024-08-13 11:35:28 -07:00
Matt Arsenault
edded8d7b5
AMDGPU: Stop handling legacy amdgpu-unsafe-fp-atomics attribute (#101699)
This is now autoupgraded to annotate atomicrmw instructions in
old bitcode.
2024-08-13 22:02:25 +04:00
Brox Chen
afd42fb303
[AMDGPU][True16][CodeGen] Support AND/OR/XOR and LDEXP True16 format (#102620)
Support AND/OR/XOR true16 and LDEXP true/fake16 format.

These instructions are previously implemented with fake16 profile.
Fixing the implementation.

Added a RA hint so that when using 16bit register in a 32bit
instruction, try to use the register directly without an extra 16bit
move

---------

Co-authored-by: guochen2 <guochen2@amd.com>
2024-08-13 12:23:39 -04:00
SpencerAbson
fa9f6b5828
[AArch64][NEON] Add famax/famin codegen patterns (#103027)
- min(abs(a), abs(b)) -> famin(a, b), max(abs(a), abs(b))-> famax(a, b)

- Changes to LLVM
    - llvm/lib/Target/AArch64InstrInfo.td
	    - Add pattern for NEON types
     - +llvm/test/CodeGen/AArch64/aarch64-neon-faminmax.ll
	    - Add tests with and without +faminmax flag.
2024-08-13 17:06:02 +01:00
Matt Arsenault
1ae507d109
AMDGPU: Do not create phi user for atomicrmw with no uses (#103061) 2024-08-13 19:24:52 +04:00
Simon Pilgrim
ed7ad0a1a0 [X86] Promote cttz_i32(x) -> cttz_i64((i64)x | (1 << 32)) (#102900) (REAPPLIED)
On 64bit targets we can promote i32 CTTZ nodes to i64 CTTZ_ZERO_UNDEF by setting the 32nd bit.

#57811 also queried about whether we should use BTS instead of MOVABS+OR to avoid a i64 immediate - I'm willing to tweak the DAGToDAG isel peephole for these cases if reviewers think it worthwhile. But most recent CPUs can actually handle MOVABS faster than BTS/C/R.......

Reapplied with missing costmodel changes - the cost tables can probably be improved in a follow up patch.

Fixes #57811
2024-08-13 12:48:27 +01:00
Jacek Caban
d550ada5ab
[CodeGen][ARM64EC] Define hybrid_patchable EXP thunk symbol as a function. (#102898)
This is needed for MSVC link.exe to generate redirection metadata for hybrid patchable thunks.
2024-08-13 13:39:42 +02:00
Simon Pilgrim
346a1c5161
Revert "[X86] Promote cttz_i32(x) -> cttz_i64((i64)x | (1 << 32))" (#103030)
Reverts llvm/llvm-project#102900 - missed a analysis cost model test change
2024-08-13 10:30:22 +01:00
Simon Pilgrim
71d2de8ecd
[X86] Promote cttz_i32(x) -> cttz_i64((i64)x | (1 << 32)) (#102900)
On 64bit targets we can promote i32 CTTZ nodes to i64 CTTZ_ZERO_UNDEF by setting the 32nd bit.

#57811 also queried about whether we should use BTS instead of MOVABS+OR to avoid a i64 immediate - I'm willing to tweak the DAGToDAG isel peephole for these cases if reviewers think it worthwhile. But most recent CPUs can actually handle MOVABS faster than BTS/C/R.......

Fixes #57811
2024-08-13 10:15:49 +01:00