7436 Commits

Author SHA1 Message Date
Pierre van Houtryve
9372e1a7f1
[AMDGPU] Add gfx12 run lines to fence MMRA tests (#94333) 2024-06-04 12:49:31 +02:00
Nikita Popov
deab451e7a
[IR] Remove support for icmp and fcmp constant expressions (#93038)
Remove support for the icmp and fcmp constant expressions.

This is part of:
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179

As usual, many of the updated tests will no longer test what they were
originally intended to -- this is hard to preserve when constant
expressions get removed, and in many cases just impossible as the
existence of a specific kind of constant expression was the cause of the
issue in the first place.
2024-06-04 08:31:03 +02:00
paperchalice
9b0e1c2ca2
[NewPM][CodeGen] Port finalize-isel to new pass manager (#94214)
It should preserve more analysis results, but it happens immediately
after instruction selection.
2024-06-04 09:23:52 +08:00
paperchalice
7652a59407
Reland "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94149)
- Fix build with `EXPENSIVE_CHECKS`
- Remove unused `PassName::ID` to resolve warning
- Mark `~SelectionDAGISel` virtual so AArch64 backend can work properly
2024-06-04 08:10:58 +08:00
Krzysztof Drewniak
e31bfc040a
[AMDGPU] Strengthen preload intrinsics to noundef and nonnull (#92801)
The various preloaded registers (workitem IDs, workgroup IDs, and
various implicit pointers) always have a finite, invariant, well-defined
value throughout a well-defined program.

In cases where the compiler infers or the user declares that some
implicit input will not be used (ex. via amdgcn-no-workitem-id-y), the
behavior of the entire program is undefined, since that misdeclaration
can cause arbitrary other preloaded-register intrinsics to access the
wrong register. This case is not expected to arise in practice, but
could occur when the no implicit argument attributes were not cleared
correctly in the presence of external functions, indrect calls, or other
means of executing un-analyzable code. Failure to detect that case would
be a bug in the attributor.

This commit updates the documentation to reflect this long-standing
reality.

Then, on the basis that all implicit arguments are defined in all
correct programs, the intrinsics that return those values are
annototated with `noundef``. Some implicit pointer arguments gain a
`nonnull`, but the kernel argument segment pointer or implicit argument
pointers don't necessarily have this property.

This will prevent spurious calls to `freeze` in front-end optimizations
that destroy user-provided ranges on built-in IDs.

(While I'm here, this commit adds a test for `noundef` on kernel
arguments which is currently unimplemented)
2024-06-03 16:37:08 -05:00
Dhruv Chawla
e12bf36d23
[GISel][CombinerHelper] Combine op(trunc(x), trunc(y)) -> trunc(op(x, y)) (#89023) 2024-06-03 10:42:10 +05:30
paperchalice
8917afaf0e
Revert "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94146)
This reverts commit de37c06f01772e02465ccc9f538894c76d89a7a1 to
de37c06f01772e02465ccc9f538894c76d89a7a1

It still breaks EXPENSIVE_CHECKS build. Sorry.
2024-06-02 14:31:52 +08:00
paperchalice
d2cdc8ab45
[NewPM][CodeGen] Port selection dag isel to new pass manager (#83567)
Port selection dag isel to new pass manager.
Only `AMDGPU` and `X86` support new pass version. `-verify-machineinstrs` in new pass manager belongs to verify instrumentation, it is enabled by default.
2024-06-02 09:12:33 +08:00
Matt Arsenault
b263033c2b
AMDGPU: Remove arbitrary SCC liveness scan threshold (#94097) 2024-06-01 14:19:38 +02:00
Egor Pasko
cab81dd038
[EntryExitInstrumenter] Move passes out of clang into LLVM default pipelines (#92171)
Move EntryExitInstrumenter(PostInlining=true) to as late as possible and
EntryExitInstrumenter(PostInlining=false) to an early pre-inlining stage
(but skip for ThinLTO post-link).

This should fix the issues reported in
https://github.com/rust-lang/rust/issues/92109 and
https://github.com/llvm/llvm-project/issues/52853. These are caused
by https://reviews.llvm.org/D97608.
2024-05-31 12:48:45 -07:00
Stanislav Mekhanoshin
fc21387b65
[AMDGPU] Enable constant offset promotion to immediate FLAT (#93884)
Currently it is only supported for FLAT Global.
2024-05-31 12:23:27 -07:00
Konstantin Zhuravlyov
775f1cd34d
AMDGPU: Add gfx12-generic target (#93875) 2024-05-31 12:46:44 -04:00
Nikita Popov
63dc31b68b Reapply [IR] Avoid creating icmp/fcmp constant expressions (#92885)
Reapply after https://github.com/llvm/llvm-project/pull/93548,
which should address the lldb failure on macos.

-----

Do not create icmp/fcmp constant expressions in IRBuilder etc anymore,
i.e. treat them as "undesirable". This is in preparation for removing
them entirely.

Part of:
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179
2024-05-31 08:55:59 +02:00
Matt Arsenault
419d363385
AMDGPU: Add kernarg input attributes to preloaded kernarg test (#93669)
Explicitly mark the unused implicit arguments in the test, since this
should be sensitive to the number of free user SGPRs.

This is in preparation for #83131.
2024-05-30 23:50:24 +02:00
Stanislav Mekhanoshin
215f92b979
[AMDGPU] Fix crash in the SILoadStoreOptimizer (#93862)
It does not properly handle situation when address calculation uses
V_ADDC_U32 0, 0, carry-in (i.e. with both src0 and src1 immediates).
2024-05-30 14:27:33 -07:00
Christudasan Devadasan
e8de977716
[AMDGPU] Fixed subtarget name in the lit test check-prefix string (NFC). (#93780) 2024-05-30 20:27:24 +05:30
Jay Foad
ed25d1a82e [AMDGPU] Regenerate checks in inst-select-load-global.s96.mir 2024-05-30 14:02:15 +01:00
Sergei Barannikov
3fee8b3469
[GISel] LegalizationArtifactCombiner: Elide redundant G_SEXT_INREG (#93687)
This is similar to 373c343a, but for targets with zero-or-negative-one
booleans.
The difference in tests is mostly due to G_SEXT_INREG being illegal for
some targets, in which case it gets expanded into G_SHL/G_ASHR pair,
which is not currently optimized by the combiner.
2024-05-30 12:40:42 +03:00
Nikita Popov
d10b76552f
[ConstantFold] Remove notional over-indexing fold (#93697)
The data-layout independent constant folding currently has some rather
gnarly code for canonicalizing GEP indices to reduce "notional
overindexing", and then infers inbounds based on that canonicalization.

Now that we canonicalize to i8 GEPs, this canonicalization is
essentially useless, as we'll discard it as soon as the GEP hits the
data-layout aware constant folder anyway. As such, I'd like to remove
this code entirely.

This shouldn't have any impact on optimization capabilities.
2024-05-30 08:36:44 +02:00
Thorsten Schütt
6d90ac1e06
[GlobalIsel] Combine freeze (#93239) 2024-05-29 18:05:33 +02:00
Jay Foad
fbe98da623 [AMDGPU] Fix filecheck annotation typos
Co-authored-by: klensy <nightouser@gmail.com>
2024-05-29 15:01:48 +01:00
Lu Weining
5553f27d5a
[AMDGPU][test] Fix the wrong triples in lower-work-group-id-intrinsics-{hsa,pal}.ll. NFC (#93501)
- hsa -> amdhsa
- Use amdgcn-amd-amd{hsa,pal} for lower-work-group-id-intrinsics-{hsa,pal}.ll respectively
2024-05-29 17:42:41 +08:00
Pengcheng Wang
4e0bd3fab4
[MachineLICM] Hoist copies of constant physical register (#93285)
Previously, we just check if the source is a virtual register and
this prevents some potential hoists.

We can see some improvements in AArch64/RISCV tests.
2024-05-29 14:10:01 +08:00
Matt Arsenault
16a5fd3fdb
DAG: Use flags in isLegalToCombineMinNumMaxNum (#93555) 2024-05-28 18:57:38 +02:00
Matt Arsenault
fe5d791517 AMDGPU: Add some multi-use negative tests for minimum3/maximum3 2024-05-28 14:47:56 +02:00
Ralender
d46e37348e
[DebugCounter] Add support for non-continous ranges. (#89470) 2024-05-28 12:40:39 +02:00
Pierre van Houtryve
0e73bbd345
[AMDGPU][PromoteAlloca] Don't stop when an alloca is too big to promote (#93466)
When I rewrote this, I made a mistake in the control flow. I thought we
could just stop promoting if an alloca is too big to vectorize, but we
can't. Other allocas in the list may be promotable and fit within the
budget.

Fixes SWDEV-455343
2024-05-28 08:05:50 +02:00
Mirko Brkušanin
1e6a82b8ef
[AMDGPU] Legalize and select raw/struct_buffer_load with tfe (#93310) 2024-05-27 14:09:17 +02:00
Pierre van Houtryve
c1ac6d2dd4
[AMDGPU] Add amdgpu-as MMRA for fences (#78572)
Using MMRAs, allow `builtin_amdgcn_fence` to emit fences that only
target one or more address spaces, instead of fencing all address spaces
at once.

This is done through a `amdgpu-as` MMRA. Currently focused on OpenCL
fences, but can very easily support more AS names and codegen on more
than just fences.
2024-05-27 12:17:04 +02:00
Nikita Popov
1579e9ca9c Revert "Run ObjCContractPass in Default Codegen Pipeline (#92331)"
This reverts commit 8cc8e5d6c6ac9bfc888f3449f7e424678deae8c2.
This reverts commit dae55c89835347a353619f506ee5c8f8a2c136a7.

Causes major compile-time regressions for unoptimized builds.
2024-05-24 08:14:26 +02:00
Nuri Amari
8cc8e5d6c6
Run ObjCContractPass in Default Codegen Pipeline (#92331)
Prior to this patch, when using -fthinlto-index= the ObjCARCContractPass isn't run prior to CodeGen, and instruction selection fails on IR containing arc intrinsics. This patch is motivated by that usecase.

The pass was previously added in various places codegen is performed. This patch adds the pass to the default codegen pipepline, makes sure it bails immediately if no arc intrinsics are found, and removes the adhoc scheduling of the pass. 

Co-authored-by: Nuri Amari <nuriamari@fb.com>
2024-05-23 10:04:55 -07:00
Phoebe Wang
a0e6f83781
[ConstraintFP] Model rounding control registers for inline asm (#92846)
We have an internal test affected by
a69673615b.
The reason is we haven't modeled rounding control registers for inline
asm under constraint FP.
Here is a reduced case:  https://godbolt.org/z/s4EsKP94e
2024-05-23 22:50:39 +08:00
Matt Arsenault
2401b6126d
AMDGPU: Fix creating minimum3/maximum3 nodes pre-gfx12 (#93027)
These would fail to select.
2024-05-23 15:59:43 +02:00
Matt Arsenault
f06c1ce860
AMDGPU: Clean up maximum3/minimum3 tests (#93025)
These were using patterns copied from older tests, before non-kernel
functions were supported and manually written checks. Also stop using
-flat-for-global, which only exists to try to share tests between SI/CI
and VI+.

This was also missing test coverage, we're incorrectly forming
maximum3/minimum3 pre-gfx12. This is a pre-commit before fixing that.
2024-05-23 15:30:12 +02:00
Simon Pilgrim
85b9826c34 [AMDGPU] Regenerate sad.ll test checks
Improve test checks for better codegen review of #92576
2024-05-23 12:17:11 +01:00
csstormq
8a3537bfef
[llvm][ScheduleDAG] SUnit::biasCriticalPath() does not find the critical path consistently (#93001)
Patch co-authored by AtariDreams (gfunni234@gmail.com).

Fixes #38037.

[AMDGPU] Update test results to fix build (#92982)
2024-05-23 17:27:07 +08:00
Jay Foad
990bed64fb
[AMDGPU] New intrinsic llvm.amdgcn.pops.exiting.wave.id (#89612)
This provides access to the special scalar source value
SRC_POPS_EXITING_WAVE_ID on GFX9 and GFX10.
2024-05-22 19:47:59 +01:00
Daniel Thornburgh
8baf96f306
Revert "[IR] Avoid creating icmp/fcmp constant expressions" (#93087)
Reverts llvm/llvm-project#92885 due to LLDB CI breakages.
2024-05-22 11:27:55 -07:00
Jay Foad
154d93b7f2 [AMDGPU] Add a negative test for DPP combine into v_mul_lo_u32 2024-05-22 16:08:44 +01:00
Emma Pilkington
9e0be65f24
[AMDGPU] Fix broken MIR generated by gfx11 simulated trap lowering (#91652)
This was breaking the CFG connection between uses of virtual registers
after the trap and their definitions before it. Fixes SWDEV-460384.

Fixes a bug in #85854.
2024-05-22 10:55:19 -04:00
Janek van Oirschot
a699ccbf0c
MCExpr-ify amd_kernel_code_t (#91587)
Redefines the amd_kernel_code_t struct with MCExprs for members that would be
derived from SIProgramInfo MCExpr members.
2024-05-22 13:45:45 +01:00
Nikita Popov
1cf75cc77e Revert "[AMDGPU] Update test results to fix build (#92982)"
This reverts commit c769079b1098f6b108544176c0bd8b5bba986f2c.

This doesn't fix the build, it breaks it.
2024-05-22 08:49:38 +02:00
AtariDreams
c769079b10
[AMDGPU] Update test results to fix build (#92982) 2024-05-22 08:10:14 +02:00
Nikita Popov
108575f02e
[IR] Avoid creating icmp/fcmp constant expressions (#92885)
Do not create icmp/fcmp constant expressions in IRBuilder etc anymore,
i.e. treat them as "undesirable". This is in preparation for removing
them entirely.

Part of:
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179
2024-05-22 07:40:08 +02:00
Matt Arsenault
dab1f7c8d3
AMDGPU: Emit 1/llvm.sqrt(x) instead of rsqrt calls in libcall handling (#92863)
With the contract flag we should end up codegening to the rsqrt
instruction, or denormal corrected rsqrt sequence present in the
library.
2024-05-21 18:42:45 +02:00
Matt Arsenault
67e3514692
AMDGPU: Implement getRoundingControlRegisters (#92884) 2024-05-21 18:41:32 +02:00
Matt Arsenault
2bad2ee050 AMDGPU: Fix testing unaligned double atomics in atomic scan tests
Fix copy paste error using align 4 double atomics. Test the meaningful
correctly aligned cases, the unaligned case emits broken libcalls.
2024-05-21 11:20:08 +02:00
Matt Arsenault
e96948a6e8 AMDGPU: Fix typos in test names 2024-05-21 11:20:08 +02:00
Nikita Popov
74f1e29133 [Tests] Regenerate test checks (NFC)
To minimize future diffs.
2024-05-21 10:28:40 +02:00
Matt Arsenault
66b76faffb
AMDGPU: Directly emit sqrt intrinsic when folding rootn(x, 2) (#92598)
This avoids depending on pre/post link runs.

Depends #92595
2024-05-21 07:57:04 +02:00