79 Commits

Author SHA1 Message Date
Matt Arsenault
66ee794064 AMDGPU: Fix verifier error on splatted opencl fmin/fmax and ldexp calls
Apparently the spec has overloads for fmin/fmax and ldexp with one of
the operands as scalar. We need to broadcast the scalars to the vector
type.

https://reviews.llvm.org/D158077
2023-08-16 09:42:26 -04:00
Matt Arsenault
d251761660 AMDGPU: Replace log libcalls with log intrinsics 2023-08-15 10:48:46 -04:00
Matt Arsenault
d45022b094 AMDGPU: Remove special case constant folding of divide
We should probably just swap this out for the fdiv, but that's what
the implementation is anyway.
2023-08-14 18:36:01 -04:00
Matt Arsenault
483cc21866 AMDGPU: Remove special case folding of sqrt 2023-08-14 18:36:01 -04:00
Matt Arsenault
416f6af976 AMDGPU: Remove special case folding of fma/mad
These just get replaced with an intrinsic now. This was also
introducing host dependence on the result since it relied on the
compiler choice to contract or not.
2023-08-14 18:36:01 -04:00
Matt Arsenault
0eabe65bfb AMDGPU: Replace ldexp libcalls with intrinsic 2023-08-14 18:36:01 -04:00
Matt Arsenault
f337a77c99 AMDGPU: Replace rounding libcalls with intrinsics 2023-08-14 18:36:01 -04:00
Matt Arsenault
c7876c55ac AMDGPU: Replace fabs and copysign libcalls with intrinsics
Preserves flags and metadata like the other cases.
2023-08-14 18:28:21 -04:00
Matt Arsenault
a70006c4c5 AMDGPU: Replace some libcalls with intrinsics
OpenCL loses fast math information by going through libcall wrappers
around intrinsics.

Do this to preserve call site flags which are lost when inlining. It's
not safe in general to propagate flags during inline, so avoid dealing
with this by just special casing some of the useful calls.
2023-08-14 18:20:47 -04:00
Matt Arsenault
f44beecb78 AMDGPU: Try to use private version of sincos if available
The comment was out of date, the device libs build does provide all
the pointer overloads. An extremely pedantic interpretation of the
spec would suggest only the flat version exists, but the overloads do
exist in the implementation.

https://reviews.llvm.org/D156720
2023-08-14 11:40:04 -04:00
Matt Arsenault
42c6e4209c AMDGPU: Handle multiple uses when matching sincos
Match how the generic implementation handles this. We now will leave
behind the dead other user for later passes to deal with.

https://reviews.llvm.org/D156707
2023-08-14 11:28:41 -04:00
Matt Arsenault
6dbd458128 AMDGPU: Remove pointless libcall optimization of fma/mad
After the library is linked and trivially inlined, the generic fma and
fmuladd intrinsics already handle these cases, and with precise flag
handling. This was requiring all fast math flags when we really just
need nsz for the fma(a, b, 0) case.

https://reviews.llvm.org/D156677
2023-08-09 19:37:52 -04:00
Matt Arsenault
6448d5ba58 AMDGPU: Remove pointless libcall recognition of native_{divide|recip}
This was trying to constant fold these calls, and also turn some of
them into a regular fmul/fdiv. There's no point to doing that, the
underlying library implementation should be using those in the first
place. Even when the library does use the rcp intrinsics, the backend
handles constant folding of those. This was also only performing the
folds under overly strict fast-evertyhing-is-required conditions.

The one possible plus this gained over linking in the library is if
you were using all fast math flags, it would propagate them to the new
instructions. We could address this in the library by adding more fast
math flags to the native implementations.

The constant fold case also had no test coverage.

https://reviews.llvm.org/D156676
2023-08-09 18:48:46 -04:00
Matt Arsenault
b0f4b6587a AMDGPU: Delete probably wrong constant folding of expm1
It's not really correct to implement this as exp(x) - 1, it was maybe
OK for the restricted float-as-double case handled here. There's not a
strong reason to special case it with the host function, as the
implementation naturally constant folds anyway. InstSimplify can fully
handle everything in it, so just running the inliner alone with a
constant argument produces the fully constant folded result. This also
had no tests.

https://reviews.llvm.org/D156892
2023-08-04 21:01:44 -04:00
Matt Arsenault
54bda79335 AMDGPU: Simplify and improve sincos matching
The first trivial example I tried failed to merge due to the user scan
logic. Remove the complicated scan of users handling with distance
thresholds, with a same block restriction. The actual expansion of
sincos is basically the same size as sin or cos individually. Copy the
technique the generic optimization uses, which is to just use the
input instruction as the insert point or just insert at the start of
the entry block.

https://reviews.llvm.org/D156706
2023-08-02 17:48:35 -04:00
Matt Arsenault
9a806551a0 AMDGPU: Delete old PM support for libcall passes
This has no reason to run in the codegen pipeline.
2023-08-01 18:22:02 -04:00
Matt Arsenault
5dfdd3494b AMDGPU: Don't try to fold wavefrontsize intrinsic in libcall simplify
It's not a libcall so doesn't really belong here to begin
with. Relying on checking the target name and explicit features isn't
particularly sound either. The library doesn't use the intrinsic
anymore, so it doesn't matter anyway.
2023-08-01 18:20:50 -04:00
Matt Arsenault
3b2f3238a4 AMDGPU: Don't try memory optimizations in libcall optimizer
This was trying to find a loaded value for some reason when looking
for sincos arguments. This is untested and shouldn't be necessary.

https://reviews.llvm.org/D156746
2023-08-01 18:10:22 -04:00
Matt Arsenault
db4d6ef9ef AMDGPU: Directly emit fabs intrinsic instead of new libcall 2023-07-31 19:19:56 -04:00
Matt Arsenault
f63cdfc4cf AMDGPU: Move check of compatible libcall
https://reviews.llvm.org/D156681
2023-07-31 16:47:07 -04:00
Matt Arsenault
c2c22c6c95 AMDGPU: Don't store current instruction in AMDGPULibCalls member
This was adding confusing global state which was shadowed most of the
time.

https://reviews.llvm.org/D156680
2023-07-31 11:44:21 -04:00
Matt Arsenault
8f38138090 AMDGPU: Refactor libcall simplify to help with future refined fast math flag usage
https://reviews.llvm.org/D156678
2023-07-31 11:23:12 -04:00
Matt Arsenault
94d55450d2 AMDGPU: Don't parse name of sin/cos twice in libcall simplify 2023-07-31 11:23:12 -04:00
Matt Arsenault
8a677a7ff0 AMDGPU: Partially respect nobuiltin in libcall simplifier
There are more contexts where it's not handled correctly but this is
the simplest one.

https://reviews.llvm.org/D156682
2023-07-31 10:56:46 -04:00
Matt Arsenault
2dc1a27449 AMDGPU: Some AMDGPULibCalls cleanups
dyn_cast instead of isa+cast, and initialize on declaration.
2023-07-31 10:53:09 -04:00
Matt Arsenault
ab6cd2d498 AMDGPU: Simplify early exit handling for libcall simplify
Early exit on intrinsics and don't duplicate indirect call
checks. Also let the IRBuilder constructor figure out the insert point
rather than doing it manually. Also avoid debug print about trying to
simplify calls in more unhandled scenarios.
2023-07-31 08:18:12 -04:00
Matt Arsenault
360a5d5612 AMDGPU: Remove some typed pointer handling 2023-07-31 08:05:12 -04:00
Nico Weber
90f7f24b20 try to fix build yet more after 16544cbe64b8 2022-09-28 15:40:52 -04:00
Kazu Hirata
ae998555ba [AMDGPU] Remove a redundant variable (NFC)
ArrayRef has operator[], so we don't need to access the contents via
data().
2022-07-23 12:29:05 -07:00
Kazu Hirata
6cbfffb3a3 [AMDGPU] Declare TableRef in terms of ArrayRef (NFC) 2022-07-16 10:56:20 -07:00
Simon Pilgrim
8de7297374 [AMDGPU] Pull out repeated getVecSize() calls. NFC.
This is guaranteed to be evaluated so we can avoid repeated calls.

Helps the static analyzer as it couldn't recognise that each getVecSize() would return the same value.
2022-02-10 16:31:36 +00:00
serge-sans-paille
e188aae406 Cleanup header dependencies in LLVMCore
Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E  -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after:  6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652
2022-02-02 06:54:20 +01:00
Kazu Hirata
bc360fd83a [AMDGPU] Remove unused declarations fold_exp* and fold_log* (NFC) 2021-12-31 16:50:18 -08:00
Kazu Hirata
5c4b9ea4a7 [AMDGPU] Remove replaceWithNative (NFC)
The function was introduced without any use on Aug 11, 2017 in commit
7f37794ebd2c6c36224597800e4d1e5a99ad80e9.
2021-12-31 16:43:06 -08:00
Kazu Hirata
5a667c0e74 [llvm] Use nullptr instead of 0 (NFC)
Identified with modernize-use-nullptr.
2021-12-28 08:52:25 -08:00
Neubauer, Sebastian
d1f45ed58f [AMDGPU][NFC] Fix typos
Differential Revision: https://reviews.llvm.org/D113672
2021-11-12 11:37:21 +01:00
Kazu Hirata
6fe949c4ed [Target, Transforms] Use StringRef::contains (NFC) 2021-10-22 08:52:33 -07:00
Simon Pilgrim
a750332d77 AMDGPULibCalls - constify some FuncInfo& arguments. NFCI. 2021-10-22 12:10:58 +01:00
Simon Pilgrim
99a64cc9da AMDGPULibCalls::parseFunctionName - use reference instead of pointer. NFCI.
parseFunctionName allowed a default null pointer, despite it being dereferenced immediately to be used as a reference and that all callers were taking the address of an existing reference.

Fixes static analyzer warning about potential dereferenced nulls
2021-10-22 11:45:25 +01:00
Kazu Hirata
c1e32b3fc0 [Target] Migrate from getNumArgOperands to arg_size (NFC)
Note that getNumArgOperands is considered a legacy name.  See
llvm/include/llvm/IR/InstrTypes.h for details.
2021-10-02 12:06:29 -07:00
Jacob Lambert
dc6e8dfdfe [AMDGPU][NFC] Correct typos in lib/Target/AMDGPU/AMDGPU*.cpp files. Test commit for new contributor. 2021-09-20 14:48:50 -07:00
Martin Storsjö
42f74e8249 [llvm] Rename StringRef _lower() method calls to _insensitive()
This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods don't seem to be used outside of the llvm subproject, so
this doesn't break building of the rest of the monorepo.
2021-06-25 00:22:01 +03:00
Nikita Popov
9914200393 [CodeGen] Add missing includes (NFC)
These currently rely on the IRBuilder.h include in TargetLowering.h.
Make them explicit.
2021-06-06 15:48:27 +02:00
Serge Guelton
d6de1e1a71 Normalize interaction with boolean attributes
Such attributes can either be unset, or set to "true" or "false" (as string).
throughout the codebase, this led to inelegant checks ranging from

        if (Fn->getFnAttribute("no-jump-tables").getValueAsString() == "true")

to

        if (Fn->hasAttribute("no-jump-tables") && Fn->getFnAttribute("no-jump-tables").getValueAsString() == "true")

Introduce a getValueAsBool that normalize the check, with the following
behavior:

no attributes or attribute set to "false" => return false
attribute set to "true" => return true

Differential Revision: https://reviews.llvm.org/D99299
2021-04-17 08:17:33 +02:00
dfukalov
560d7e0411 [NFC][AMDGPU] Split AMDGPUSubtarget.h to R600 and GCN subtargets
... to reduce headers dependency.

Reviewed By: rampitec, arsenm

Differential Revision: https://reviews.llvm.org/D95036
2021-01-20 22:22:45 +03:00
dfukalov
6a87e9b08b [NFC][AMDGPU] Reduce include files dependency.
Reviewed By: rampitec

Differential Revision: https://reviews.llvm.org/D93813
2021-01-07 22:22:05 +03:00
Arthur Eubanks
8e293fe6ad [NewPM][AMDGPU] Pass TargetMachine to AMDGPUSimplifyLibCallsPass
Missed in https://reviews.llvm.org/D93863.
2021-01-04 13:48:09 -08:00
Arthur Eubanks
9abc457724 [NewPM][AMDGPU] Port amdgpu-simplifylib/amdgpu-usenative
And add them to the pipeline via
AMDGPUTargetMachine::registerPassBuilderCallbacks(), which mirrors
AMDGPUTargetMachine::adjustPassManager().

These passes can't be unconditionally added to PassRegistry.def since
they are only present when the AMDGPU backend is enabled. And there are
no target-specific headers in llvm/include, so parsing these pass names
must occur somewhere in the AMDGPU directory. I decided the best place
was inside the TargetMachine, since the PassBuilder invokes
TargetMachine::registerPassBuilderCallbacks() anyway. If we come up with
a cleaner solution for target-specific passes in the future that's fine,
but there aren't too many target-specific IR passes living in
target-specific directories so it shouldn't be too bad to change in the
future.

Reviewed By: ychen, arsenm

Differential Revision: https://reviews.llvm.org/D93863
2020-12-28 10:38:51 -08:00
Stanislav Mekhanoshin
c9821cec74 [AMDGPU] Mark sin/cos load folding as modifying the function.
When the load value is folded into the sin/cos operation, the
AMDGPU library calls simplifier could still mark the function
as unmodified. Instead ensure if there is an early return,
return whether the load was folded into the sin/cos call.

Authored by MJDSys

Differential Revision: https://reviews.llvm.org/D91401
2020-11-13 14:49:33 -08:00
Kazu Hirata
902cbcd59e Use llvm::is_contained where appropriate (NFC)
Summary:
This patch replaces std::find with llvm::is_contained where
appropriate.

Reviewers: efriedma, nhaehnle

Reviewed By: nhaehnle

Subscribers: arsenm, jvesely, nhaehnle, hiraditya, rogfer01, kerbowa, llvm-commits, vkmr

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D84489
2020-07-27 10:20:44 -07:00