1. There are failed LIT tests at the moment due to type inference
errors.
```
Failed Tests (3):
LLVM :: CodeGen/SPIRV/pointers/ptr-eq-types.ll
LLVM :: CodeGen/SPIRV/validate/sycl-hier-par-basic.ll
LLVM :: CodeGen/SPIRV/validate/sycl-tangle-group-algorithms.ll
```
This PR improves type inference to fix the errors.
2. The following tests start passing:
```
Unexpectedly Passed Tests (2):
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast-2.ll
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast.ll
```
This PR removes XFAILS in those two test cases.
`SIMDScalarLShiftDTied` should be using `vecshiftL64` to match
the immediate argument rather than `vecshiftR64` as the latter
prevents the pattern from matching 0 (and allows 64 instead).
Fixes#139879.
Replace all instances of `INS(IMPLICIT_DEF, 0, v, idx)` with
`DUP(v, idx)` in instruction selection.
`INS` (e.g. `mov v0.s[0], v1.s[1]`) has a value dependency on its output
register, which becomes a false dependency when we're inserting into an
`IMPLICIT_DEF` register. We can break this false dependency by using
`DUP` (e.g. `mov s0, v1.s[1]`) instead.
It is used to mark a value that we are sure that it is not some fcType.
The examples include:
* An arguments of a function is marked with nofpclass
* Output value of an intrinsic can be sure to not be some type
So that the following operation can make some assumptions.
FullLTO has to run this early before module splitting occurs otherwise
module splitting won't work as expected. There was a targeted fix for
fortran on another branch that disables the LTO run but that'd break
full LTO module splitting entirely.
Test changes are due to metadata indexes shifting.
See #122891
SMECallAttrs is a new helper class that holds all the SMEAttrs for a
call. The interfaces to query actions needed for the call (e.g. change
streaming mode) have been moved to the SMECallAttrs class.
The main motivation for this change is to make the split between the
caller, callee, and callsite attributes more apparent.
Before this change, we would always merge callsite and callee
attributes. The main reason to do this was to handle indirect calls,
however, we also occasionally used callsite attributes on direct calls
in tests (mainly to avoid creating multiple function declarations). With
this patch, we now explicitly handle indirect calls and disallow
incompatible attributes on direct calls (so this patch is not entirely
an NFC).
Same as #137239, but with a change to avoid inferring SME attributes for
function definitions. This allows stubbing the SME ABI routines in C/C++
(and matches the old behaviour).
Thread-local globals live, by default, in the default globals address
space, which may not be 0, so we need to overload @llvm.thread.pointer
to support other address spaces, and use the default globals address
space in Clang.
Fixes#138997 and does refactoring for low-precision-related flags.
The shader feature flags MinimumPrecision and NativeLowPrecision were
not being set, leading to validation errors.
This PR sets these shader feature flags [as in
DXC](377c4ca6d8/lib/DXIL/DxilShaderFlags.cpp (L58-L63))
and adds tests for them.
This PR also performs some refactoring of low-precision-related flags to
make it less confusing.
- The `UseNativeLowPrecision` DXIL module flag has been renamed to
`NativeLowPrecisionMode` to imply that it is setting some execution
state which the module should be interpreted with
- The LLVM module flag `dx.nativelowprec` is now read only once and sets
a bool to be used by `updateFunctionFlags()` and for setting the DXIL
module flag `NativeLowPrecisionMode`
- The `MinimumPrecision`, `NativeLowPrecision`, and
`LowPrecisionPresent` shader feature flags are all set together under
`updateFunctionFlags()`
- Moved the logic for setting DXIL module flags `NativeLowPrecisionMode`
and `ResMayNotAlias` out of the per-function loop and placed it
alongside the logic for setting other DXIL module flags
(`DisableOptimizations`, `Max64UAVs`, and `UAVsAtEveryStage` flags)
---------
Co-authored-by: Justin Bogner <mail@justinbogner.com>
Don't split i64 load/store when we have Zilsd.
In the future, we should enhanced the LoadStoreOptimizer pass to do
this, but this is a good starting point. Even if we support it in
LoadStoreOptimizer, we might still want this for volatile loads/stores
to guarantee the use of Zilsd.
Packed structs get a different layout than a struct that is not packed.
This is handled by assigning different offset decoration when
appropriate. The `CPacked` decoration is not required, and is in fact
not valid when creating a shader.
Similaraly the alignment decoration is not allows when creating a
shader. We must avoid generating that decoration.
Fixes https://github.com/llvm/llvm-project/issues/138268
Fold `(srl (lop x, (shl (zext y), c1)), c1) -> (lop (srl x, c1), (zext y))` where c1 <= leadingzeros(zext(y)).
This is equivalent of existing fold chain `(srl (shl (zext y), c1), c1) -> (and (zext y), mask) -> (zext y)`, but logical op in the middle prevents it from combining.
Profit : Allow to reduce the number of instructions.
---------
Signed-off-by: Alexander Peskov <apeskov@nvidia.com>
This adds a GISelValueTrackingPrinterPass that can print the known bits
and sign bit of each def in a function. It is built on the new pass
manager and so adds a NPM GISelValueTrackingAnalysis, renaming the older
class to GISelValueTrackingAnalysisLegacy.
The first 2 functions from the AArch64GISelMITest are ported over to an
mir test to show it working. It also runs successfully on all files in
llvm/test/CodeGen/AArch64/GlobalISel/*.mir that are not invalid. It can
hopefully be used to test GlobalISel known bits analysis more directly
in common cases, without jumping through the hoops that the C++ tests
requires.
This change adds intrinsics and clang builtins for the cvt instruction
variants of type (FP4) `.e2m1x2`. introduced in PTX 8.6 for `sm_100a`,
`sm_101a`, and `sm_120a`.
Tests are added in `NVPTX/convert-sm100a.ll` and
`clang/test/CodeGen/builtins-nvptx.c` and verified through ptxas 12.8.0.
PTX Spec Reference:
https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-cvt
Canonicalize it to a G_AND instead so that ISel patterns can pick it
up and ignore it, as the shift instructions only read low bits.
G_ZEXT would be lowered to a v/s_and anyway in most cases.
I'm also looking at making a DAG version of this in a separate patch.
Before V_CNDMASK_B32_e64 gets converted to SDWA form, a conversion to
V_CNDMASK_B32_e32 occurs.
The vcc use of this instruction must be fixed into a vcc_lo use for wave32.
This fix only happens after the final conversion to the SDWA form. This led
to a compiler error in situations where the conversion to SDWA aborts.
Make sure that the vcc-fix gets applied even if the SDWA conversion is
not completed.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
RISC-V Zicfiss-based shadow stack needs to let the linker/loader know if
the binary is built with the mechanism enabled to support proper
link-time/load-time management of this feature. The information is
encoded as a bit in the `.note.gnu.property` section. This patch
implements emitting the section for RISC-V targets when Zicfiss-based
shadow stack is enabled.
When Clang receives the `-fcf-protection=return` flag, it adds the
`hw-shadow-stack` attribute to LLVM functions, and adds a non-zero
valued attribute named `cf-protection-return` to the LLVM module it
generates. The backend depends on the `hw-shadow-stack` attributes to
generate Zicfiss-based shadow stack instructions for each function, but
at the module scope, the `cf-protection-return` attribute is a better
indication of whether the translation unit is built with Zicfiss-based
shadow stack enabled, so this patch emits the `.note.gnu.property`
section with the "Zicfiss-based shadow stack" bit toggled on when it
sees the `cf-protection-return` attribute.
Add inreg test for sgpr purpose
This is the second PR after
https://github.com/llvm/llvm-project/pull/135729.
To test sgpr inputs and outputs, using inreg cases for bit-conversions
---------
Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com>
Move dxil resource access legacy pass before intrinsic expansion legacy
pass so TypedBuffer Loads and Stores will be created before intrinsic
expansion.
This is to facilitate #104423
This updates all the extensions to their version in the v0.11.0 spec.
All changes from this version are already implemented or are not
relevant to LLVM.
This change also alphabetises the lists of Xqci extensions, to make
future checks easier, and removes irrelevant info from the usage docs.
The alignment on a ISD::DYNAMIC_STACKALLOC node may be 0 to indicate
that the default stack alignment should be used. Prior to this change,
we passed this alignment through unchanged leading to an error in
ptxas. Now, we use the stack-alignment in this case. Also did a little
cleanup while I'm here.
This adds basic support for objc_claimAutoreleasedReturnValue, which is
mostly equivalent to objc_retainAutoreleasedReturnValue, with the
difference that it doesn't require the marker nop to be emitted between
it and the call it was attached to.
To achieve that, this also teaches the AArch64 attachedcall bundle
lowering to pick whether the marker should be emitted or not based on
whether the attachedcall target is claimARV or retainARV.
Co-authored-by: Ahmed Bougacha <ahmed@bougacha.org>
Fix the Gather's base and index for one use or multiple uses of Index Node. Using the approach to update the Scale if SHL Opcode and followed by truncate.
---------
Co-authored-by: Rohit Aggarwal <Rohit.Aggarwal@amd.com>
Co-authored-by: Simon Pilgrim <llvm-dev@redking.me.uk>
Preserving the original result element type when splitting vector setcc
operations removes redundant extensions that are awkward to optimise
after the fact.
When -use-constant-{int,fp}-for-fixed-length-splat are enabled, constant
vector splats take the form of ConstantInt/FP instead of ConstantVector.
These constants get linked to MachineInstrs via constant pools for later
processing. The processing assumes ConstantInt/FP to always represent
scalar constants with this PR extending the code to support vector
types.
NOTE: The test choices are somewhat artificial because pretty much all
the vector tests failed without these changes when the new constants are
enabled.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
GlobalISel was previously inefficient in handling bitreverses of vector
types. This deals with i16, i32, i64 vector types and converts them into
i8 bitreverses and rev instructions.
Fixes buildbot failure
https://lab.llvm.org/buildbot/#/builders/16/builds/18873 originating
from #138448. Normally ignored silently but fails on higher error
levels.
Buildbot errors:
```
/b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/llc < /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/CodeGen/AArch64/reserveXreg.ll -mtriple=aarch64-unknown-linux-gnu | /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/FileCheck /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/CodeGen/AArch64/reserveXreg.ll # RUN: at line 6
+ /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/FileCheck /b/1/llvm-clang-x86_64-expensive-checks-debian/llvm-project/llvm/test/CodeGen/AArch64/reserveXreg.ll
+ /b/1/llvm-clang-x86_64-expensive-checks-debian/build/bin/llc -mtriple=aarch64-unknown-linux-gnu
'+reserve-x8' is not a recognized feature for this target (ignoring feature)
'+reserve-x8' is not a recognized feature for this target (ignoring feature)
'+reserve-x16' is not a recognized feature for this target (ignoring feature)
'+reserve-x16' is not a recognized feature for this target (ignoring feature)
'+reserve-x17' is not a recognized feature for this target (ignoring feature)
'+reserve-x17' is not a recognized feature for this target (ignoring feature)
```
This reapplies 067caaa and 382a085 (reverting b35f6e2) with fixes to
issues detected by the address sanitizer (MIs have to be removed from
live intervals before being removed from their parent MBB).
Original commit description below.
AMDGPU scheduler's `PreRARematStage` attempts to increase function
occupancy w.r.t. ArchVGPR usage by rematerializing trivial
ArchVGPR-defining instruction next to their single use. It first
collects all eligible trivially rematerializable instructions in the
function, then sinks them one-by-one while recomputing occupancy in all
affected regions each time to determine if and when it has managed to
increase overall occupancy. If it does, changes are committed to the
scheduler's state; otherwise modifications to the IR are reverted and
the scheduling stage gives up.
In both cases, this scheduling stage currently involves repeated queries
for up-to-date occupancy estimates and some state copying to enable
reversal of sinking decisions when occupancy is revealed not to
increase. The current implementation also does not accurately track
register pressure changes in all regions affected by sinking decisions.
This commit refactors this scheduling stage, improving RP tracking and
splitting the stage into two distinct steps to avoid repeated occupancy
queries and IR/state rollbacks.
- Analysis and collection (`canIncreaseOccupancyOrReduceSpill`). The
number of ArchVGPRs to save to reduce spilling or increase function
occupancy by 1 (when there is no spilling) is computed. Then,
instructions eligible for rematerialization are collected, stopping as
soon as enough have been identified to be able to achieve our goal
(according to slightly optimistic heuristics). If there aren't enough of
such instructions, the scheduling stage stops here.
- Rematerialization (`rematerialize`). Instructions collected in the
first step are rematerialized one-by-one. Now we are able to directly
update the scheduler's state since we have already done the occupancy
analysis and know we won't have to rollback any state. Register
pressures for impacted regions are recomputed only once, as opposed to
at every sinking decision.
In the case where the stage attempted to increase occupancy, and if both
rematerializations alone and rescheduling after were unable to improve
occupancy, then all rematerializations are rollbacked.
Currently the apss tries to combine floating point reductions, without
checking for the correct fast-math flags and it also creates invalid IR
(using llvm.reduce.add for FP types).
For now, just bail out for non-integer types.
PR: https://github.com/llvm/llvm-project/pull/139469
