Change getPointersDiff to return an std::optional<int64_t>, and fill
this value with using APInt::trySExtValue. This simple change requires
changes to other functions in LAA, and major changes in SLPVectorizer
changing types from 32-bit to 64-bit.
Fixes#139202.
1. There are failed LIT tests at the moment due to type inference
errors.
```
Failed Tests (3):
LLVM :: CodeGen/SPIRV/pointers/ptr-eq-types.ll
LLVM :: CodeGen/SPIRV/validate/sycl-hier-par-basic.ll
LLVM :: CodeGen/SPIRV/validate/sycl-tangle-group-algorithms.ll
```
This PR improves type inference to fix the errors.
2. The following tests start passing:
```
Unexpectedly Passed Tests (2):
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast-2.ll
LLVM :: CodeGen/SPIRV/pointers/resource-addrspacecast.ll
```
This PR removes XFAILS in those two test cases.
`SIMDScalarLShiftDTied` should be using `vecshiftL64` to match
the immediate argument rather than `vecshiftR64` as the latter
prevents the pattern from matching 0 (and allows 64 instead).
Fixes#139879.
Replace all instances of `INS(IMPLICIT_DEF, 0, v, idx)` with
`DUP(v, idx)` in instruction selection.
`INS` (e.g. `mov v0.s[0], v1.s[1]`) has a value dependency on its output
register, which becomes a false dependency when we're inserting into an
`IMPLICIT_DEF` register. We can break this false dependency by using
`DUP` (e.g. `mov s0, v1.s[1]`) instead.
Setting unitialized pointers to nullptr in InnerLoopVectorizer()
constructor. These were noticed during a review of the code. Seems like
a good idea to clean them up.
Prior to this commit PREFETCH_* instructions considered their operand
register to be a simple GPR which is not entirely correct as it is a
base address for possible prefetching operation (According to cmobase
v1.0.1)
For now expansion still happens in SelectionDAGBuilder when
GET_ACTIVE_LANE_MASK is not legal on the target.
This patch also includes changes in AArch64ISelLowering to replace
handling of the get.active.lane.mask intrinsic to use the ISD node.
Tablegen patterns are added which match to whilelo for scalable types.
A follow up change will add support for more types to be lowered to
GET_ACTIVE_LANE_MASK by allowing splitting of the node.
It is used to mark a value that we are sure that it is not some fcType.
The examples include:
* An arguments of a function is marked with nofpclass
* Output value of an intrinsic can be sure to not be some type
So that the following operation can make some assumptions.
FullLTO has to run this early before module splitting occurs otherwise
module splitting won't work as expected. There was a targeted fix for
fortran on another branch that disables the LTO run but that'd break
full LTO module splitting entirely.
Test changes are due to metadata indexes shifting.
See #122891
SMECallAttrs is a new helper class that holds all the SMEAttrs for a
call. The interfaces to query actions needed for the call (e.g. change
streaming mode) have been moved to the SMECallAttrs class.
The main motivation for this change is to make the split between the
caller, callee, and callsite attributes more apparent.
Before this change, we would always merge callsite and callee
attributes. The main reason to do this was to handle indirect calls,
however, we also occasionally used callsite attributes on direct calls
in tests (mainly to avoid creating multiple function declarations). With
this patch, we now explicitly handle indirect calls and disallow
incompatible attributes on direct calls (so this patch is not entirely
an NFC).
Same as #137239, but with a change to avoid inferring SME attributes for
function definitions. This allows stubbing the SME ABI routines in C/C++
(and matches the old behaviour).
Make this consistent with other operations with respect to
signaling nan quieting. This was specifying that quieting is
required, which is true for IEEE. Make this consistent with other
IR operations, and signaling nan quieting is possible but optional
in the case where there are two nan inputs.
This permits directly selecting the intrinsic to the hardware
instruction in the default floating-point environment for shaders.
Adds constructor for resources with implicit binding and applies it to
all resources without binding at the global scope.
Adds Clang builtin function
`__builtin_hlsl_resource_handlefromimplicitbinding` that gets translated
to `llvm.dx|spv.resource.handlefromimplicitbinding` intrinsic calls.
Specific bindings are assigned in DXILResourceImplicitBinding pass.
Design proposals:
https://github.com/llvm/wg-hlsl/blob/main/proposals/0024-implicit-resource-binding.mdhttps://github.com/llvm/wg-hlsl/blob/main/proposals/0025-resource-constructors.md
One change from the proposals is that the `orderId` parameter is added
onto the constructor. Originally it was supposed to be generated in
codegen when the `llvm.dx|spv.resource.handlefromimplicitbinding` call
is emitted, but that is not possible because the call is inside a
constructor, and the constructor body is generated once per resource
type and not resource instance. So the only way to inject instance-based
data like `orderId` into the
`llvm.dx|spv.resource.handlefromimplicitbinding` call is that it must
come in via the constructor argument.
Closes#136784
Replace deprecated use of getDeclaration that was added in #132489
llvm/lib/IR/AutoUpgrade.cpp:1480:26: error: 'getDeclaration' is deprecated: Use getOrInsertDeclaration instead [-Werror,-Wdeprecated-declarations]
1480 | NewFn = Intrinsic::getDeclaration(
| ^~~~~~~~~~~~~~
| getOrInsertDeclaration
Reformat a few header files under llvm/include/llvm/CodeGen/Passes.h
with clang-format in preparation for a codemod.
This is just a formatting change; no functionality is impacted.
This patch disables the skip unsupported instructions test on AArch64. On
machines that have hardware support for pointer authentication the test will
try and disable certain features, but currently fails depending upon the
configuration.
Thread-local globals live, by default, in the default globals address
space, which may not be 0, so we need to overload @llvm.thread.pointer
to support other address spaces, and use the default globals address
space in Clang.
When we enable EVL-based loop vectorization w/ predicated tail-folding,
each vectorized loop has effectively two induction variables: one
calculates the step using (VF x vscale) and the other one increases the
IV by values returned from experiment.get.vector.length. The former,
also known as canonical IV, is more favorable for analyses as it's
"countable" in the sense of SCEV; the latter (EVL-based IV), however, is
more favorable to codegen, at least for those that support scalable
vectors like AArch64 SVE and RISC-V.
The idea is that we use canonical IV all the way until the end of all
vectorizers, where we replace it with EVL-based IV using EVLIVSimplify
introduced here. Such that we can have the best from both worlds.
This Pass is enabled by default in RISC-V. However, since we haven't
really vectorize loops with predicate tail-folding by default, this Pass
is no-op at this moment.
## Purpose
Add proper preprocessor guards for all `dump()` methods in the LLVM
support library. This change ensures these methods are not part of the
public ABI for release builds.
## Overview
* Annotates all `dump` methods in Support and ADT source with the
`LLVM_DUMP_METHOD` macro.
* Conditionally includes all `dump` method definitions in Support and
ADT source so they are only present on debug/assert builds and when
`LLVM_ENABLE_DUMP` is explicitly defined.
NOTE: For many of these `dump` methods, the implementation was already
properly guarded but the declaration in the header file was not.
## Background
This PR is a redo of #139804 with some changes to fix clang and unit
test build breaks.
This issue was raised in comments on #136629. I am addressing it as a
separate change since it is independent from the changes being made in
that PR.
According to [this
documentation](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/Support/Compiler.h#L637),
`dump` methods should be annotated with `LLVM_DUMP_METHOD` and
conditionally included as follows:
```
#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
LLVM_DUMP_METHOD void dump() const;
#endif
```
## Validation
* Local release build succeeds.
* CI
- Use StringRef instead of std::string for `InstructionMemo::Name`.
- Use range for loops, zip_equal and structured bindings in loops.
- Use llvm::any_of instead of manual loops.
- Use ListSeparator.
- Remove {} around single-line if-else chains.
- Use ArrayRef<> instead of const vector reference for function args.
- Change `getLegalCName` to accept a `StringRef` to avoid
StringRef->std::string casting in several places.
- Use StringRef instead of std::string for `OpcodeName` (and in
associated maps).
Tested by verifying no changes in .inc files with and without this
change.
Ideally we should perhaps even error out here; if the user requested
building with profiling info, but that profile wasn't found, it seems
like the build configuration is subtly broken, and I would expect the
user to want to know about it. But that may be a too disruptive change,
as users may very well rely on being able to build things in such a
setup anyway thanks to Hyrum's Law.
To make things clearer, at least print a warning in this case.
This is a follow-up to #138203. The added test used lld but lld is not
always available, which breaks builds. Make the test use llvm-lto2. Also
make the test a bit more similar to other tests in the same directory.
This change was separated from
https://github.com/llvm/llvm-project/pull/119001.
When cloning functions, use IdentityMDPredicate to ensure that if
DISubprogram is not cloned, then its DILocalVariables are not cloned
either.
This is currently expected to be an NFC, as DILocalVariables only
reference their subprograms (via DILocalScopes) and types, and inlined
DISubprograms and DITypes are not cloned. Thus, DILocalVariables are
mapped to self in ValueMapper (in mapTopLevelUniquedNode).
However, it will be needed for the original PR
https://github.com/llvm/llvm-project/pull/119001, where a
DILocalVariable may refer to a local type of a DISubprogram that is
being cloned. In that case, ValueMapper will clone DILocalVariable even
if it belongs to an inlined DISubprogram that is not cloned, which
should be avoided.
I'm making this change into a separate PR to make the original PR a bit
smaller, and because this has more to do with variables than with types.
Fixes#138997 and does refactoring for low-precision-related flags.
The shader feature flags MinimumPrecision and NativeLowPrecision were
not being set, leading to validation errors.
This PR sets these shader feature flags [as in
DXC](377c4ca6d8/lib/DXIL/DxilShaderFlags.cpp (L58-L63))
and adds tests for them.
This PR also performs some refactoring of low-precision-related flags to
make it less confusing.
- The `UseNativeLowPrecision` DXIL module flag has been renamed to
`NativeLowPrecisionMode` to imply that it is setting some execution
state which the module should be interpreted with
- The LLVM module flag `dx.nativelowprec` is now read only once and sets
a bool to be used by `updateFunctionFlags()` and for setting the DXIL
module flag `NativeLowPrecisionMode`
- The `MinimumPrecision`, `NativeLowPrecision`, and
`LowPrecisionPresent` shader feature flags are all set together under
`updateFunctionFlags()`
- Moved the logic for setting DXIL module flags `NativeLowPrecisionMode`
and `ResMayNotAlias` out of the per-function loop and placed it
alongside the logic for setting other DXIL module flags
(`DisableOptimizations`, `Max64UAVs`, and `UAVsAtEveryStage` flags)
---------
Co-authored-by: Justin Bogner <mail@justinbogner.com>
For i1 vectors, we used an i8 fixed vector as the storage type.
If the known minimum number of elements of the scalable vector type is
less than 8, we were doing the cast through memory. This used a load or
store from a fixed vector alloca. If is less than 8, DataLayout
indicates that the load/store reads/writes vscale bytes even if vscale
is known and vscale*X is less than or equal to 8. This means the load or
store is outside the bounds of the fixed size alloca as far as
DataLayout is concerned leading to undefined behavior.
This patch avoids this by widening the i1 scalable vector type with zero
elements until it is divisible by 8. This allows it be bitcasted to/from
an i8 scalable vector. We then insert or extract the i8 fixed vector
into this type.
Hopefully this enables #130973 to be accepted.
Don't split i64 load/store when we have Zilsd.
In the future, we should enhanced the LoadStoreOptimizer pass to do
this, but this is a good starting point. Even if we support it in
LoadStoreOptimizer, we might still want this for volatile loads/stores
to guarantee the use of Zilsd.
## Purpose
Add proper preprocessor guards for all `dump()` methods in the LLVM
support library. This change ensures these methods are not part of the
public ABI for release builds.
## Overview
* Annotates all `dump` methods in Support and ADT source with the
`LLVM_DUMP_METHOD` macro.
* Conditionally includes all `dump` method definitions in Support and
ADT source so they are only present on debug/assert builds and when
`LLVM_ENABLE_DUMP` is explicitly defined.
NOTE: For many of these `dump` methods, the implementation was already
properly guarded but the declaration in the header file was not.
## Background
This issue was raised in comments on #136629. I am addressing it as a
separate change since it is independent from the changes being made in
that PR.
According to [this
documentation](https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/Support/Compiler.h#L637),
`dump` methods should be annotated with `LLVM_DUMP_METHOD` and
conditionally included as follows:
```
#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
LLVM_DUMP_METHOD void dump() const;
#endif
```
## Validation
* Local release build succeeds.
* CI
