This PR aims to reland work done by @arsenm which was previously
reverted due to some tangentially related scheduler issues as discussed
on #76416.
This PR cherry-picks the original commit (0e46b49de433), and adds
another patch on top with the following changes:
* The code in `updateRegDefsUses` now updates subranges when
subreg-liveness-tracking is enabled.
* When adding an implicit-def operand for the super-register,
the code in `reMaterializeTrivialDef` which tries to remove
undefined subranges should now take into account that the lanes
from the super-reg are no longer undefined.
Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com>
When emitting an inline stack probe loop, we can't use SUBri to
calculate the loop bound if it doesn't fit in a 32-bit (possibly
sign-extended) immediate.
Fixes#113218.
I am planning to add some optimization remarks to the
`PartiallyInlineLibCalls` pass. However, since this pass does not emit any
optimization remarks yet, I have to add the "infrastructure" for that first, which
is what this PR is about.
- Redefines `DXILAttribute` to denote a function attribute, compatible
to how it was define in DXC/LLVM 3.7
- Fix how `DXILAttribute` is emitted to be a struct of set attributes
instead of an "or" of the enums
- Implement the lowering of `DXILAttribute` to LLVM function attributes
in `DXILOpBuilder.cpp`. A custom mapping is defined.
- Audit all current ops to specify the correct attributes consistent
with DXC. This is done here to allow for testing.
- Update testcases in `llvm/test/CodeGen/DirectX` of all ops with
attributes to match that attributes are set
- Update testcases of ops that had previously incorrectly set attributes
to check there is no attributes set
- Defines `DXILProperty` to denote the other type of attributes from DXC
used to query properties.
- Emit `DXILProperty` as a struct of set attributes.
- Updates `DXIL.td` to specify applicable `DXILProperty`s on ops
Note: `DXILProperty` was referred to as 'queryable attributes' in design
discussion. Changed to property to allow for better expression in
`DXIL.td`
Resolves#114461Resolves#115912
.gfids$y contains a list of indirect calls for Control Flow Guard. This
wasn't working properly for ARM64EC: direct calls were being treated as
indirect calls. Make sure we correctly filter out direct calls.
This improves the protection from Control Flow Guard, and also fixes a
link error when using certain functions from oldnames.lib.
Fixes#99152
Tasks completed:
- Implement `reflect` in `clang/lib/Headers/hlsl/hlsl_intrinsics.h`
- Implement the `reflect` SPIR-V target built-in in
`clang/include/clang/Basic/BuiltinsSPIRV.td`
- Add a SPIR-V fast path in `clang/lib/Headers/hlsl/hlsl_detail.h` in
the form
```c++
#if (__has_builtin(__builtin_spirv_reflect))
return __builtin_spirv_reflect(...);
#else
return ...; // regular behavior
#endif
```
- Add codegen for the SPIR-V `reflect` built-in to
`EmitSPIRVBuiltinExpr` in `clang/lib/CodeGen/CGBuiltin.cpp`
- Add HLSL codegen tests to
`clang/test/CodeGenHLSL/builtins/reflect.hlsl`
- Add SPIR-V built-in codegen tests to
`clang/test/CodeGenSPIRV/Builtins/reflect.c`
- Add sema tests to `clang/test/SemaHLSL/BuiltIns/reflect-errors.hlsl`
- Add SPIR-V sema tests to
`clang/test/CodeGenSPIRV/Builtins/reflect-errors.c`
- Create the `int_spv_reflect` intrinsic in
`llvm/include/llvm/IR/IntrinsicsSPIRV.td`
- In `llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp` create the
`reflect` lowering and map it to `int_spv_reflect` in
`SPIRVInstructionSelector::selectIntrinsic`
- Create a SPIR-V backend test case in
`llvm/test/CodeGen/SPIRV/hlsl-intrinsics/reflect.ll`
Additional tasks completed:
- Implement sema check for the `reflect` SPIR-V built-in in
`clang/lib/Sema/SemaSPIRV.cpp`
- Required for HLSL codegen to work via the SPIR-V fast path, because
the types defined in `clang/include/clang/Basic/BuiltinsSPIRV.td` are
being overridden
- Create SPIR-V backend error test case in
`llvm/test/CodeGen/SPIRV/opencl/reflect-error.ll`
- Since `reflect` is only available in the GLSL extended instruction
set, using it in OpenCL should result in an error
Incomplete tasks:
- Create SPIR-V backend test case in
`llvm/test/CodeGen/SPIRV/opencl/reflect.ll`
- An OpenCL test is not applicable in this case because the [OpenCL
SPIR-V extended instruction
set](https://registry.khronos.org/SPIR-V/specs/unified1/OpenCL.ExtendedInstructionSet.100.html)
does not include a `reflect` function
This is a follow-up to #117878 and allows the usage of vrgather if the index
we are accessing in VT is constant and within bounds.
This patch replaces the previous behavior of bailing out if the length of the
search vector is greater than the vector of elements we are searching for.
Since matchSplatAsGather works on EXTRACT_VECTOR_ELT, and we know the index
from which the element is extracted, we only need to check if we are doing an
insert from a larger vector into a smaller one, in which we do an extract
instead.
Co-authored-by: Luke Lau luke_lau@icloud.com
Co-authored-by: Philip Reames preames@rivosinc.com
So that we can turn masked operations with all-ones masks into their
unmasked counterpart. Note: loads other than segmented ones had been
marked as masked.
For shuffle vector splats with undef lanes in the mask,
this was introducing real values. Filter out build_vector
results based on the undef elements in the mask.
This avoids AMDGPU test regressions in a future change.
test/CodeGen/X86/urem-seteq-illegal-types.ll looks worse
but I didn't investigate.
Adding SPIRV to LLVM_ALL_TARGETS
(https://github.com/llvm/llvm-project/pull/119653) revealed a series of
minor compilation problems and sanitizer complaints. This PR is to fix
debug-type-pointer.ll test case.
SVE2.2 introduces instructions with predicated forms with zeroing of
the inactive lanes. This allows in some cases to save a `movprfx` or
a `mov` instruction when emitting code for `_x` or `_z` variants of
intrinsics.
This patch adds support for emitting the zeroing forms of certain
`CLS`, `CLZ`, `CNT`, `CNOT`, and `NOT` instructions.
SVE2.2 introduces instructions with predicated forms with zeroing of
the inactive lanes. This allows in some cases to save a `movprfx` or
a `mov` instruction when emitting code for `_x` or `_z` variants of
intrinsics.
This patch adds support for emitting the zeroing forms of certain
`SCVTF`, and `UCVTF` instructions.
PR https://github.com/llvm/llvm-project/pull/118823 added a
DAG combine for extracting elements of a vector returned from
SETCC, however it doesn't correctly deal with the case where
the vector element type is not i1. In this case we have to
take account of the boolean contents, which are represented
differently between vectors and scalars. The code now
explicitly performs an inreg sign extend in order to get the
same result.
Fixes https://github.com/llvm/llvm-project/issues/121372
This started out as trying to combine bf16 fpround to BFCVT2
instructions, but ended up removing the aarch64.neon.nfcvt intrinsics in
favour of generating fpround instructions directly. This simplifies the
patterns and can lead to other optimizations. The BFCVT2 instruction is
adjusted to makes sure the types are valid, and a bfcvt2 is now
generated in more place. The old intrinsics are auto-upgraded to fptrunc
instructions too.
Commit 1eed46960c217f9480865702f06fb730c7521e61 added logic to
reassociate a (add (add x y) -c) operand to a CSEL instruction with a
comparison involving x and c (or a similar constant) in order to obtain
a common (SUBS x c) instruction.
This commit extends this logic to non-constants. In this way, we also
reassociate a (sub (add x y) z) operand of a CSEL instruction to
(add (sub x z) y) if the CSEL compares x and z, for example.
Alive proof: https://alive2.llvm.org/ce/z/SEVpR
This PR fixes operand order of `ILVOD.df` when lowering
`VECTOR_SHUFFLE`, the result was `<y[1], x[1]>` while it should be
`<x[1], y[1]>`.
* This PR is split from #123040.
Add some generated tests with every shuffle permutation
for relevant vector element types and sizes. Not sure if this
is going overboard with the number of tests. I pruned out the largest
cases (16 and 32-bit cases are impractically large), and there's
redundancy when testing the pointer cases (at least for SelectionDAG).
This uses inline assembly to produce sample values because of how the
ABI is lowered when using a function argument. Since we break all
arguments into 32-bit pieces, a shuffle never ends up forming. We
need separate handling to reconstruct shuffles in contexts involving
physical registers in ABI contexts.
I wrote a small tool to generate these, so I can easily change the
exact test body. Not sure if it's worth posting anywhere.
This is in preparation for making better use of v_pk_mov_b32,
v_mov_b64 and s_mov_b64 in shuffles.
This patch adds support for the next-generation arch15
CPU architecture to the SystemZ backend.
This includes:
- Basic support for the new processor and its features.
- Detection of arch15 as host processor.
- Assembler/disassembler support for new instructions.
- Exploitation of new instructions for code generation.
- New vector (signed|unsigned|bool) __int128 data types.
- New LLVM intrinsics for certain new instructions.
- Support for low-level builtins mapped to new LLVM intrinsics.
- New high-level intrinsics in vecintrin.h.
- Indicate support by defining __VEC__ == 10305.
Note: No currently available Z system supports the arch15
architecture. Once new systems become available, the
official system name will be added as supported -march name.
The current lowering for ptr addrspace(7) assumed that the instruction
selector can handle arbtrary LLVM types, which is not the case. Code
generation can't deal with
- Values that aren't 8, 16, 32, 64, 96, or 128 bits long
- Aggregates (this commit only handles arrays of scalars, more may come)
- Vectors of more than one byte
- 3-word values that aren't a vector of 3 32-bit values (for axample, a
<6 x half>)
This commit adds a buffer contents type legalizer that adds the needed
bitcasts, zero-extensions, and splits into subcompnents needed to
convert a load or store operation into one that can be successfully
lowered through code generation.
In the long run, some of the involved bitcasts (though potentially not
the buffer operation splitting) ought to be handled by the instruction
legalizer, but SelectionDAG makes this difficult.
It also takes advantage of the new `nuw` flag on `getelementptr` when
lowering GEPs to offset additions.
We don't currently plumb through `nsw` on GEPs since that should likely
be a separate change and would require declaring what we mean by "the
address" in the context of the GEP guarantees.
The indices of SGPR register pairs need to be 2-aligned and SGPR
quadruplets need to be 4-aligned. With this patch, we report an error
when inline asm register constraints specify a misaligned register
index, instead of silently dropping the specified index.
Fixes#123208
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
After a30e50fc, AMDGPUAAResult is being called in more situations where
BasicAA isn't sure. This exposed some regressions where NoAlias is being
incorrectly returned for two identical pointers.
The fix is to check the underlying objects for equality before returning
NoAlias.
Based on feedback from the clastb codegen PR, I'm refactoring basic codegen for the vector.extract.last.active intrinsic to lower to an ISD node in SelectionDAGBuilder then expand in LegalizeVectorOps, instead of doing everything in the builder.
The new ISD node (vector_find_last_active) only covers finding the index of the last active element of the mask, and extracting the element + handling passthru is left to existing ISD nodes.
Instruction `preld` is used to prefetch one cache-line of data from
memory in advance into the cache.
This commit allows it to be generated automatically.
If linker relaxation enabled, relaxable code sequence expanded from
pseudos should avoid being separated by instruction scheduling. This
commit tags scheduling boundary for them to avoid being scheduled.
(Except for `tls_le/tls_ie` and `call36/tail36`. Because `tls_le/tls_ie`
can be scheduled and have no influence to relax, `call36/tail36` are
expanded later in `LoongArchExpandPseudo` pass.)
A new mask target-flag is added to attach relax relocs to the relaxable
code sequence. (No need to add it for `tls_le` and `call36/tail36`
because we can simply add relax relocs for them according to their
relocs. But for other code sequence, such as `PCALA_{HI20/LO12}`, we
must use the mask flag, mainly because relax should not be added when
code model is large.)
Because of the new mask target-flag, get "direct" flags is necessary
when using their target-flags. In addition, code sequence after being
optimized by `MergeBaseOffset` pass may not relaxable any more, so the
relax "bitmask" flag should be removed.
