Fixes asserting with windows-elf triples. Should fix regression
reported in https://github.com/llvm/llvm-project/pull/147225#issuecomment-3054190938
I'm not sure this is a valid triple, but I'm guessing the MCJIT usage
is an accident. This does change the behavior from trying to use WinEH
to DwarfCFI; however the backend crashes with WinEH so I'm assuming following
ELF is the more correct option.
On most operating systems, the x16 and x17 registers are not special,
so there is no benefit, and only a code size cost, to constraining AUT to
only using them. Therefore, adjust the backend to only use the AUT pseudo
(renamed AUTx16x17 for clarity) on Darwin platforms. All other platforms
use an unconstrained variant of the pseudo, AUTxMxN, for selection.
Reviewers: ahmedbougacha, kovdan01, atrosinenko
Reviewed By: atrosinenko
Pull Request: https://github.com/llvm/llvm-project/pull/132857
When true16 is enabled, isel start to emit sgpr_lo16 register when a
trunc/sext i16/i32 is generated, or a salu32 is used by vgpr16 or vice
versa. And this causes a problem as sgpr_lo16 is not fully supported in
the pipeline.
True16 mode works fine in -O3 mode since folding pass remove sgpr_lo16
from the pipeline. However it hit a problem in -O0 mode as folding pass
is skipped.
This patch did:
1. stop emitting sgpr_lo16 from isel
2. update codegen pattern to split uniformed/divergent pattern for
i16/i32 conversion
3. update fix-sgpr-copy pass to address legalization requirement in
true16 mode, update fix-sgpr-copies-f16-true16.mir
test to include all possible combinations
This patch is tested with cts and downstream repo with -O0 testing
Anatoly Trosinenko found that when hasSideEffect was set to 0 in the
definition of LOADgotAUTH, MultiSource/Benchmarks/Ptrdist/ks/ks test
from llvm-test-suite started to crash. The issue was traced down to
MachineLICM pass placing LOADgotAUTH right after an unrelated copy to
x16 like rewriting this code:
````
bb.0:
renamable $x16 = COPY renamable $x12
B %bb.1
bb.1:
...
/* use $x16 */
...
renamable $x20 = LOADgotAUTH target-flags(aarch64-got) @some_variable, implicit-def dead $x16, implicit-def dead $x17, implicit-def dead $nzcv
/* use $x20 */
...
````
like the following:
````
bb.0:
renamable $x16 = COPY renamable $x12
renamable $x20 = LOADgotAUTH target-flags(aarch64-got) @some_variable, implicit-def dead $x16, implicit-def dead $x17, implicit-def dead $nzcv
B %bb.1
bb.1:
...
/* use $x16 */
...
/* use $x20 */
...
```
The issue was caused by inconsistent logic between implicit and explicit
operand definitions, where the implicit side was incorrectly skipping
checking RUDefs for dead operands, leading to RuledOut not being set
for the X16 operand.
Because there isn't really a semantic difference between implicit and
explicit operands at this point, let's remove the isImplicit check and
adjust the logic to do the same thing in both cases:
- For implicit operands, we now check and update RUDefs in the same way
as explicit operands.
- For explicit operands, we now allow dead operands to be skipped.
Reviewers: arsenm, s-barannikov, atrosinenko
Reviewed By: arsenm, s-barannikov
Pull Request: https://github.com/llvm/llvm-project/pull/147624
This pr resolves some discrepancies in verification during `validate` in
`DXILRootSignature.cpp`.
Note: we don't add a backend test for version 1.0 flag values because it
treats the struct as though there is no flags value. However, this will
be used when we use the verifications in the frontend.
- Updates `verifyDescriptorFlag` to check for valid flags based on
version, as reflected [here](https://github.com/llvm/wg-hlsl/pull/297)
- Add test to demonstrate updated flag verifications
- Adds `verifyNumDescriptors` to the validation of `DescriptorRange`s
- Add a test to demonstrate `numDescriptors` verification
- Updates a number of tests that mistakenly had an invalid
`numDescriptors` specified
Resolves: https://github.com/llvm/llvm-project/issues/147107
This change cleans up DAG-to-DAG instruction selection around FTZ and
SETP comparison mode. Largely these changes do not impact functionality
though support for `{sin.cos}.approx.ftz.f32` is added.
Callsite offsets will help map addresses to the right position in the
basic block (before or after a callsite).
This PR also bumps the BBAddrMap version to 3.
The encoding/decoding ability is already pushed upstream
8d7a8fcc3ab9f6d4c4a7e4312876fe94ed3d6c4f.
Introduces saturated truncate instructions to Global ISel:
G_TRUNC_SSAT_S, G_TRUNC_SSAT_U, G_TRUNC_USAT_U. These were previously
introduced to SDAG to reduce redundant code.
The patch only initially introduces the instruction, a later patch will
follow to add combines and legalization for each instruction.
For the fixed vector cases, we already support this, but the
deinterleave intrinsic cases (primary used by scalable vectors) didn't.
Supporting it requires plumbing through the Factor separately from the
extracts, as there can now be fewer extracts than the Factor. Note that
the fixed vector path handles this slightly differently - it uses the
shuffle and indices scheme to achieve the same thing.
When performing a 64-bit sra of a negative value with a shift range from
[32-63], create the hi-half with a move of -1.
Alive verification: https://alive2.llvm.org/ce/z/kXd7Ac
Also, preserve exact flag. Alive verification:
https://alive2.llvm.org/ce/z/L86tXf.
---------
Signed-off-by: John Lu <John.Lu@amd.com>
Normally, pac-ret hardening is emitted as part of function prologues and
epilogues, thus it is affected by the shrink-wrapping optimization.
As protecting LR when it is spilled to the stack is already handled by
regular -mbranch-protection=pac-ret option, it is reasonable to assume
that pac-ret+leaf option means the user wants to apply pac-ret hardening
to as much code as possible. For that reason, if pac-ret+leaf hardening
mode is requested, this patch moves the emission of PAUTH_PROLOGUE (or
PAUTH_EPILOGUE) pseudos from emitPrologue (emitEpilogue) methods of the
AArch64FrameLowering class to processFunctionBeforeFrameIndicesReplaced.
This change does not currently affect targets that emit WinCFI unwind
information.
This commit only affects where LR is signed and authenticated, but does
not otherwise prevents the shrink-wrapping optimization. Moreover,
without "+leaf" modifier PAUTH_(PROLOGUE|EPILOGUE) pseudos respect the
shrink-wrapping optimization just as any other prologue/epilogue code.
This adds some test cases for v2 and v3 half vector fadd reductions. In doing
so it appears that GlobalISel was having trouble lowering the smaller vector
sizes. Add some basic handling by widening to a power2 or scalarizing if
necessary. Larger vectors are still having problems as FewerElements currently
requires the number of elements to be modula the vector length.
The patch adds intrinsics and lowering logic for GlobalSize,
GlobalOffset, SubgroupMaxSize, NumWorkgroups, WorkgroupSize,
WorkgroupId, LocalInvocationId, GlobalInvocationId, SubgroupSize,
NumSubgroups, SubgroupId and SubgroupLocalInvocationId SPIR-V builtins.
The patch also extend spv_thread_id, spv_group_id and
spv_thread_id_in_group to return anyint rather than i32. This allows the
intrinsics to support the opencl environment.
For each of the intrinsics, new clang builtins were added as well as a
binding for the SPIR-V "friendly" format. The original format doesn't
define such binding (uses global variables) but it is not possible to
express the Input SC which is normally required by the environement
specs, and using builtin functions is the most usual approach for other
backend and programming models.
Extend lowerVectorXRINT to also do a FP_EXTEND_VL when the source
element type is [b]f16, and wire up this custom-promote. Updating the
cost-model to not give these an invalid cost is left to a companion
patch.
The combine was added to find a get.active.lane.mask used by two
extract subvectors and try to replace it with the paired whilelo
instruction. This extends the combine to cover cases where there
are more than two extracts.
We were previously limited to abs(sub(zext(),zext()) patterns, but add
handling for a number of other abdu patterns until a topological sorted
dag allows us to rely on a ABDU node having already been created.
Now that we don't just match zext() sources, I've generalised the
createPSADBW helper to explicitly zext/truncate to the expected vXi8
source type - it still assumes the sources are correct for a PSADBW
node.
Fixes#143456
Emit a context error and delete the instruction. This
allows removing the AMDGPU hack where some atomic libcalls
are falsely added. NVPTX also later copied the same hack,
so remove it there too.
For now just emit the generic error, which is not good. It's
missing any useful context information (despite taking the instruction).
It's also confusing in the failed atomicrmw case, since it's reporting
failure at the intermediate failed cmpxchg instead of the original
atomicrmw.
Use `emitValueToAlignment` as the section does not contain code.
`emitCodeAlignment` would lead to ALIGN relocations on RISC-V and
LoongArch with linker relaxation.
In addition, change the alignment to wordsize, sufficient for the
runtime requirement (`XRayFunctionSledIndex`).
Related to #147322
A disjoint OR can be converted to XOR. And a XOR+NOT is XNOR. Idea
taken from #147279.
I changed the existing xnor pattern to have the not on the outside
instead of the inside. These are equivalent for xor since xor is
associative. Tablegen was already generating multiple variants
of the isel pattern using associativity.
There are some issues here. The disjoint flag isn't preserved
through type legalization. I was hoping we could recover it
manually for the masked merge cases, but that doesn't work either.
When vectorizing a loop with a fixed-order recurrence we use a splice,
which gets lowered to a vslidedown and vslideup pair.
However with the way we lower it today we end up with extra vl toggles
in the loop, especially with EVL tail folding, e.g:
.LBB0_5: # %vector.body
# =>This Inner Loop Header: Depth=1
sub a5, a2, a3
sh2add a6, a3, a1
zext.w a7, a4
vsetvli a4, a5, e8, mf2, ta, ma
vle32.v v10, (a6)
addi a7, a7, -1
vsetivli zero, 1, e32, m2, ta, ma
vslidedown.vx v8, v8, a7
sh2add a6, a3, a0
vsetvli zero, a5, e32, m2, ta, ma
vslideup.vi v8, v10, 1
vadd.vv v8, v10, v8
add a3, a3, a4
vse32.v v8, (a6)
vmv2r.v v8, v10
bne a3, a2, .LBB0_5
Because the vslideup overwrites all but UpOffset elements from the
vslidedown, we currently set the vslidedown's AVL to said offset.
But in the vslideup we use either VLMAX or the EVL which causes a
toggle.
This increases the AVL of the vslidedown so it matches vslideup, even if
the extra elements are overridden, to avoid the toggle.
A new tuning feature +vl-dependent-latency has been added which keeps
the old behaviour for microarchitectures that dynamically dispatch uops
based on vl, e.g. sifive-x280.
+vl-dependent-latency can be reused for the recently proposed Ovlt
optimization directive if/when it's ratified:
https://lists.riscv.org/g/tech-privileged/message/2487
If we wanted to aggressively optimise for vl at the expense of
introducing more toggles we could probably look at doing this in
RISCVVLOptimizer.
This PR doesn't really need to change anything else, since the
instruction is
already supported, but just not tested.
Co-authored-by: Mekhanoshin, Stanislav <Stanislav.Mekhanoshin@amd.com>
Fixes#147394
References DXC for the implementation logic:
d751c827ed/lib/HLSL/DxilPreparePasses.cpp (L693-L699)
If DXIL Version < 1.6 then replace lifetime intrinsics with stores
- For validator version >= 1.6, store an undef
- For validator version < 1.6, store zeros
else keep the lifetime intrinsics in the DXIL.
After this PR, the number of DML shaders failing validation due to
#146974 is reduced from 157 to 50.
This applies the same check as llvm.vector.splice which checks that the immediate is in the range [-VL, VL-1] where VL is the minimum vector length. If vscale_range is available, the lower bound is used to increase the known minimum vector length for this check. This ensures the immediate is in range for any possible value of vscale that satisfies the vscale_range.
Fixes#146973
When an object with alignment requirements is placed on the stack, this
causes a stack realignment which causes AArch64 to use x19 to refer to
objects on the stack as there may be a gap between local variables and
the Stack Pointer. This causes issues with the MSVC C++ exception
personality as the offset to the catch object recorded in the handler
table no longer matches the object being used in the catch block itself.
The fix for this is to place catch objects into the fixed object area.
Re-land #146582 now that the Flang bugs have been fixed.
There is no way in Arm64 Windows to indicate that a given function has
used the Frame Pointer as a General Purpose Register, as such stack
walks will always assume that the frame chain is valid and will follow
whatever value has been saved for the Frame Pointer (even if it is
pointing to data, etc.).
This change makes the Frame Pointer always reserved when building for
Arm64 Windows to avoid this issue.
We will be updating the official Windows ABI documentation to reflect
this requirement, and I will provide a link once it's available.
