With ptrauth-calls, function pointers are supposed to be signed.
On Darwin that includes the TLS indirection accessor (`_tlv_get_addr`).
We simply sign it with the plain function-pointer schema (IA,0), which
lets us do a `blraaz` when calling it.
Note that this doesn't have any kind of diversity, even when function
pointer diversity is enabled in the frontend. On arm64e this accessor
is never signed that way, but the obvious alternative where this (or
another backend-generated) function pointer needs to be diversified
would need more than the "ptrauth-calls" attribute as it exists today.
Fix an issue in #97618 - if the two basic blocks involved are not
predecessor / successor to each other, treat the candidate as illegal
for critical edge splitting.
Closes#98477 (checked in test copied from its comment).
For DXIL which is based on llvm 3.7, max supported behavior flag for
module flags is 6.
The commit will check all module flags, for behavior flag > 6, change it
to 2 (Warning).
This is to fix the behavior flag part for #96912.
Summary:
This patch implements support for variadic functions for NVPTX targets.
The implementation here mainly follows what was done to implement it for
AMDGPU in https://github.com/llvm/llvm-project/pull/93362.
We change the NVPTX codegen to lower all variadic arguments to functions
by-value. This creates a flattened set of arguments that the IR lowering
pass converts into a struct with the proper alignment.
The behavior of this function was determined by iteratively checking
what the NVCC copmiler generates for its output. See examples like
https://godbolt.org/z/KavfTGY93. I have noted the main methods that
NVIDIA uses to lower variadic functions.
1. All arguments are passed in a pointer to aggregate.
2. The minimum alignment for a plain argument is 4 bytes.
3. Alignment is dictated by the underlying type
4. Structs are flattened and do not have their alignment changed.
5. NVPTX never passes any arguments indirectly, even very large ones.
This patch passes the tests in the `libc` project currently, including
support for `sprintf`.
This patch fixes a problem that existed before where in some situations
a `UCMP`/`SCMP` node which operated on 1-element vectors had a legal
result type (i.e. `v1i64` on AArch64), but illegal operands (i.e.
`v1i65`). This meant that operand scalarization was performed on the
node and the operands were changed to a legal scalar type, but the
result wasn't. This then led to `UCMP`/`SCMP` nodes with different
vector-ness of operands and result appearing in the SDAG. This patch
addresses this issue by fully scalarizing the `UCMP`/`SCMP` node and
then turning its result back into a 1-element vector using a
`SCALAR_TO_VECTOR` node.
It also adds several assertions to `SelectionDAG::getNode()` to avoid
this or a similar issue arising in the future. I wasn't sure if these
two changes are unrelated enough to warrant two small separate PRs, but
I'm happy to split this PR into two if that's deemed more appropriate.
Summary:
The target information needs to configure that the platform has a
maximum integer size of 64 in order for it to enable i128 support. The
motivation behind this patch is that the i128 libcalls seem to be the
only ones used by the NVPTX backend and it would be ideal to disable
those completely. That would allow LTO to optimize libcalls properly
after https://github.com/llvm/llvm-project/pull/98512.
- Add `MachineBlockFrequencyAnalysis`.
- Add `MachineBlockFrequencyPrinterPass`.
- Use `MachineBlockFrequencyInfoWrapperPass` in legacy pass manager.
- `LazyMachineBlockFrequencyInfo::print` is empty, drop it due to new
pass manager migration.
Reverts llvm/llvm-project#98295, which reverted llvm/llvm-project#97998
The failure in the "InOneWeekend" test of the HIP test suite on
clang-hip-vega20
(https://lab.llvm.org/buildbot/#/builders/123/builds/1498) seems to be
unrelated; I observed it (and a similar failure for the "TheNextWeek"
test in the same suite) intermittently on my system, with and without
the patch applied. (It occurred in 2 out of 50 repeated runs without the
patch and in 1 out of 50 runs with the patch.)
Most of the time when we coalesce and delete a vsetvli, we shrink the
LiveInterval of its AVL register now that there is one less use. However
there's one edge case we were missing where if we have two vsetvlis with
no users of vl or vtype in between, we coalesced a vsetvli without
shrinking it's AVL.
This fixes it by shrinking the LiveInterval whenever we delete a
vsetvli, and also makes the LiveIntervals consistent in-situ by not
removing the use before shrinking.
This fixes a -verify-machineinstrs assertion in an MIR test case I found
while investigating
https://github.com/llvm/llvm-project/pull/97264#issuecomment-2218036877.
I couldn't recreate this at the LLVM IR level, seemingly because
RISCVInsertVSETVLI will just avoid inserting extra vsetvlis that don't
need coalesced.
For synchronous unwind tables, the call frame information can be
slightly reduced by bundling the `.cfi_negate_ra_state` instruction with
other CFI instructions in the prolog, saving 1 byte per function used
for `DW_CFA_advance_loc`.
This was suggested in
[D156428](https://reviews.llvm.org/D156428#4554317).
Update availability information added in 1eb7f055d9a. exp10 is available
on iOS >= 7.0 and macOS >= 10.9. On all other platforms, it is available
on any version. Also drop the x86 check, as the availability only
depends on the OS version, not the target platform.
PR: https://github.com/llvm/llvm-project/pull/98542
Doing so allows one side to fold entirely into the mask applied to the
other recursive call (or a vmerge.vv at worst). This is a generalization
of the existing IsSelect case (both operands are selects), so I removed
that code in the process.
This actually started as an attempt to remove the IsSelect bit as I'd
thought it was fully redundant
with the recursive formulation, but digging into test deltas revealed
that we depended on that
to catch the majority of the identity cases, and that in turn we were
missing some cases where only RHS was an identity.
Across a basic-block we might have in i32 extract from a value that only
operates on upper bits (for example a sxtw). We can replace the COPY
with a new version skipping the sxtw.
Previously, if a patchpoint had a non-native integer type result, e.g.
i8 or i16 on AArch64, or some non-power-of-two wide integer type (e.g.
i29), the type legalizer would crash.
This adds intrinsics defined in ARM-software/acle#260
Doing this requires some changes to the GCS instruction definitions, as
these intrinsics make use of how some instructions don't modify the
input register when GCS is disabled, and they need to be correctly
marked with mayLoad/mayStore/hasSideEffects for instruction selection to
work.
Given an AVL that's computed from vlenb, if it's equal to VLMAX then we
can replace it with the VLMAX sentinel value.
The main motiviation is to be able to express an EVL of VLMAX in VP
intrinsics whilst emitting vsetvli a0, zero, so that we can replace
llvm.riscv.masked.strided.{load,store} with their VP counterparts.
This is done in RISCVVectorPeephole (previously RISCVFoldMasks, renamed
to account for the fact that it no longer just folds masks) instead of
SelectionDAG since there are multiple places places where VP nodes are
lowered that would have need to have been handled.
This also avoids doing it in RISCVInsertVSETVLI as it's much harder to
lookup the value of the AVL, and in RISCVVectorPeephole we can take
advantage of DeadMachineInstrElim to remove any leftover
PseudoReadVLENBs.
This PR improves type inference of operand presented by opaque pointers
and aggregate types:
* tries to restore original function return type for aggregate types so
that it's possible to deduce a correct type during emit-intrinsics step
(see llvm/test/CodeGen/SPIRV/SpecConstants/restore-spec-type.ll for the
reproducer of the previously existed issue when spirv-val found a
mismatch between object and ptr types in OpStore due to the incorrect
aggregate types tracing),
* explores untyped pointer operands of store to deduce correct pointee
types,
* creates an extension type to track pointee types from emit-intrinsics
step and further instead of direct and naive usage of TypePointerType
that led previously to crashes due to ban of creation of Value of
TypePointerType type,
* tracks instructions with uncomplete type information and tries to
improve their type info after pass calculated types for all machine
functions (it doesn't traverse a code but rather checks only those
instructions which were tracked as uncompleted),
* address more cases of removing unnecessary bitcasts (see, for example,
changes in test/CodeGen/SPIRV/transcoding/OpGenericCastToPtr.ll where
`CHECK-SPIRV-NEXT` in LIT checks show absence of unneeded bitcasts and
unmangled/mangled versions have proper typing now with equivalent type
info),
* address more cases of well known types or relations between types
within instructions (see, for example, atomic*.ll test cases and
Event-related test cases for improved SPIR-V code generated by the
Backend),
* fix the issue of removing unneeded ptrcast instructions in
pre-legalizer pass that led to creation of new assign-type instructions
with the same argument as source in ptrcast and caused errors in type
inference (the reproducer `complex.ll` test case is added to the PR).
This extension consists of 8 additional 16-bit compressed forms for
existing standard load/store opcodes.
These opcodes are found in some RISC-V microcontrollers from WCH /
Nanjing Qinheng Microelectronics.
As discussed in the Discourse forums, this uses incompatible extension
and opcode names vs the vendor binary toolchain. The chosen names
instead follow the conventions for other vendor extensions listed on the
"riscv-non-isa" project.
It's a similar patch as a214c521f8763b36dd400b89017f74ad5ae4b6c7 for
vp.stride.load. Some targets prefer pattern (vmv.v.x (load)) instead of
vlse with zero stride.
It's IR version of #97798.
This re-introduces the effective behaviour that was reverted in
7ad481e76c9bee5b9895ebfa0fdb52f31cb7de77.
This time we're not using the same mechanism, exposing another
reservation feature
that prevents only regalloc from using the register, but not for other
required uses
like ABIs.
This also fixes a consequent issue with reserving LR, which is that
frame lowering
was only adding live-in flags for non-reserved regs. This would cause
issues later
since the outliner needs accurate flags to determine when LR needs to be
preserved.
rdar://131313095
This also adds support for auth stubs on MachO using __DATA,__auth_ptr.
Some of the machinery for auth stubs is already implemented; this
generalizes that a bit to support MachO, and moves some of the shared
logic into MMIImpls.
In particular, this originally had an AuthStubInfo struct, but we no
longer need it beyond a single MCExpr. So this provides variants of
the symbol stub helper type declarations and functions for "expr
stubs", where a stub points at an arbitrary MCExpr, rather than
a simple MCSymbol (and a bit).
Summary:
These used the now-removed `nvptx` target. Because they used
`ptxas-verify` they would then fail since it's been removed. Update
these to 64-bit.
It's possible that we are unable to coerce all the incoming values of a
PHINode (A). Thus, we are unable to coerce the PHINode. In this
situation, we previously would add the PHINode back to the ValMap. This
would cause a problem is PhiNode (B) was a user of A. In this scenario,
if B has been coerced, we would hit an assert regarding the incompatible
type between the PHINode and its incoming value.
Deleting non-coerced PHINodes from the map, and propagating the removal
to users, resolves the issue.
Some cores implement an optimization for a strided load with an x0
stride, which results in fewer memory operations being performed then
implied by VL since all address are the same. It seems to be the case
that this is the case only for a minority of available implementations.
We know that sifive-x280 does, but sifive-p670 and spacemit-x60 both do
not.
(To be more precise, measurements on the x60 appear to indicate that a
stride of x0 has similar latency to a non-zero stride, and that both
are about twice a vleN.v. I'm taking this to mean the x0
case is not optimized.)
We had an existing flag by which a processor could opt out of this
assumption but no upstream users. Instead of adding this flag to the
p670 and x60, this patch reverses the default and adds the opt-in flag
only to the x280.
This avoids some cases where LSR produces results that lead to very poor
codegen. There's a chance we'll see minor degradations for some inputs
in the case that our metrics say the found solution is worse, but in
reality it's better than the starting point.
Per the review thread, at least one vendor has been enabling this by
defualt for some time and found overall it's an improvement. As such,
we'll enable by default and aim to fix any as-yet-unknown regressions
in-tree.
- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use `std::unique_ptr` instead of raw pointer for `LICalc`, so
destructor and default move constructor can handle it correctly.
This would be the last analysis required by `PHIElimination`.
So far branch protection, sign return address, guarded control stack
attributes are
only emitted as module flags to indicate the functions need to be
generated with
those features.
The problem is in case of an LTO build the module flags are merged with
the `min`
rule which means if one of the module is not build with sign return
address then the features
will be turned off for all functions. Due to the functions take the
branch-protection and
sign-return-address features from the module flags. The
sign-return-address is
function level option therefore it is expected functions from files that
is
compiled with -mbranch-protection=pac-ret to be protected.
The inliner might inline functions with different set of flags as it
doesn't consider
the module flags.
This patch adds the attributes to all functions and drops the checking
of the module flags
for the code generation.
Module flag is still used for generating the ELF markers.
Also drops the "true"/"false" values from the
branch-protection-enforcement,
branch-protection-pauth-lr, guarded-control-stack attributes as presence
of the
attribute means it is on absence means off and no other option.
Releand with test fixes.
Memcpy intrinsics with statically unknown loop sizes are lowered with
two load/store loops: one with access widths specified by the target,
and a residual loop that copies remaining bytes individually.
As the residual loop operates byte-wise, its accesses are only
1-aligned. However, we currently use the alignment that is optimal for
the first loop in both, which is unsound. With this patch, we use the
correct alignment in the residual loop.
The lowering of memcpy with a static size already handles alignments for
the residual correctly.
