Weaken barriers for atomic ops to the form that's just enough to enforce
memory model constraints.
In particular, we try to avoid emitting expensive #StoreLoad barriers
whenever possible.
The barriers emitted conform to V9's RMO and V8's PSO memory model, and
is compatible with GCC's lowering.
A quick test with `pgbench` on a T4-1 shows some small (up to about 4%),
but consistent speedup.
Do not move meta instructions like `FAKE_USE`/`@llvm.fake.use` into
delay slots, as they don't correspond to real machine instructions.
This should fix crashes when compiling with, for example, `clang -Og`.
Pad argument registers to preserve overaligned structs in LLVM IR.
Additionally, since i128 values will be lowered as split i64 pairs in
the backend, correctly set the alignment of such arguments as 16 bytes.
This should make clang compliant with the ABI specification and fix
https://github.com/llvm/llvm-project/issues/144709.
`half` currently uses the default legalization of promoting to a `f32`;
however, this implementation implements math in a way that results in
incorrect rounding. Switch to the soft promote implementation, which
does not have this problem.
The SPARC ABI does not specify a `_Float16` type, so there is no concern
with keeping interface compatibility.
Fixes the SPARC part of
https://github.com/llvm/llvm-project/issues/97975
Fixes the SPARC part of
https://github.com/llvm/llvm-project/issues/97981
Many backends are missing either all tests for lrint, or specifically
those for f16, which currently crashes for `softPromoteHalf` targets.
For a number of popular backends, do the following:
* Ensure f16, f32, f64, and f128 are all covered
* Ensure both a 32- and 64-bit target are tested, if relevant
* Add `nounwind` to clean up CFI output
* Add a test covering the above if one did not exist
* Always specify the integer type in intrinsic calls
There are quite a few FIXMEs here, especially for `f16`, but much of
this will be resolved in the near future.
This argument is the number of bytes to adjust the stack by for the
duration of the call. In most cases, PEI is able to eliminate the
corresponding call frame pseudos, folding them into the initial stack
frame allocation (rounded up to stack alignment), where it just ends up
allocating more space than needed. However, in the rare case where this
cannot be done, e.g. due to the use of a dynamic alloca, the 1 byte
stack adjustment persists and results in a misaligned stack for the
duration of the call. This has been the case ever since TLS support was
added in cb1dca602c43 ("[Sparc] Add support for TLS in sparc."), and I
can only assume that 1 was used erroneously thinking that it is the
number of arguments (as there is 1 register argument for the call), not
the number of bytes for on-stack arguments.
Fixes: https://github.com/llvm/llvm-project/issues/149808
7dce16f69dc3e26cb74d5ad38b0648a6f47f9640 removed a libcall for
STACKPROTECTOR_CHECK_FAIL from OpenBSD but added no tests.
Add a basic test copied from RISCV into all the backends on
the OpenBSD page of supported architectures before I potentially
break in in RuntimeLibcalls refactoring.
Some SPARC ISA levels and/or extensions are defined in a way such that
the availability of it implies the availability of other, more fundamental
ISA features (for example, targeting 64-bit environment implies that
V9 instructions are available).
Properly set those in the TableGen definitions.
Fixes https://github.com/llvm/llvm-project/issues/142388.
* adjustFixupValue is called even when a R_SPARC_HIX22/R_SPARC_LOX10
relocation is generated. This will be fixed shortly.
* Enhanced the %h44 test to show that we don't check overflow.
* Test R_SPARC_DISP32 in .gcc_except_table and .eh_frame . The original
support did not test -filetype=obj output.
The code below the removed check looks generic enough to support
arbitrary integer widths. This change helps 32-bit targets avoid
expensive expansion/libcalls in the case of zero input.
Pull Request: https://github.com/llvm/llvm-project/pull/137197
CTTZ can be implemented in terms of CTLZ, for which there's a native
instruction in VIS3.
Promote i32 CTTZ in that case so that the native instruction gets used.
Reviewers: rorth, brad0, s-barannikov
Reviewed By: s-barannikov
Pull Request: https://github.com/llvm/llvm-project/pull/135894
Update the tests to reflect the change in instruction ordering.
Otherwise there are no changes from the previous commit.
This reverts commit 5e9650ec2deb2f2bb6d5ad28e83bb6cd3c4189e4.
`weightCalcHelper` is responsible for adding hints to MRI. Prior to this
PR, we fell back on register ID as the last tie breaker for sorting
hints. However, there is an opportunity to add an additional sorting
characteristic: whether or not a register is a callee-saved-register.
I thought of this idea because I saw that `AllocationOrder::create`
calls `RegisterClassInfo::getOrder`, which returns a list of registers
such that the registers which alias callee-saved-registers come last.
From this, I conclude that the register allocator prefers an order such
that callee-saved-registers are allocated after
non-callee-saved-registers to avoid having to spill the CSR.
This sorting characteristic occurs only as a tie breaker to the Weight
calculation. This is a good idea since the weight calculation is pretty
complex and I'm sure it is a pretty stable metric. I think its pretty
reasonable to agree that whether a register is callee-saved or not is a
better tie breaker than register ID. I think this is evident by the test
diff, since the changes all seem to have no impact or improve the
register allocation.
The standard libcalls for half to float and float to half conversion are
__extendhfsf2 and __truncsfhf2. However, LLVM currently uses
__gnu_h2f_ieee and __gnu_f2h_ieee instead. As far as I can tell, these
libcalls are an ARM-ism and only provided by libgcc on that platform.
compiler-rt always provides both libcalls.
Use the standard libcalls by default, and only use the __gnu libcalls on
ARM.
This fixes the handling of subregister extract copies. This
will allow AMDGPU to remove its implementation of
shouldRewriteCopySrc, which exists as a 10 year old workaround
to this bug. peephole-opt-fold-reg-sequence-subreg.mir will
show the expected improvement once the custom implementation
is removed.
The copy coalescing processing here is overly abstracted
from what's actually happening. Previously when visiting
coalescable copy-like instructions, we would parse the
sources one at a time and then pass the def of the root
instruction into findNextSource. This means that the
first thing the new ValueTracker constructed would do
is getVRegDef to find the instruction we are currently
processing. This adds an unnecessary step, placing
a useless entry in the RewriteMap, and required skipping
the no-op case where getNewSource would return the original
source operand. This was a problem since in the case
of a subregister extract, shouldRewriteCopySource would always
say that it is useful to rewrite and the use-def chain walk
would abort, returning the original operand. Move the process
to start looking at the source operand to begin with.
This does not fix the confused handling in the uncoalescable
copy case which is proving to be more difficult. Some currently
handled cases have multiple defs from a single source, and other
handled cases have 0 input operands. It would be simpler if
this was implemented with isCopyLikeInstr, rather than guessing
at the operand structure as it does now.
There are some improvements and some regressions. The
regressions appear to be downstream issues for the most part. One
of the uglier regressions is in PPC, where a sequence of insert_subrgs
is used to build registers. I opened #125502 to use reg_sequence instead,
which may help.
The worst regression is an absurd SPARC testcase using a <251 x fp128>,
which uses a very long chain of insert_subregs.
We need improved subregister handling locally in PeepholeOptimizer,
and other pasess like MachineCSE to fix some of the other regressions.
We should handle subregister composes and folding more indexes
into insert_subreg and reg_sequence.
We have two forceExpandWideMUL functions. One takes the low and high
half of 2 inputs and calculates the low and high half of their product.
This does not calculate the full 2x width product.
The other signature takes 2 inputs and calculates the low and high half
of their full 2x width product. Previously it did this by sign/zero
extending the inputs to create the high bits and then calling the other
function.
We can instead copy the algorithm from the other function and use the
Signed flag to determine whether we should do SRA or SRL. This avoids
the need to multiply the high part of the inputs and add them to the
high half of the result. This improves the generated code for signed
multiplication.
This should improve the performance of #123262. I don't know yet how
close we will get to gcc.
Similar to 806761a7629df268c8aed49657aeccffa6bca449
-mtriple= specifies the full target triple while -march= merely sets the
architecture part of the default target triple (e.g. Windows, macOS),
leaving a target triple which may not make sense.
Therefore, -march= is error-prone and not recommended for tests without a target
triple. The issue has been benign as we recognize sparc*-apple-darwin as ELF instead
of rejecting it outrightly.
SPARC ABI doesn't use stack realignment, so let LLVM know about it in
`SparcFrameLowering`. This has the side effect of making all overaligned
allocations go through `LowerDYNAMIC_STACKALLOC`, so implement the
missing logic there too for overaligned allocations.
This makes the SPARC backend not crash on overaligned `alloca`s and fix
https://github.com/llvm/llvm-project/issues/89569.
The previous behavior could be harmful in some edge cases, such as
emitting a call to `fma()` in the `fma()` implementation itself.
Do this by just being more accurate in `isFMAFasterThanFMulAndFAdd()`.
This was already done for PowerPC; this commit just extends that to Arm,
z/Arch, and x86. MIPS and SPARC already got it right, but I added tests
for them too, for good measure.
Note: I don't have commit access.
Some targets (e.g. PPC and Hexagon) already did this. I think it's best
to do this consistently so that frontend authors don't run into
inconsistent results when they emit `naked` functions. For example, in
Zig, we had to change our emit code to also set `frame-pointer=none` to
get reliable results across targets.
Note: I don't have commit access.
This patch adds a pass that provides workarounds for the errata
described in GRLIB-TN-0009, GRLIB-TN-0010, GRLIB-TN-0011, GRLIB-TN-0012,
and GRLIB-TN-0013, that are applicable to the GR712RC and UT700. The
documents are available for download from here:
https://www.gaisler.com/index.php/information/app-tech-notes
The pass will detect certain sensitive instruction sequences and prevent
them from occurring by inserting NOP instruction. Below is an overview
of each of the workarounds. A similar implementation is available in
GCC.
GRLIB-TN-0009:
* Insert NOPs to prevent the sequence (stb/sth/st/stf) -> (single
non-store/load instruction) -> (any store)
* Insert NOPs to prevent the sequence (std/stdf) -> (any store)
GRLIB-TN-0010:
* Insert a NOP between load instruction and atomic instruction (swap and
casa).
* Insert a NOP at branch target if load in delay slot and atomic
instruction at branch target.
* Do not allow functions to begin with atomic instruction.
GRLIB-TN-0011:
* Insert .p2align 4 before atomic instructions (swap and casa).
GRLIB-TN-0012:
* Place a NOP at the branch target of an integer branch if it is a
floating-point operation or a floating-point branch.
GRLIB-TN-0013:
* Prevent (div/sqrt) instructions in the delay slot.
* Insert NOPs to prevent the sequence (div/sqrt) -> (two or three
floating point operations or loads) -> (div/sqrt).
* Do not insert NOPs if any of the floating point operations have a
dependency on the destination register of the first (div/sqrt).
* Do not insert NOPs if one of the floating point operations is a
(div/sqrt).
* Insert NOPs to prevent (div/sqrt) followed by a branch.
This adds support for using the L and H argument modifiers for twinword
operands in inline asm code, such as in:
```
%1 = tail call i64 asm sideeffect "rd %pc, ${0:L} ; srlx ${0:L}, 32, ${0:H}", "={o4}"()
```
This is needed by the Linux kernel.
When in 32-bit mode, the backend doesn't currently implement 64-bit
atomics, even though the hardware is capable if you have specified a V9
CPU. Thus, limit the width to 32-bit, for now, leaving behind a TODO.
This fixes a regression triggered by PR #73176.
This adds support for marking arbitrary general purpose registers -
except for those with special purpose (G0, I6-I7, O6-O7) - as reserved,
as needed by some software like the Linux kernel.
On 64-bit target, prefer using RDPC over CALL to get the value of %pc.
This is faster on modern processors (Niagara T1 and newer) and avoids
polluting the processor's predictor state.
The old behavior of using a fake CALL is still done when tuning for
classic UltraSPARC processors, since RDPC is much slower there.
A quick pgbench test on a SPARC T4 shows about 2% speedup on SELECT
loads, and about 7% speedup on INSERT/UPDATE loads.
Reviewed By: @s-barannikov
Github PR: https://github.com/llvm/llvm-project/pull/78280
