The libz compression library on SystemZ by default makes use of the
platform's hardware-accelerated compression facility. This is much
faster than the regular software implementation, but often results in
slightly different outputs. This causes failures with the
compressed-debug-level test case.
To fix this, set the DFLTCC environment variable to zero while running
tests on SystemZ, which prevents use of hardware compression and falls
back to the software implementation.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D149273
Before this patch, export entries with empy RVA were displayed in the output. In some cases, when the module had exports with sparse ordinals, `llvm-objdump` used to print a lot of `0 0` lines.
We now skip over these empty entries in the output, just as `dumpbin` or binutils `objdump` does.
Differential Revision: https://reviews.llvm.org/D149610
The `valueDelta` map was used to calculate the symbol value deltas from
the previous iteration. Since the symbol values themselves are also
updated every iteration, the following invariant holds:
```
sa[i].offset == sa[i].d->value + valueDelta[sa[i].d]
```
Note that `sa[i].offset` contains the original value of `sa[i].d` and is
never changed.
This means that the current way of updating symbol values can be
rewritten to not need the `valueDelta` map:
```
sa[i].d->value -= delta - valueDelta.find(sa[i].d)->second;
<=> (replace invariant)
sa[i].d->value -= delta - (sa[i].offset - sa[i].d->value);
<=>
sa[i].d->value = sa[i].d->value - (delta - (sa[i].offset - sa[i].d->value));
<=>
sa[i].d->value = sa[i].d->value - delta + sa[i].offset - sa[i].d->value;
<=>
sa[i].d->value = sa[i].offset - delta;
```
This patch implements this simplification. I believe this improves the
readability of the code as it took me quite some time to understand the
use of `valueDelta`. It might also have a slight performance benefit as
it removes one iteration over all relocations every relax iteration.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D149735
OutputSection::checkDynRelAddends() incorrectly reports an internal
linker error for large addends on 32-bit targets. This is caused by the
lack of sign extension in DynamicReloc::computeAddend() for 32-bit
addends.
Differential Revision: https://reviews.llvm.org/D149347
This reverts commit c117c2c8ba4afd45a006043ec6dd858652b2ffcc.
itaniumDemangle calls std::strlen with the results of
std::string_view::data() which may not be NUL-terminated. This causes
lld/test/wasm/why-extract.s to fail when "expensive checks" are enabled
via -DLLVM_ENABLE_EXPENSIVE_CHECKS=ON. See D149675 for further
discussion. Back this out until the individual demanglers are converted
to use std::string_view.
As suggested by @erichkeane in
https://reviews.llvm.org/D141451#inline-1429549
There's potential for a lot more cleanups around these APIs. This is
just a start.
Callers need to be more careful about sub-expressions producing strings
that don't outlast the expression using ``llvm::demangle``. Add a
release note.
Reviewed By: MaskRay, #lld-macho
Differential Revision: https://reviews.llvm.org/D149104
That patch adds a check for threadIndex being used with only threads
created by ThreadPoolExecutor. This helps catch two types of errors:
1. If a thread is created not by ThreadPoolExecutor its index may clash
with the index of another thread. Using threadIndex, in that case, may
lead to a data race.
2. Index of the main thread(threadIndex == 0) currently clashes with
the index of thread0 in ThreadPoolExecutor threads. That may lead
to a data race if main thread and thread0 are executed concurrently.
This patch allows execution tasks on the main thread only in case
parallel::strategy.ThreadsRequested == 1. In all other cases,
assertions check that threadIndex != UINT_MAX(i.e. that task
is executed on a thread created by ThreadPoolExecutor).
Differential Revision: https://reviews.llvm.org/D148916
In a link map, the input section name gives more information. See the updated
merge-entsize.s for an example. The output file is unchanged.
Compiler generated input sections with the SHF_MERGE flag have names such as
.rodata.str1.1 and .rodata.cstN, and are not affected by -fdata-sections.
Reviewed By: peter.smith
Differential Revision: https://reviews.llvm.org/D149466
Currently, most Mach-O tests fail when executed on a big-endian host. This is
because the Mach-O back-end does not perform the necessary byte swaps when
accessing the (little-endian) binary file format.
For now, simply consider all Mach-O tests unsupported on big-endian hosts,
to enable running the test suite at all on such hosts.
Reviewed by: oontvoo
Differential Revision: https://reviews.llvm.org/D149270
Relocations R_AVR_LO8_LDI_GS/R_AVR_HI8_LDI_GS (indirect calls
via function pointers) only cover range 128KiB. They are
equivalent to R_AVR_LO8_LDI_PM/R_AVR_HI8_LDI_PM within this
range.
But for function addresses beyond this range, GNU-ld emits
trampolines. And this patch implements corresponding thunks
for them in lld.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D147364
The libz compression library on SystemZ by default makes use of the
platform's hardware-accelerated compression facility. This is much
faster than the regular software implementation, but often results in
slightly different outputs. This causes failures with the
compressed-debug-level test case.
To fix this, run this test while setting the DFLTCC environment
variable to zero, which prevents use of hardware compression and falls
back to the software implementation. (This should not have any effect
on other platforms.)
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D149273
When running the LLD test suite on a big-endian host, the
COFF/pdb-framedata.yaml test case currently fails.
As it turns out, this is because code in DebugSHandler::finish
intended to relocate RvaStart entries of FDO records does not
work correctly when compiled for a big-endian host.
Fixed by always reading file data in little-endian mode.
Reviewed By: aganea
Differential Revision: https://reviews.llvm.org/D149268
--remap-inputs-file= can be specified multiple times, each naming a
remap file that contains `from-glob=to-file` lines or `#`-led comments.
('=' is used a separator a la -fdebug-prefix-map=)
--remap-inputs-file= can be used to:
* replace an input file. E.g. `"*/libz.so=exp/libz.so"` can replace a resolved
`-lz` without updating the input file list or (if used) a response file.
When debugging an application where a bug is isolated to one single
input file, this option gives a convenient way to test fixes.
* remove an input file with `/dev/null` (changed to `NUL` on Windows), e.g.
`"a.o=/dev/null"`. A build system may add unneeded dependencies.
This option gives a convenient way to test the result removing some inputs.
`--remap-inputs=a.o=aa.o` can be specified to provide one pattern without using
an extra file.
(bash/zsh process substitution is handy for specifying a pattern without using
a remap file, e.g. `--remap-inputs-file=<(printf 'a.o=aa.o')`, but it may be
unavailable in some systems. An extra file can be inconvenient for a build
system.)
Exact patterns are tested before wildcard patterns. In case of a tie, the first
patterns wins. This is an implementation detail that users should not rely on.
Co-authored-by: Marco Elver <elver@google.com>
Link: https://discourse.llvm.org/t/rfc-support-exclude-inputs/70070
Reviewed By: melver, peter.smith
Differential Revision: https://reviews.llvm.org/D148859
This patch allows to specify that some part of tasks should be
done in sequential order. It makes it possible to not use
condition operator for separating sequential tasks:
TaskGroup tg;
for () {
if(condition) ==> tg.spawn([](){fn();}, condition)
fn();
else
tg.spawn([](){fn();});
}
It also prevents execution on main thread. Which allows adding
checks for getThreadIndex() function discussed in D142318.
The patch also replaces std::stack with std::deque in the
ThreadPoolExecutor to have natural execution order in case
(parallel::strategy.ThreadsRequested == 1).
Differential Revision: https://reviews.llvm.org/D148728
MSVC link.exe added this flag and MS STL started using this flag in
.drectve [1] when compiling with Clang with asan enabled, as reported
on https://github.com/llvm/llvm-project/issues/56300. This causes issues
with lld-link because it rejects any unknown flags in .drective sections.
As dc07867dc9991c982bd3441da19d6fcc16ea54d6 noted that, when using Clang
as the driver it explicitly passes the proper asan libraries. Therefore
it should be acceptable to ignore this flag in lld-link to at least
unbreak building with clang-cl and linking with lld-link.
[1]: faaf094ee1/stl/inc/__msvc_sanitizer_annotate_container.hpp (L35)
Differential Revision: https://reviews.llvm.org/D149023
The AArch64 branch immediate instruction has a 128MiB range. This
makes it suitable for use a short range thunk in the same way as
short thunks are implemented in Arm and PPC. This patch adds
support for short range thunks to AArch64.
Adding short range thunk support should mean that OutputSections
can grow to nearly 256 MiB in size without needing long-range
indirect branches.
Differential Revision: https://reviews.llvm.org/D148701
Adding BTI to those PLT's which accessed with by a range extension thunk due to those preform an indirect call.
Fixes: #62140
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D148704
In particular, make it `foo.a(foo.o)$ARCHIVE_OFFSET`. The goal is to
make it more similar to both ld64 implementation, which uses the
`foo.a(foo.o)$MODULE_ID` format. We dump some of these names in LTO
code, so matching ld64's format is helpful. This format is also more
similar to LLD-ELF's, which is `foo.a(foo.o at $ARCHIVE_OFFSET)`.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D148828
When --threads= is unspecified, we set it to
`parallel::strategy.compute_thread_count()`, which uses
sched_getaffinity (Linux)/cpuset_getaffinity (FreeBSD)/std:🧵:hardware_concurrency (others).
With extensive testing on many machines (many configurations from
{aarch64,x86-64} x {Linux,FreeBSD,Windows} x allocators(native,mimalloc,rpmalloc) combinations)
with varying workloads, we discovered that when the concurrency is larger than
16, the linking process is slower than using --threads=16 due to parallelism
overhead outweighs optimizations. This is particularly harmful for machines with
many cores or when the link job competes with other jobs.
Cap parallel::strategy when --threads= is unspecified.
For some workloads changing the concurrency from 8 to 16 has nearly no improvement.
--thinlto-jobs= is unchanged since ThinLTO backend compiles are embarrassingly
parallel.
Link: https://discourse.llvm.org/t/avoidable-overhead-from-threading-by-default/69160
Reviewed By: peter.smith, andrewng
Differential Revision: https://reviews.llvm.org/D147493
As pointed out in
https://discourse.llvm.org/t/undeterministic-thin-index-file/69985, the
block count added to distributed ThinLTO index files breaks incremental
builds on ThinLTO - if any linked file has a different number of BBs,
then the accumulated sum placed in the index files will change, causing
all ThinLTO backend compiles to be redone.
The block count is only used for scaling of partial sample profiles, and
was added in D80403 for D79831.
This patch simply removes this field from the index files of non partial
sample profile compiles, which is NFC on the output of the compiler.
We subsequently need to see if this can be removed for partial sample
profiles without signficant performance loss, or redesigned in a way
that does not destroy caching.
Differential Revision: https://reviews.llvm.org/D148746
This reverts commit da68d2164efcc1f5e57f090e2ae2219056b120a0.
This change is correct, but left a `config->threadCount` use that is error-prone
and may harm performance when parallel::strategy.compute_thread_count() > 16.
Embedded systems that do not use an ELF loader locate the
.ARM.exidx exception table via linker defined __exidx_start and
__exidx_end rather than use the PT_ARM_EXIDX program header. This
means that some linker scripts such as the picolibc C library's
linker script, do not have the .ARM.exidx sections at offset 0 in
the OutputSection. For example:
.except_unordered : {
. = ALIGN(8);
PROVIDE(__exidx_start = .);
*(.ARM.exidx*)
PROVIDE(__exidx_end = .);
} >flash AT>flash :text
This is within the specification of Arm exception tables, and is
handled correctly by ld.bfd.
This patch has 2 parts. The first updates the writing of the data
of the .ARM.exidx SyntheticSection to account for a non-zero
OutputSection offset. The second part makes the PT_ARM_EXIDX program
header generation a special case so that it covers only the
SyntheticSection and not the parent OutputSection. While not strictly
necessary for programs locating the exception tables via the symbols
it may cause ELF utilities that locate the exception tables via
the PT_ARM_EXIDX program header to fail. This does not seem to be the
case for GNU and LLVM readelf which seems to look for the
SHT_ARM_EXIDX section.
Differential Revision: https://reviews.llvm.org/D148033
This implements support for relaxing these relocations to use the GP
register to compute addresses of globals in the .sdata and .sbss
sections.
This feature is off by default and must be enabled by passing
--relax-gp to the linker.
The GP register might not always be the "global pointer". It can
be used for other purposes. See discussion here
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/371
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D143673
When --threads= is unspecified, we set it to
`parallel::strategy.compute_thread_count()`, which uses
sched_getaffinity (Linux)/cpuset_getaffinity (FreeBSD)/std:🧵:hardware_concurrency (others).
With extensive testing on many machines (many configurations from
{aarch64,x86-64} x {Linux,FreeBSD,Windows} x allocators(native,mimalloc,rpmalloc) combinations)
with varying workloads, we discovered that when the concurrency is larger than
16, the linking process is slower than using --threads=16 due to parallelism
overhead outweighs optimizations. This is particularly harmful for machines with
many cores or when the link job competes with other jobs.
Cap parallel::strategy when --threads= is unspecified.
For some workloads changing the concurrency from 8 to 16 has nearly no improvement.
--thinlto-jobs= is unchanged since ThinLTO backend compiles are embarrassingly
parallel.
Link: https://discourse.llvm.org/t/avoidable-overhead-from-threading-by-default/69160
Reviewed By: peter.smith
Differential Revision: https://reviews.llvm.org/D147493
This patch fixes:
mlir/lib/Dialect/Linalg/TransformOps/LinalgTransformOps.cpp:1334:51:
warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
lld/wasm/Writer.cpp:250:39: warning: suggest parentheses around ‘&&’
within ‘||’ [-Wparentheses]
As of Xcode 14.3 it passes -reproducible by default to ld64. It seems
this flag was added in ld64 with Xcode 14.0, but it is not documented.
Through my testing the only thing I have seen it do is the same as
ZERO_AR_DATE, but it's possible it does more, or will do more in the
future. Since we already default to this option, this is more about
handling the command line flag to maintain Xcode compatibility than
anything else.
Differential Revision: https://reviews.llvm.org/D147663
Also, fix checking of first line in ::parse. We can't use the
::getLines helper here since that already does comment stripping
internally.
Differential Revision: https://reviews.llvm.org/D147548
Currently, the --thinlto-prefix-replace="oldpath;newpath" option is used during
distributed ThinLTO thin links to specify the mapping of the input bitcode object
files' directory tree (oldpath) to the directory tree (newpath) used for both:
1) the output files of the thin link itself (the .thinlto.bc index files and the
optional .imports files)
2) the specified object file paths written to the response file given in the
--thinlto-index-only=${response} option, which is used by the final native
link and must match the paths of the native object files that will be
produced by ThinLTO backend compiles.
This patch expands the --thinlto-prefix-replace option to allow a separate directory
tree mapping to be specified for the object file paths written to the response file
(number 2 above). This is important to support builds and build systems where the
same output directory may not be written by multiple build actions (e.g. the thin link
and the ThinLTO backend compiles).
The new format is: --thinlto-prefix-replace="origpath;outpath[;objpath]"
This replaces the origpath directory tree of the thin link input files with
outpath when writing the thin link index and imports outputs (number 1
above). If objpath is specified it replaces origpath of the input files with
objpath when writing the response file (number 2 above), otherwise it
falls back to the old behavior of using outpath for this as well.
Reviewed By: tejohnson, MaskRay
Differential Revision: https://reviews.llvm.org/D144596
For functions that use DWARF encodings, their compact unwind entry will
contain a hint about the offset of their DWARF entry from the start of
the `__eh_frame` section. The encoding only has 3 bytes to encode this
hint.
Previously, I neglected to check for overflow (and didn't realize that
the value was merely a hint without needing to be exact.) So for large
`__eh_frame` sections, the hint would overflow and cause the compact
unwind MODE flag to be corrupted, leading to uncaught exceptions at
runtime.
This diff fixes things by encoding zero as the hint for offsets that are
too large. The unwinder will start a linear search at the hint location
for the matching CFI record. The only requirement is that the hint
points to a valid CFI record start, and the start of the section is
always the start of a CFI record (in well-formed programs).
I'm not adding a test for this because generating the test inputs takes
a bit too much time. However, I have been testing locally with this lit
file, which takes about 15s to run on my machine:
```
# RUN: rm -rf %t; mkdir %t
# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-macos11.0 %s -o %t/test.o
# RUN: %lld -dylib -lSystem %t/test.o -o %t/test
.subsections_via_symbols
.text
.p2align 2
_f:
.cfi_startproc
.rept 0x7fffff
.cfi_escape 0x2e, 0x10
.endr
ret
.cfi_endproc
_g:
.cfi_startproc
.cfi_escape 0x2e, 0x10
ret
.cfi_endproc
```
Reviewed By: #lld-macho, smeenai
Differential Revision: https://reviews.llvm.org/D147505
@oontvoo reports that the current implementation crashes on a bunch of
their builds. Let's leave it disabled by default for now.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D147341
New versions I2.1, F2.2, D2.2 A2.1
Make F and Zfinx imply Zicsr.
Make G imply Zifencei.
This should have no impact to generated code. We have no plans to require Zicsr/Zifencei extension to be explicitly enabled to use Zicsr/Zifencei instructions in assembly.
See https://reviews.llvm.org/D147183 for documentation regarding what version specification we implement.
Reviewed By: asb
Differential Revision: https://reviews.llvm.org/D147179
There are cases where stub library processing can trigger new exports
which might require them to be included at LTO time.
Specifically `processStubLibraries` marks symbols as `forceExports`
which even effect the LTO process.
And since the LTO process can generate new undefined symbols
(specifically libcall function) we need to also process the stub
libraries after LTO.
Differential Revision: https://reviews.llvm.org/D147190
+load methods are static initializers and treated specially by the
runtime: all +load methods for a class & its categories are called when
loading that class, unlike regular methods where only one definition
will get called per message. Thus, there is no need to check for
duplicates.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D147230
We were previously assuming that to hold, but it appears that `swiftc`
may generate classes that link against `__objc_empty_cache` for their
rodata pointer.
This should allow us to re-land {D142916} (as a stack).
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D147229
