335 Commits

Author SHA1 Message Date
Fangrui Song
0e47dfede4
[ELF] Add isStaticRelSecType to simplify SHT_REL/SHT_RELA testing. NFC
and make it easier to introduce a new relocation format.

https://discourse.llvm.org/t/rfc-relleb-a-compact-relocation-format-for-elf/77600

Pull Request: https://github.com/llvm/llvm-project/pull/85893
2024-03-20 09:58:56 -07:00
Jinyang He
06a728f3fe
[lld][ELF] Support relax R_LARCH_ALIGN (#78692)
Refer to commit 6611d58f5bbc ("Relax R_RISCV_ALIGN"), we can relax
R_LARCH_ALIGN by same way. Reuse `SymbolAnchor`, `RISCVRelaxAux` and
`initSymbolAnchors` to simplify codes. As `riscvFinalizeRelax` is an
arch-specific function, put it override on `TargetInfo::finalizeRelax`,
so that LoongArch can override it, too.

The flow of relax R_LARCH_ALIGN is almost consistent with RISCV. The
difference is that LoongArch only has 4-bytes NOP and all executable
insn is 4-bytes aligned. So LoongArch not need rewrite NOP sequence.
Alignment maxBytesEmit parameter is supported in psABI v2.30.
2024-02-06 09:09:13 +08:00
Fangrui Song
26cc651f1b
[ELF] Improve comment of InputSection::file and update getFile. NFC (#79550)
Clarify a comment after #78944.

All uses of `getFile()` assert `file` is non-null. `getFile` is not used
with a
synthetic section. Replace `cast_or_null` with `cast`.
2024-01-26 10:43:02 -08:00
Fangrui Song
43b13341fb
[ELF] Add internal InputFile (#78944)
Based on https://reviews.llvm.org/D45375 . Introduce a new InputFile
kind `InternalKind`, use it for

* `ctx.internalFile`: for linker-defined symbols and some synthesized
`Undefined`
* `createInternalFile`: for symbol assignments and --defsym

I picked "internal" instead of "synthetic" to avoid confusion with
SyntheticSection.

Currently a symbol's file is one of: nullptr, ObjKind, SharedKind,
BitcodeKind, BinaryKind. Now it's non-null (I plan to add an
`assert(file)` to Symbol::Symbol and change `toString(const InputFile
*)`
separately).

Debugging and error reporting gets improved. The immediate user-facing
difference is more descriptive "File" column in the --cref output. This
patch may unlock further simplification.

Currently each symbol assignment gets its own
`createInternalFile(cmd->location)`. Two symbol assignments in a linker
script do not share the same file. Making the file the same would be
nice, but would require non trivial code.
2024-01-22 09:09:46 -08:00
Fangrui Song
ae7fb21b5a [ELF] Make some InputSection/InputFile member functions const. NFC 2023-11-16 20:24:14 -08:00
Fangrui Song
56aa727907 [ELF] Add getEnclosingSymbol for code sharing. NFC 2023-11-03 13:43:28 -07:00
Job Noorman
649cac3b62 [ELF][RISCV] Implement --emit-relocs with relaxation
Linker relaxation may change relocations (offsets and types). However,
when --emit-relocs is used, relocations are simply copied from the input
section causing a mismatch with the corresponding (relaxed) code
section.

This patch fixes this as follows: for non-relocatable RISC-V binaries,
`InputSection::copyRelocations` reads relocations from the relocated
section's `relocations` array (since this gets updated by the relaxation
code). For all other cases, relocations are read from the input section
directly as before.

In order to reuse as much code as possible, and to keep the diff small,
the original `InputSection::copyRelocations` is changed to accept the
relocations as a range of `Relocation` objects. This means that, in the
general case when reading from the input section, raw relocations need
to be converted to `Relocation`s first, which introduces quite a bit of
boiler plate. It also means there's a slight code size increase due to
the extra instantiations of `copyRelocations` (for both range types).

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D159082
2023-09-09 12:06:39 +02:00
Elliot Goodrich
b0abd4893f [llvm] Add missing StringExtras.h includes
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
2023-06-25 15:42:22 +01:00
Fangrui Song
8d85c96e0e [lld] StringRef::{starts,ends}with => {starts,ends}_with. NFC
The latter form is now preferred to be similar to C++20 starts_with.
This replacement also removes one function call when startswith is not inlined.
2023-06-05 14:36:19 -07:00
Roland McGrath
9d37ea95df [lld][RISCV] Handle relaxation reductions of more than 65536 bytes
In a real-world case with functions that have many, many
R_RISCV_CALL_PLT relocations due to asan and ubsan
instrumentation, all these can be relaxed by an instruction and
the net result is more than 65536 bytes of reduction in the
output .text section that totals about 1.2MiB in final size.

This changes InputSection to use a 32-bit field for bytesDropped.
The RISCV relaxation keeps track in a 64-bit field and detects
32-bit overflow as it previously detected 16-bit overflow. It
doesn't seem likely that 32-bit overflow will arise, but it's not
inconceivable and it's cheap enough to detect it.

This unfortunately increases the size of InputSection on 64-bit
hosts by a word, but that seems hard to avoid.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D150722
2023-05-16 14:59:36 -07:00
serge-sans-paille
984b800a03
Move from llvm::makeArrayRef to ArrayRef deduction guides - last part
This is a follow-up to https://reviews.llvm.org/D140896, split into
several parts as it touches a lot of files.

Differential Revision: https://reviews.llvm.org/D141298
2023-01-10 11:47:43 +01:00
Guillaume Chatelet
08e2a76381 [lld][NFC] rename ELF alignment into addralign 2022-12-01 16:20:12 +00:00
Fangrui Song
c3c9e45312 [ELF] Add InputSectionBase::{addRelocs,relocs} and GotSection::addConstant to add/access relocations
to prepare for changing `relocations` from a SmallVector to a pointer.

Also change the `isec` parameter in `addAddendOnlyRelocIfNonPreemptible` to `GotSection &`.
2022-11-21 04:12:03 +00:00
Fangrui Song
e8fafafe8e [ELF] Replace rawData+size with content_+size+compressedSize
compressedSize resides in an existing union. sizeof(InputSection) decreases from
160 to 152 on ELF64 systems.
2022-11-20 23:22:32 +00:00
Fangrui Song
2bf5d86422 [ELF] Change rawData to content() and data() to contentMaybeDecompress()
Clarify data() which may trigger decompression and make it feasible to refactor
the member variable rawData.
2022-11-20 22:43:22 +00:00
Fangrui Song
53b674ee15 [ELF] InputSectionBase: add bool compressed to avoid overloading size with compressed semantics
Rename uncompressedSize to size, to prepare for shrinking rawData+size from 3 words to 2 words.
2022-11-20 21:56:13 +00:00
Fangrui Song
898b6c5f3a [ELF] Shuffle SectionBase fields. NFC
Move frequently used fields (e.g. sectionKind, partition) to the top. My x86-64 lld executable is 600+B smaller.
2022-11-20 21:23:30 +00:00
Fangrui Song
685b212553 [ELF] Make relocateAlloc target specific. NFC
The target-specific code (AArch64, PPC64) does not fit into the generic code and
adds virtual function overhead. Move relocateAlloc into ELF/Arch/ instead. This
removes many virtual functions (relaxTls*). In addition, this helps get rid of
getRelocTargetVA dispatch and many RelExpr members in the future.
2022-10-17 11:01:11 -07:00
Fangrui Song
14f996dca8 [ELF] Move inputSections/ehInputSections into Ctx. NFC 2022-10-16 00:49:48 -07:00
Fangrui Song
961439cd7e [ELF] Add LLVM_LIBRARY_VISIBILITY to some global variables. NFC 2022-10-02 13:23:52 -07:00
Fangrui Song
c682c26942 [ELF] Rename InputSectionBase::uncompress to decompress. NFC
The canonical verb is "decompress" (also used in llvm-objcopy). "uncompressed"
describes the state.
2022-09-09 10:18:46 -07:00
Fangrui Song
3e9adff456 [ELF] Split EhInputSection::pieces into cies and fdes
This simplifies code, removes a read32 (for id==0 check), and makes it feasible
to combine some operations in EhInputSection::split and EhFrameSection::addRecords.

Mostly NFC, but fixes "Relocation not in any piece" assertion failure in an
erroneous case when a relocation offset precedes all CIE/FDE pices.
2022-07-31 16:16:10 -07:00
Fangrui Song
c09d323599 [ELF] Move EhInputSection out of inputSections. NFC
inputSections temporarily contains EhInputSection objects mainly for
combineEhSections. Place EhInputSection objects into a new vector
ehInputSections instead of inputSections.
2022-07-31 11:58:08 -07:00
Fangrui Song
a465e79f19 [ELF] Move SyntheticSections to InputSection.h. NFC
Keep the main SectionBase hierarchy in InputSection.h.
And inline MergeInputSection::getParent.
2022-07-30 17:42:08 -07:00
Fangrui Song
6611d58f5b [ELF] Relax R_RISCV_ALIGN
Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle
-mrelax object files (i.e. -mno-relax is no longer needed) and creates a
framework for future relaxation.

`relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing
auxiliary information for relaxation. In the first pass, `relaxAux` is allocated.
The main data structure is `relocDeltas`: when referencing `relocations[i]`, the
actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`.

`relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text
section. Then, adjust st_value/st_size for symbols relative to this section
based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows
that the size has changed.

Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for
convergence of text sections and other address dependent sections (e.g.
SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases
but has issues in some linker script edge cases.

After convergence, compute section contents: shrink the NOP sequence of each
R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of
memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let
the next memcpy skip the desired number of bytes. Section content computation is
parallelizable, but let's ensure the implementation is mature before
optimizations. Technically we can save a copy if we interleave some code with
`OutputSection::writeTo`, but let's not pollute the generic code (we don't have
templated relocation resolving, so using conditions can impose overhead to
non-RISCV.)

Tested:
`make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable.
FreeBSD RISCV64 system using -mrelax is bootable.
bash/curl/firefox/libevent/vim/tmux using -mrelax works.

Differential Revision: https://reviews.llvm.org/D127581
2022-07-07 10:16:09 -07:00
Fangrui Song
e09f77d394 [ELF] Remove support for legacy .zdebug sections
.zdebug is unlikely used any longer: gcc -gz switched from legacy
.zdebug to SHF_COMPRESSED with binutils 2.26 (2016), which has been
several years. clang 14 dropped -gz=zlib-gnu support. According to
Debian Code Search (`gz=zlib-gnu`), no project uses -gz=zlib-gnu.

Remove .zdebug support to (a) simplify code and (b) allow removal of llvm-mc's
--compress-debug-sections=zlib-gnu.

In case the old object file `a.o` uses .zdebug, run `objcopy --decompress-debug-sections a.o`

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D126793
2022-06-02 13:37:19 -07:00
Greg McGary
ebc2529206 [ELF] Move InputSectionBase::rawData member [NFC] 2022-05-09 21:20:14 -07:00
Fangrui Song
4645311933 [ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations
Two code paths may reach the EHFrame case in SectionBase::getOffset:

* .eh_frame reference
* relocation copy for --emit-relocs

The first may be used by clang_rt.crtbegin.o and GCC crtbeginT.o to get the
start address of the output .eh_frame. The relocation has an offset of 0 or
(x86-64 PC-relative leaq for clang_rt.crtbegin.o) -4. The current code just
returns `offset`, which handles this case well.

The second is related to InputSection::copyRelocations on .eh_frame (used by
--emit-relocs). .eh_frame pieces may be dropped due to GC/ICF, so we should
convert the input offset to the output offset. Use the same way as
MergeInputSection with a special case handling outSecOff==-1 for an invalid
piece (see eh-frame-marker.s).

This exposes an issue in mips64-eh-abs-reloc.s that we don't reliably
handle anyway. Just add --no-check-dynamic-relocations to paper over it.

Differential Revision: https://reviews.llvm.org/D122459
2022-03-29 09:51:41 -07:00
Fangrui Song
48e251b1d6 Revert D122459 "[ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations"
This reverts commit 6faba31e0d88ce71e87567ddb51d2444524b8a81.

It may cause "offset is outside the section".
2022-03-28 20:26:21 -07:00
Fangrui Song
6faba31e0d [ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations
.eh_frame pieces may be dropped due to GC/ICF. When --emit-relocs adds
relocations against .eh_frame, the offsets need to be adjusted. Use the same
way as MergeInputSection with a special case handling outSecOff==-1 for an
invalid piece (see eh-frame-marker.s).

This exposes an issue in mips64-eh-abs-reloc.s that we don't reliably
handle anyway. Just add --no-check-dynamic-relocations to paper over it.

Original patch by Ayrton Muñoz

Differential Revision: https://reviews.llvm.org/D122459
2022-03-28 16:23:13 -07:00
Fangrui Song
8565a87fd4 [ELF] Simplify MergeInputSection::getParentOffset. NFC
and remove overly verbose comments.
2022-03-28 10:02:35 -07:00
Fangrui Song
72bedf46c7 [ELF] Inline InputSection::getParent. NFC
Combined with the previous change, lld executable is ~2K smaller and some code
paths using InputSection::getParent are more efficient.

The fragmented headers lead to a design limitation that OutputSection has to be
incomplete, so we cannot use static_cast.
2022-03-08 11:26:12 -08:00
Fangrui Song
fee78961f5 [ELF] Optimize SectionBase::Kind values to make isa<InputSection> more efficient. NFC
Surprisingly my lld executable is 1.5KiB smaller.
2022-02-28 00:24:25 -08:00
Fangrui Song
bb3eeac773 [ELF] Make InputSection::classof inline. NFC 2022-02-28 00:16:45 -08:00
Fangrui Song
ae1ba6194f [ELF] Replace uncompressed InputSectionBase::data() with rawData. NFC
In many call sites we know uncompression cannot happen (non-SHF_ALLOC, or the
data (even if compressed) must have been uncompressed by a previous pass).
Prefer rawData in these cases. data() increases code size and prevents
optimization on rawData.
2022-02-21 00:39:26 -08:00
Fangrui Song
27bb799095 [ELF] Clean up headers. NFC 2022-02-07 21:53:34 -08:00
Fangrui Song
457273fda5 [ELF] splitStrings: replace entSize==1 special case with manual loop unswitch. NFC
My x86-64 lld executable is actually smaller.
2022-01-30 17:15:45 -08:00
Fangrui Song
7cd0c45364 [ELF] Simplify SectionBase::partition handling and make it live by default. NFC
Previously an InputSectionBase is dead (`partition==0`) by default.
SyntheticSection calls markLive and BssSection overrides that with markDead.

It is more natural to make InputSectionBase live by default and let
--gc-sections mark InputSectionBase dead.

When linking a Release build of clang:

* --no-gc-sections:, the removed `inputSections` loop decreases markLive time from 4ms to 1ms.
* --gc-sections: the extra `inputSections` loop increases markLive time from 0.181296s to 0.188526s.
  This is as of we lose the removing one `inputSections` loop optimization (4374824ccf6e7ae68264d996a9ae5bb5e3be7fc5).
  I believe the loss can be mitigated if we refactor markLive.
2022-01-30 15:12:09 -08:00
Fangrui Song
bc1369fae3 [ELF] Optimize MergeInputSection::splitNonStrings with resize_for_overwrite. NFC 2022-01-30 00:10:52 -08:00
Fangrui Song
94e97e668c [ELF] Reorder InputSectionBase::parent. NFC
Move it before others.
2022-01-29 16:20:40 -08:00
Fangrui Song
5f404a749a [ELF] De-template InputSectionBase::getLocation. NFC 2022-01-18 17:33:58 -08:00
Fangrui Song
49f646a9ed [ELF] Change EhInputSection::pieces to SmallVector
The decreased size does not matter that much as one file contributes at most one EhInputSection.
2021-12-27 21:34:38 -08:00
Fangrui Song
315554e873 [ELF] Unify sizeof(InputSection) limits for _WIN32 and others
Windows sizeof(InputSection) seems to match non-Windows now.
2021-12-26 23:02:24 -08:00
Fangrui Song
e90c8c0422 [ELF] Optimize basic block section bytesDropped/jumpInstrMods
and make them more space efficient. This decreases sizeof(InputSection) from 176
to 160, and decreases peak memory usage by 0.3% when linking Chrome.
2021-12-26 22:17:30 -08:00
Fangrui Song
469144ffa3 [ELF] De-template InputSectionBase::getEnclosingFunction 2021-12-26 15:21:22 -08:00
Fangrui Song
213896bc5a [ELF] Remove unused InputSection::getOffsetInFile 2021-12-26 15:18:56 -08:00
Fangrui Song
e1b6b5be46 [ELF] Avoid referencing SectionBase::repl after ICF
It is fairly easy to forget SectionBase::repl after ICF.
Let ICF rewrite a Defined symbol's `section` field to avoid references to
SectionBase::repl in subsequent passes. This slightly improves the --icf=none
performance due to less indirection (maybe for --icf={safe,all} as well if most
symbols are Defined).

With this change, there is only one reference to `repl` (--gdb-index D89751).
We can undo f4fb5fd7523f8e3c3b3966d43c0a28457b59d1d8 (`Move Repl to SectionBase.`)
but move `repl` to `InputSection` instead.

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D116093
2021-12-24 12:09:48 -08:00
Fangrui Song
ba948c5a9c [ELF] Use SmallVector for some global variables (*Files and *Sections). NFC
My lld executable is 26+KiB smaller.
2021-12-22 22:30:08 -08:00
Fangrui Song
4c98d08841 [ELF] Speed up MergeInputSection::split*. NFC 2021-12-16 21:17:02 -08:00
Fangrui Song
bf4fa3036a [ELF] Use SmallVector for MergeInputSection::pieces. NFC
sizeof(pieces) decreases from 24 to 16 on ELF64.
One BumpPtrAllocator can store more MergeInputSections.
The lld executable becomes smaller.
2021-12-16 21:07:39 -08:00