1269 Commits

Author SHA1 Message Date
cmtice
16711b431b
[lld][ELF] Add --debug-names to create merged .debug_names. (#86508)
`clang -g -gpubnames` (with optional -gsplit-dwarf) creates the
`.debug_names` section ("per-CU" index). By default lld concatenates
input `.debug_names` sections into an output `.debug_names` section.
LLDB can consume the concatenated section but the lookup performance is
not good.

This patch adds --debug-names to create a per-module index by combining
the per-CU indexes into a single index that covers the entire load
module. The produced `.debug_names` is a replacement for `.gdb_index`.

Type units (-fdebug-types-section) are not handled yet.

Co-authored-by: Fangrui Song <i@maskray.me>

---------

Co-authored-by: Fangrui Song <i@maskray.me>
2024-04-18 14:41:14 -07:00
Fangrui Song
dcc45faa30 [ELF] PROVIDE: fix spurious "symbol not found"
When archive member extraction involving ENTRY happens after
`addScriptReferencedSymbolsToSymTable`,
`addScriptReferencedSymbolsToSymTable` may fail to define some PROVIDE
symbols used by ENTRY. This is an edge case that regressed after #84512.
(The interaction with PROVIDE and ENTRY-in-archive was not considered
before).

While here, also ensure that --undefined-glob extracted object files
are parsed before `addScriptReferencedSymbolsToSymTable`.

Fixes: ebb326a51fec37b5a47e5702e8ea157cd4f835cd

Pull Request: https://github.com/llvm/llvm-project/pull/87530
2024-04-04 09:38:01 -07:00
Daniil Kovalev
cca9115b1c
[lld][AArch64][ELF][PAC] Support AUTH relocations and AUTH ELF marking (#72714)
This patch adds lld support for:

- Dynamic R_AARCH64_AUTH_* relocations (without including RELR compressed AUTH
relocations) as described here:
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#auth-variant-dynamic-relocations

- .note.AARCH64-PAUTH-ABI-tag section as defined here
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#elf-marking

Depends on #72713 and #85231

---------

Co-authored-by: Peter Collingbourne <peter@pcc.me.uk>
Co-authored-by: Fangrui Song <i@maskray.me>
2024-04-04 12:38:09 +03:00
Fangrui Song
070d7af0c5 [ELF] --export-dynamic: don't create dynamic sections for non-PIC static links
The CloudABI (removed from Clang Driver) change from
https://reviews.llvm.org/D29982 does not make sense. GNU ld and gold
don't create dynamic sections for a non-PIC static link when
--export-dynamic is specified.

Creating dynamic sections is harmful in this scenario because we would
consider undefined weak symbols preemptible and generate GLOB_DAT
relocations, breaking the expectation that non-PIC static links only
contain IRELATIVE relocations.

In addition, there are other options that export symbols
(--export-dynamic-symbol, --dynamic-list, etc). It does not make sense
to special case --export-dynamic.
2024-03-27 20:04:59 -07:00
Sam Clegg
3e046ee0c9
[lld][ELF] Add --why-extract for bitcode libcalls (#78781)
The Wasm linker already records these and its seems useful to do so.
2024-03-25 21:50:00 -07:00
Parth Arora
ebb326a51f [ELF] Fix unnecessary inclusion of unreferenced provide symbols
Previously, linker was unnecessarily including a PROVIDE symbol which
was referenced by another unused PROVIDE symbol. For example, if a
linker script contained the below code and 'not_used_sym' provide symbol
is not included, then linker was still unnecessarily including 'foo' PROVIDE
symbol because it was referenced by 'not_used_sym'. This commit fixes
this behavior.

PROVIDE(not_used_sym = foo)
PROVIDE(foo = 0x1000)

This commit fixes this behavior by using dfs-like algorithm to find
all the symbols referenced in provide expressions of included provide
symbols.

This commit also fixes the issue of unused section not being garbage-collected
if a symbol of the section is referenced by an unused PROVIDE symbol.

Closes #74771
Closes #84730

Co-authored-by: Fangrui Song <i@maskray.me>
2024-03-25 16:11:21 -07:00
Fangrui Song
f1f06f31b8 [ELF] Move parse files from Driver.cpp to InputFiles.cpp. NFC
Fixes: 36146d2b6be53e5e98dee3c1fce8699db9615728

When `doParseFile template defintion` in InputFiles.cpp is optimized
out, we will get a link failure. Actually, we can move the file parsing
loop from Driver.too to InputFiles.cpp and merge it with
parseArmCMSEImportLib.
2024-03-25 16:02:34 -07:00
Fangrui Song
36146d2b6b [ELF] Make LinkerDrive::link a template. NFC
This avoids many invokeELFT in `link`.
2024-03-19 17:12:40 -07:00
Fangrui Song
1bab88df07 [ELF] Move some operations from link to setConfigs. NFC 2024-03-19 16:44:25 -07:00
Fangrui Song
e115c00565
[ELF] Reject certain unknown section types (#85173)
Unknown section sections may require special linking rules, and
rejecting such sections for older linkers may be desired. For example,
if we introduce a new section type to replace a control structure (e.g.
relocations), it would be nice for older linkers to reject the new
section type. GNU ld allows certain unknown section types:

* [SHT_LOUSER,SHT_HIUSER] and non-SHF_ALLOC
* [SHT_LOOS,SHT_HIOS] and non-SHF_OS_NONCONFORMING

but reports errors and stops linking for others (unless
--no-warn-mismatch is specified). Port its behavior. For convenience, we
additionally allow all [SHT_LOPROC,SHT_HIPROC] types so that we don't
have to hard code all known types for each processor.

Close https://github.com/llvm/llvm-project/issues/84812
2024-03-15 09:50:23 -07:00
Fangrui Song
f1ca2a0967
[ELF] Add --compress-section to compress matched non-SHF_ALLOC sections
--compress-sections <section-glib>=[none|zlib|zstd] is similar to
--compress-debug-sections but applies to broader sections without the
SHF_ALLOC flag. lld will report an error if a SHF_ALLOC section is
matched. An interesting use case is to compress `.strtab`/`.symtab`,
which consume a significant portion of the file size (15.1% for a
release build of Clang).

An older revision is available at https://reviews.llvm.org/D154641 .
This patch focuses on non-allocated sections for safety. Moving
`maybeCompress` as D154641 does not handle STT_SECTION symbols for
`-r --compress-debug-sections=zlib` (see `relocatable-section-symbol.s`
from #66804).

Since different output sections may use different compression
algorithms, we need CompressedData::type to generalize
config->compressDebugSections.

GNU ld feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=27452

Link: https://discourse.llvm.org/t/rfc-compress-arbitrary-sections-with-ld-lld-compress-sections/71674

Pull Request: https://github.com/llvm/llvm-project/pull/84855
2024-03-12 10:56:14 -07:00
Fangrui Song
f6455606bb [ELF] Move getSymbol/getRelocTargetSym from ObjFile<ELFT> to InputFile. NFC
This removes lots of unneeded `template getFile<ELFT>()`.
2024-03-10 23:01:26 -07:00
Fangrui Song
78762357d4 [ELF] Support placing .lbss/.lrodata/.ldata after .bss
https://reviews.llvm.org/D150510 places .lrodata before .rodata to
minimize the number of permission transitions in the memory image.
However, this layout is less ideal for -fno-pic code (which is still
important).

Small code model -fno-pic code has R_X86_64_32S relocations with a range
of `[0,2**31)` (if we ignore the negative area). Placing `.lrodata`
earlier exerts relocation pressure on such code. Non-x86 64-bit
architectures generally have a similar `[0,2**31)` limitation if they
don't use PC-relative relocations.

If we place .lrodata later, we will need one extra PT_LOAD. Two layouts
are appealing:

* .bss/.lbss/.lrodata/.ldata (GNU ld)
* .bss/.ldata/.lbss/.lrodata

The GNU ld layout has the nice property that there is only one BSS
(except .tbss/.relro_padding). Add -z lrodata-after-bss to support
this layout.

Since a read-only PT_LOAD segment (for large data sections) may appear
after RW PT_LOAD segments. The placement of `_etext` has to be adjusted.

Pull Request: https://github.com/llvm/llvm-project/pull/81224
2024-02-20 13:59:49 -08:00
Ulrich Weigand
fe3406e349
[lld] Add target support for SystemZ (s390x) (#75643)
This patch adds full support for linking SystemZ (ELF s390x) object
files. Support should be generally complete:
- All relocation types are supported.
- Full shared library support (DYNAMIC, GOT, PLT, ifunc).
- Relaxation of TLS and GOT relocations where appropriate.
- Platform-specific test cases.

In addition to new platform code and the obvious changes, there were a
few additional changes to common code:

- Add three new RelExpr members (R_GOTPLT_OFF, R_GOTPLT_PC, and
R_PLT_GOTREL) needed to support certain s390x relocations. I chose not
to use a platform-specific name since nothing in the definition of these
relocs is actually platform-specific; it is well possible that other
platforms will need the same.

- A couple of tweaks to TLS relocation handling, as the particular
semantics of the s390x versions differ slightly. See comments in the
code.

This was tested by building and testing >1500 Fedora packages, with only
a handful of failures; as these also have issues when building with LLD
on other architectures, they seem unrelated.

Co-authored-by: Tulio Magno Quites Machado Filho <tuliom@redhat.com>
2024-02-13 11:29:21 +01:00
Rahman Lavaee
acec6419e8
[SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (#74128)
Today `-split-machine-functions` and `-fbasic-block-sections={all,list}`
cannot be combined with `-basic-block-sections=labels` (the labels
option will be ignored).
The inconsistency comes from the way basic block address map -- the
underlying mechanism for basic block labels -- encodes basic block
addresses
(https://lists.llvm.org/pipermail/llvm-dev/2020-July/143512.html).
Specifically, basic block offsets are computed relative to the function
begin symbol. This relies on functions being contiguous which is not the
case for MFS and basic block section binaries. This means Propeller
cannot use binary profiles collected from these binaries, which limits
the applicability of Propeller for iterative optimization.
    
To make the `SHT_LLVM_BB_ADDR_MAP` feature work with basic block section
binaries, we propose modifying the encoding of this section as follows.

First let us review the current encoding which emits the address of each
function and its number of basic blocks, followed by basic block entries
for each basic block.

| | |
|--|--|
| Address of the function | Function Address |
|  Number of basic blocks in this function | NumBlocks |
|  BB entry 1
|  BB entry 2
|   ...
|  BB entry #NumBlocks
    
To make this work for basic block sections, we treat each basic block
section similar to a function, except that basic block sections of the
same function must be encapsulated in the same structure so we can map
all of them to their single function.
    
We modify the encoding to first emit the number of basic block sections
(BB ranges) in the function. Then we emit the address map of each basic
block section section as before: the base address of the section, its
number of blocks, and BB entries for its basic block. The first section
in the BB address map is always the function entry section.
| | |
|--|--|
|  Number of sections for this function   | NumBBRanges |
| Section 1 begin address                     | BaseAddress[1]  |
| Number of basic blocks in section 1 | NumBlocks[1]    |
| BB entries for Section 1
|..................|
| Section #NumBBRanges begin address | BaseAddress[NumBBRanges] |
| Number of basic blocks in section #NumBBRanges |
NumBlocks[NumBBRanges] |
| BB entries for Section #NumBBRanges
    
The encoding of basic block entries remains as before with the minor
change that each basic block offset is now computed relative to the
begin symbol of its containing BB section.
    
This patch adds a new boolean codegen option `-basic-block-address-map`.
Correspondingly, the front-end flag `-fbasic-block-address-map` and LLD
flag `--lto-basic-block-address-map` are introduced.
Analogously, we add a new TargetOption field `BBAddrMap`. This means BB
address maps are either generated for all functions in the compiling
unit, or for none (depending on `TargetOptions::BBAddrMap`).
    
This patch keeps the functionality of the old
`-fbasic-block-sections=labels` option but does not remove it. A
subsequent patch will remove the obsolete option.

We refactor the `BasicBlockSections` pass by separating the BB address
map and BB sections handing to their own functions (named
`handleBBAddrMap` and `handleBBSections`). `handleBBSections` renumbers
basic blocks and places them in their assigned sections.
`handleBBAddrMap` is invoked after `handleBBSections` (if requested) and
only renumbers the blocks.
  - New tests added:
- Two tests basic-block-address-map-with-basic-block-sections.ll and
basic-block-address-map-with-mfs.ll to exercise the combination of
`-basic-block-address-map` with `-basic-block-sections=list` and
'-split-machine-functions`.
- A driver sanity test for the `-fbasic-block-address-map` option
(basic-block-address-map.c).
- An LLD test for testing the `--lto-basic-block-address-map` option.
This reuses the LLVM IR from `lld/test/ELF/lto/basic-block-sections.ll`.
- Renamed and modified the two existing codegen tests for basic block
address map (`basic-block-sections-labels-functions-sections.ll` and
`basic-block-sections-labels.ll`)
- Removed `SHT_LLVM_BB_ADDR_MAP_V0` tests. Full deprecation of
`SHT_LLVM_BB_ADDR_MAP_V0` and `SHT_LLVM_BB_ADDR_MAP` version less than 2
will happen in a separate PR in a few months.
2024-02-01 17:50:46 -08:00
Fangrui Song
37efa7038d [ELF] Fix spurious warning for -z rel && -z rela
Fixes: 665f913e4509e3e4f531aa4a4ebe92ec2ea5c23f
2024-01-22 18:18:21 -08:00
Fangrui Song
43b13341fb
[ELF] Add internal InputFile (#78944)
Based on https://reviews.llvm.org/D45375 . Introduce a new InputFile
kind `InternalKind`, use it for

* `ctx.internalFile`: for linker-defined symbols and some synthesized
`Undefined`
* `createInternalFile`: for symbol assignments and --defsym

I picked "internal" instead of "synthetic" to avoid confusion with
SyntheticSection.

Currently a symbol's file is one of: nullptr, ObjKind, SharedKind,
BitcodeKind, BinaryKind. Now it's non-null (I plan to add an
`assert(file)` to Symbol::Symbol and change `toString(const InputFile
*)`
separately).

Debugging and error reporting gets improved. The immediate user-facing
difference is more descriptive "File" column in the --cref output. This
patch may unlock further simplification.

Currently each symbol assignment gets its own
`createInternalFile(cmd->location)`. Two symbol assignments in a linker
script do not share the same file. Making the file the same would be
nice, but would require non trivial code.
2024-01-22 09:09:46 -08:00
Fangrui Song
665f913e45 [ELF] Reimplement unknown -z options using the isClaimed bit
Maintaining the long list of known -z options
(https://reviews.llvm.org/D48621) turns out to be cumbersome. Go the
D48433 route instead.

max-page-size/common-page-size are claimed when `target` is available.

Inspired by: https://reviews.llvm.org/D48433
2024-01-21 18:15:57 -08:00
Fangrui Song
2e30e31e1e [ELF] Claim recognized -z options. NFC
... so that we can reimplement `checkZOptions` using
https://reviews.llvm.org/D48433
2024-01-21 17:43:05 -08:00
Sam Clegg
f5e58a0380
[lld][ELF] Simplify handleLibcall. NFC (#78659)
I noticed this while working on #78658
2024-01-19 12:39:35 -08:00
modiking
17c390fc78
[WPD][LLD] Allow glob matching of --lto-known-safe-vtables (#78505)
Makes it easier to exclude a pattern of safe vtable symbols

Testing:
ninja check-all
2024-01-17 14:31:12 -08:00
Adrian Prantl
2c07181424 [LEB128] Don't initialize error on success
This change removes an unnecessary branch from a hot path. It's also
questionable API to override any previous error unconditonally.
2023-11-29 12:47:27 -08:00
Adrian Prantl
69b0cb9c56 Revert "[LEB128] Don't initialize error on success"
This reverts commit 545c8e009e2b649ef38f7e432ffbc06ba8a9b813.
2023-11-29 12:40:37 -08:00
Adrian Prantl
545c8e009e [LEB128] Don't initialize error on success
This change removes an unnecessary branch from a hot path. It's also
questionable API to override any previous error unconditonally.
2023-11-29 12:16:32 -08:00
spupyrev
b53c04a8da Reapply [ELF] Making cdsort default for function reordering (#68638)
Edited lld/ELF/Options.td to cdsort as well

CDSort function reordering outperforms the existing default heuristic (
hfsort/C^3) in terms of the performance of generated binaries while
being (almost) as fast. Thus, the suggestion is to change the default.
The speedup is up to 1.5% perf for large front-end binaries, and can be
moderate/neutral for "small" benchmarks.

High-level **perf impact** on two selected binaries:
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in
[0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in
[0.8%..1.5%]

More detailed measurements on the clang binary is at
[here](https://reviews.llvm.org/D152834#4445042)
2023-11-03 16:03:06 -07:00
Fangrui Song
e9b9a1d320 [ELF] Move demoteSymbols to Writer.cpp. NFC
History of demoteSharedSymbols:

* https://reviews.llvm.org/D45536 demotes SharedSymbol
* https://reviews.llvm.org/D111365 demotes lazy symbols
* The pending #69295 will demote symbols defined in discarded sections

The pass is placed after markLive just to be clear that it needs `isNeeded`
information computed by markLive. The remaining passes in Driver.cpp do not use
symbol information. Move the pass to Writer.cpp to be closer to other
symbol-related passes.
2023-10-17 13:16:50 -07:00
Fangrui Song
60b3e05967 [ELF] Restore the --call-graph-profile-sort=hfsort default before #68638
The high time complexity of cache-directed sort is a real issue and is not
appropriate as the default, at least for now
(https://github.com/llvm/llvm-project/pull/68638#issuecomment-1760918891).
2023-10-12 22:58:42 -07:00
spupyrev
d5c1d735ad
[ELF] Making cdsort default for function reordering (#68638)
CDSort function reordering outperforms the existing default heuristic (
hfsort/C^3) in terms of the performance of generated binaries while
being (almost) as fast. Thus, the suggestion is to change the default.
The speedup is up to 1.5% perf for large front-end binaries, and can be
moderate/neutral for "small" benchmarks.

High-level **perf impact** on two selected binaries:
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in
[0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in
[0.8%..1.5%]

More detailed measurements on the clang binary is at
[here](https://reviews.llvm.org/D152834#4445042)
2023-10-10 09:06:31 -07:00
Mitch Phillips
144d127bef
[lld] [MTE] Drop MTE globals for fully static executables, not ban (#68217)
Integrating MTE globals on Android revealed a lot of cases where
libraries are built as both archives and DSOs, and they're linked into
fully static and dynamic executables respectively.

MTE globals doesn't work for fully static executables. They need a
dynamic loader to process the special R_AARCH64_RELATIVE relocation
semantics with the encoded offset. Fully static executables that had
out-of-bounds derived symbols (like 'int* foo_end = foo[16]') crash
under MTE globals w/ static executables. So, LLD in its current form
simply errors out when you try and compile a fully static executable
that has a single MTE global variable in it.

It seems like a much better idea to simply have LLD not do the special
work for MTE globals in fully static contexts, and to drop any
unnecessary metadata. This means that you can build archives with MTE
globals and link them into both fully-static and dynamic executables.
2023-10-10 17:32:10 +02:00
spupyrev
904b3f66f5 [ELF] A new code layout algorithm for function reordering [3a/3]
We are brining a new algorithm for function layout (reordering) based on the
call graph (extracted from a profile data). The algorithm is an improvement of
top of a known heuristic, C^3. It tries to co-locate hot and frequently executed
together functions in the resulting ordering. Unlike C^3, it explores a larger
search space and have an objective closely tied to the performance of
instruction and i-TLB caches. Hence, the name CDS = Cache-Directed Sort.
The algorithm can be used at the linking or post-linking (e.g., BOLT) stage.
Refer to https://reviews.llvm.org/D152834 for the actual implementation of the
reordering algorithm.

This diff adds a linker option to replace the existing C^3 heuristic with CDS.
The new behavior can be turned on by passing "--use-cache-directed-sort".
(the plan is to make it default in a next diff)

**Perf-impact**
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in [0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in [0.8%..1.5%]

Note that function layout affects the perf the most on older machines (with
smaller instruction/iTLB caches) and when huge pages are not enabled. The impact
on newer processors with huge pages enabled is likely neutral/minor.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D152840
2023-09-26 06:24:34 -07:00
Fangrui Song
8c556b7e2b [ELF] Change --call-graph-profile-sort to accept an argument
Change the FF form --call-graph-profile-sort to --call-graph-profile-sort={none,hfsort}.
This will be extended to support llvm/lib/Transforms/Utils/CodeLayout.cpp.

--call-graph-profile-sort is not used in the wild but
--no-call-graph-profile-sort is (Chromium). Make --no-call-graph-profile-sort an
alias for --call-graph-profile-sort=none.

Reviewed By: rahmanl

Differential Revision: https://reviews.llvm.org/D159544
2023-09-25 09:49:40 -07:00
modimo
272bd6f9cc [WPD][LLD] Add option to validate RTTI is enabled on all native types and prevent devirtualization on types with native RTTI
Discussion about this approach: https://discourse.llvm.org/t/rfc-safer-whole-program-class-hierarchy-analysis/65144/18

When enabling WPD in an environment where native binaries are present, types we want to optimize can be derived from inside these native files and devirtualizing them can lead to correctness issues. RTTI can be used as a way to determine all such types in native files and exclude them from WPD providing a safe checked way to enable WPD.

The approach is:
1. In the linker, identify if RTTI is available for all native types. If not, under `--lto-validate-all-vtables-have-type-infos` `--lto-whole-program-visibility` is automatically disabled. This is done by examining all .symtab symbols in object files and .dynsym symbols in DSOs for vtable (_ZTV) and typeinfo (_ZTI) symbols and ensuring there's always a match for every vtable symbol.
2. During thinlink, if `--lto-validate-all-vtables-have-type-infos` is set and RTTI is available for all native types, identify all typename (_ZTS) symbols via their corresponding typeinfo (_ZTI) symbols that are used natively or outside of our summary and exclude them from WPD.

Testing:
ninja check-all
large Meta service that uses boost, glog and libstdc++.so runs successfully with WPD via --lto-whole-program-visibility. Previously, native types in boost caused incorrect devirtualization that led to crashes.

Reviewed By: MaskRay, tejohnson

Differential Revision: https://reviews.llvm.org/D155659
2023-09-18 15:51:49 -07:00
Fangrui Song
71d7e69c56 [ELF] Implement getImplicitAddend and enable checkDynamicRelocsDefault for Hexagon 2023-09-15 23:10:17 -07:00
Fangrui Song
df979228ef [ELF] Implement getImplicitAddend and enable checkDynamicRelocsDefault for AMDGPU 2023-09-15 22:59:08 -07:00
Fangrui Song
0cbe49eade [ELF] Implement getImplicitAddend and enable checkDynamicRelocsDefault for PPC32 2023-09-15 22:49:18 -07:00
Fangrui Song
1b65b159da [ELF] Enable checkDynamicRelocsDefault for PPC64
.plt and .branch_lt have the type of SHT_NOBITS and may be relocated by dynamic
relocations with non-zero addends. They should be skipped for the
--check-dynamic-relocations check, as --apply-dynamic-relocs does not apply.

A side effect is that -z rel does not work for the two sections.

Added two --apply-dynamic-relocs --check-dynamic-relocations tests. Also checked
linking a PPC64 clang.
2023-09-15 22:38:18 -07:00
Fangrui Song
d5682e5777 [ELF] checkDynamicRelocsDefault: invert the condition
Fixes: 64c9dbbf4e82224a64dc878e2dc8f7d7541746eb
2023-09-15 22:25:09 -07:00
Fangrui Song
64c9dbbf4e [ELF] checkDynamicRelocsDefault: invert the condition
Most targets support --check-dynamic-relocations --apply-dynamic-relocs, so it
makes sense to invert the condition. We want new ports to support
getImplicitAddend.
2023-09-15 21:06:19 -07:00
Fangrui Song
5a58e98c20
[ELF] Align the end of PT_GNU_RELRO associated PT_LOAD to a common-page-size boundary (#66042)
Close #57618: currently we align the end of PT_GNU_RELRO to a
common-page-size
boundary, but do not align the end of the associated PT_LOAD. This is
benign
when runtime_page_size >= common-page-size.

However, when runtime_page_size < common-page-size, it is possible that
`alignUp(end(PT_LOAD), page_size) < alignDown(end(PT_GNU_RELRO),
page_size)`.
In this case, rtld's mprotect call for PT_GNU_RELRO will apply to
unmapped
regions and lead to an error, e.g.

```
error while loading shared libraries: cannot apply additional memory protection after relocation: Cannot allocate memory
```

To fix the issue, add a padding section .relro_padding like mold, which
is contained in the PT_GNU_RELRO segment and the associated PT_LOAD
segment. The section also prevents strip from corrupting PT_LOAD program
headers.

.relro_padding has the largest `sortRank` among RELRO sections.
Therefore, it is naturally placed at the end of `PT_GNU_RELRO` segment
in the absence of `PHDRS`/`SECTIONS` commands.

In the presence of `SECTIONS` commands, we place .relro_padding
immediately before a symbol assignment using DATA_SEGMENT_RELRO_END (see
also https://reviews.llvm.org/D124656), if present.
DATA_SEGMENT_RELRO_END is changed to align to max-page-size instead of
common-page-size.

Some edge cases worth mentioning:

* ppc64-toc-addis-nop.s: when PHDRS is present, do not append
.relro_padding
* avoid-empty-program-headers.s: when the only RELRO section is .tbss,
it is not part of PT_LOAD segment, therefore we do not append
.relro_padding.

---

Close #65002: GNU ld from 2.39 onwards aligns the end of PT_GNU_RELRO to
a
max-page-size boundary (https://sourceware.org/PR28824) so that the last
page is
protected even if runtime_page_size > common-page-size.

In my opinion, losing protection for the last page when the runtime page
size is
larger than common-page-size is not really an issue. Double mapping a
page of up
to max-common-page for the protection could cause undesired VM waste.
Internally
we had users complaining about 2MiB max-page-size applying to shared
objects.

Therefore, the end of .relro_padding is padded to a common-page-size
boundary. Users who are really anxious can set common-page-size to match
their runtime page size.

---

17 tests need updating as there are lots of change detectors.
2023-09-14 10:33:11 -07:00
Fangrui Song
d1b418f552 [ELF] Reset two member variables in Ctx::reset
Otherwise they are dangling if lldMain is called more than once.
2023-09-11 13:51:01 -07:00
Fangrui Song
65a15a56d5 [ELF] Respect orders of symbol assignments and DEFINED (#65866)
Fix #64600: the currently implementation is minimal (see
https://reviews.llvm.org/D83758), and an assignment like
`__TEXT_REGION_ORIGIN__ = DEFINED(__TEXT_REGION_ORIGIN__) ? __TEXT_REGION_ORIGIN__ : 0;`
(used by avr-ld[1]) leads to a value of zero (default value in `declareSymbol`),
which is unexpected.

Assign orders to symbol assignments and references so that
for a script-defined symbol, the `DEFINED` results match users'
expectation. I am unclear about GNU ld's exact behavior, but this hopefully
matches its behavior in the majority of cases.

[1]: https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/scripttempl/avr.sc
2023-09-11 10:54:49 -07:00
Shoaib Meenai
97e39f96c8 [ELF] Add -Bsymbolic-non-weak
This adds a new -Bsymbolic option that directly binds all non-weak
symbols. There's a couple of reasons motivating this:
* The new flag will match the default behavior on Mach-O, so you can get
  consistent behavior across platforms.
* We have use cases for which making weak data preemptible is useful,
  but we don't want to pessimize access to non-weak data. (For a large
  internal app, we measured 2000+ data symbols whose accesses would be
  unnecessarily pessimized by `-Bsymbolic-functions`.)

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D158322
2023-08-21 09:11:51 -07:00
Paul Kirth
14e3bec8fc Reland "[lld] Preliminary fat-lto-object support"
This patch adds support to lld for --fat-lto-objects. We add a new
--fat-lto-objects option to LLD, and slightly change how it chooses input
files in the driver when the option is set.

Fat LTO objects contain both LTO compatible IR, as well as generated object
code. This allows users to defer the choice of whether to use LTO or not to
link-time. This is a feature available in GCC for some time, and makes the
existing -ffat-lto-objects option functional in the same way as GCC's.

If the --fat-lto-objects option is passed to LLD and the input files are fat
object files, then the linker will chose the LTO compatible bitcode sections
embedded within the fat object and link them together using LTO. Otherwise,
standard object file linking is done using the assembly section in the object
files.

The previous version of this patch had a missing `REQUIRES: x86` line in
`fatlto.invalid.s`. Additionally, it was reported that this patch caused
a test failure in `export-dynamic-symbols.s`, however,
29112a994694baee070a2021e00f772f1913d214 disabled the
`export-dynamic-symbols.s` test on Windows due to a quotation difference
between platforms, unrelated to this patch.

Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D146778
2023-08-18 22:51:25 +00:00
Fangrui Song
ebb0a21099 [GlobPattern] Update invalid glob pattern diagnostic for unmatched '['
Update this diagnostic to mention the reason (unmatched '['), matching
the other diagnostic about stray '\'. The original pattern is omitted,
as some users may mention the original pattern in another position, not
repeating it.
2023-08-08 20:25:10 -07:00
Mitch Phillips
ca35a19aca [lld] Synthesize metadata for MTE globals
As per the ABI at
https://github.com/ARM-software/abi-aa/blob/main/memtagabielf64/memtagabielf64.rst,
this patch interprets the SHT_AARCH64_MEMTAG_GLOBALS_STATIC section,
which contains R_NONE relocations to tagged globals, and emits a
SHT_AARCH64_MEMTAG_GLOBALS_DYNAMIC section, with the correct
DT_AARCH64_MEMTAG_GLOBALS and DT_AARCH64_MEMTAG_GLOBALSSZ dynamic
entries. This section describes, in a uleb-encoded stream, global memory
ranges that should be tagged with MTE.

We are also out of bits to spare in the LLD Symbol class. As a result,
I've reused the 'needsTocRestore' bit, which is a PPC64 only feature.
Now, it's also used for 'isTagged' on AArch64.

An entry in SHT_AARCH64_MEMTAG_GLOBALS_STATIC is practically a guarantee
from an objfile that all references to the linked symbol are through the
GOT, and meet correct alignment requirements. As a result, we go through
all symbols and make sure that, for all symbols $SYM, all object files
that reference $SYM also have a SHT_AARCH64_MEMTAG_GLOBALS_STATIC entry
for $SYM. If this isn't the case, we demote the symbol to being
untagged. Symbols that are imported from other DSOs should always be
fine, as they're GOT-referenced (and thus the GOT entry either has the
correct tag or not, depending on whether it's tagged in the defining DSO
or not).

Additionally hand-tested by building {libc, libm, libc++, libm, and libnetd}
on Android with some experimental MTE globals support in the
linker/libc.

Reviewed By: MaskRay, peter.smith

Differential Revision: https://reviews.llvm.org/D152921
2023-07-31 17:07:42 +02:00
WANG Xuerui
6084ee7420 [lld][ELF] Support LoongArch
This adds support for the LoongArch ELF psABI v2.00 [1] relocation
model to LLD. The deprecated stack-machine-based psABI v1 relocs are not
supported.

The code is tested by successfully bootstrapping a Gentoo/LoongArch
stage3, complete with common GNU userland tools and both the LLVM and
GNU toolchains (GNU toolchain is present only for building glibc,
LLVM+Clang+LLD are used for the rest). Large programs like QEMU are
tested to work as well.

[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html

Reviewed By: MaskRay, SixWeining

Differential Revision: https://reviews.llvm.org/D138135
2023-07-25 17:06:07 +08:00
Paul Kirth
1733d94963 Revert "[lld] Preliminary fat-lto-object support"
This reverts commit c9953d9891a6067549a78e7d07ca8eb6a7596792 and a
forward fix in 3a45b843dec1bca195884aa1c5bc56bd0e6755b4.

D14677 causes some failure on windows bots that the forward fix did not
address. Thus I'm reverting until the underlying cause can me triaged.
2023-07-20 03:37:48 +00:00
Paul Kirth
3a45b843de [lld] Preliminary fat-lto-object support
This patch adds support to lld for --fat-lto-objects. We add a new
--fat-lto-objects flag to LLD, and slightly change how it chooses input
files in the driver when the flag is set.

Fat LTO objects contain both LTO compatible IR, as well as generated object
code. This allows users to defer the choice of whether to use LTO or not to
link-time. This is a feature available in GCC for some time, and makes the
existing -ffat-lto-objects flag functional in the same way as GCC's.

If the --fat-lto-objects option is passed to LLD and the input files are fat
object files, then the linker will chose the LTO compatible bitcode sections
embedded within the fat object and link them together using LTO. Otherwise,
standard object file linking is done using the assembly section in the object
files.

Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977

Depends on D146777

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D146778
2023-07-19 23:07:42 +00:00
Matthew Voss
ab9b3c84a5 [lld] A Unified LTO Bitcode Frontend
The unified LTO pipeline creates a single LTO bitcode structure that can
be used by Thin or Full LTO. This means that the LTO mode can be chosen
at link time and that all LTO bitcode produced by the pipeline is
compatible, from an optimization perspective. This makes the behavior of
LTO a bit more predictable by normalizing the set of LTO features
supported by each LTO bitcode file.

Example usage:

clang -flto -funified-lto -fuse-ld=lld foo.c

clang -flto=thin -funified-lto -fuse-ld=lld foo.c

clang -c -flto -funified-lto foo.c  # -flto={full,thin} are identical in
terms of compilation actions
clang -flto=thin -fuse-ld=lld foo.o # pass --lto=thin to ld.lld

clang -c -flto -funified-lto foo.c clang -flto -fuse-ld=lld foo.o

The RFC discussing the details and rational for this change is here:
https://discourse.llvm.org/t/rfc-a-unified-lto-bitcode-frontend/61774

Differential Revision: https://reviews.llvm.org/D123805
2023-07-18 16:13:58 -07:00
Alex Brachet
67a212af4c [ELF] Make subsequent opens to auxiliary files append
Previously, the same file could be used across diagnostic options but
the file would be silently overwritten by the whichever option gets
handled last.

Differential Revision: https://reviews.llvm.org/D153873
2023-07-11 11:08:57 +00:00