80b304d14b
The in-process ThinLTO backend typically generates object files in memory and adds them directly to the link, except when the ThinLTO cache is in use. DTLTO is unusual in that it adds files to the link from disk in all cases. When the ThinLTO cache is not in use, ThinLTO adds files via an `AddStreamFn` callback provided by the linker, which ultimately appends to a `SmallVector` in LLD. When the cache is in use, the linker supplies an `AddBufferFn` callback that adds files more efficiently (by moving `MemoryBuffer` ownership). This patch adds a mandatory `AddBufferFn` to the DTLTO ThinLTO backend. The backend uses this to add files to the link more efficiently. Additionally: - Move AddStream from CGThinBackend to InProcessThinBackend, for reader clarity. - Modify linker comments that implied the AddBuffer path is cache-specific. For a Clang link (Debug build with sanitizers and instrumentation) using an optimized toolchain (PGO non-LTO, llvmorg-22.1.0), measuring the mean `Add DTLTO files to the link` time trace scope duration: - On Windows (Windows 11 Pro Build 26200, AMD Family 25 @ ~4.5 GHz, 16 cores/32 threads, 64 GB RAM), this patch reduces the mean from 2799.148 ms to 157.972 ms. - On Linux (Ubuntu 24.04.3 LTS Kernel 6.14, Ryzen 9 5950X, 16 cores/32 threads, boost up to 5.09 GHz, 64 GB RAM), this patch reduces the mean from 255.291 ms to 41.630 ms. Based on work by @romanova-ekaterina and @kbelochapka.
LLVM Linker (lld)
This directory and its subdirectories contain source code for the LLVM Linker, a modular cross platform linker which is built as part of the LLVM compiler infrastructure project.
lld is open source software. You may freely distribute it under the terms of the license agreement found in LICENSE.txt.
Benchmarking
In order to make sure various developers can evaluate patches over the same tests, we create a collection of self contained programs.
It is hosted at https://s3-us-west-2.amazonaws.com/linker-tests/lld-speed-test.tar.xz
The current sha256 is 10eec685463d5a8bbf08d77f4ca96282161d396c65bd97dc99dbde644a31610f.