bbbbc093fe
This patch introduces support for Integrated Distributed ThinLTO (DTLTO) in COFF LLD. DTLTO enables the distribution of ThinLTO backend compilations via external distribution systems, such as Incredibuild, during the traditional link step: https://llvm.org/docs/DTLTO.html. Note: Bitcode members of non-thin archives are not currently supported. This will be addressed in a future change. This patch is sufficient to allow for self-hosting an LLVM build with DTLTO if thin archives are used. Testing: - LLD `lit` test coverage has been added, using a mock distributor to avoid requiring Clang. - Cross-project `lit` tests cover integration with Clang. For the design discussion of the DTLTO feature, see: https://github.com/llvm/llvm-project/pull/126654
75 lines
3.0 KiB
ReStructuredText
75 lines
3.0 KiB
ReStructuredText
Integrated Distributed ThinLTO (DTLTO)
|
|
======================================
|
|
|
|
Integrated Distributed ThinLTO (DTLTO) enables the distribution of backend
|
|
ThinLTO compilations via external distribution systems, such as Incredibuild,
|
|
during the traditional link step.
|
|
|
|
The implementation is documented here: https://llvm.org/docs/DTLTO.html.
|
|
|
|
Currently, DTLTO is only supported in ELF and COFF LLD.
|
|
|
|
ELF LLD
|
|
-------
|
|
|
|
The command-line interface is as follows:
|
|
|
|
- ``--thinlto-distributor=<path>``
|
|
Specifies the file to execute as the distributor process. If specified,
|
|
ThinLTO backend compilations will be distributed.
|
|
|
|
- ``--thinlto-remote-compiler=<path>``
|
|
Specifies the path to the compiler that the distributor process will use for
|
|
backend compilations. The compiler invoked must match the version of LLD.
|
|
|
|
- ``--thinlto-distributor-arg=<arg>``
|
|
Specifies ``<arg>`` on the command line when invoking the distributor.
|
|
Can be specified multiple times.
|
|
|
|
- ``--thinlto-remote-compiler-arg=<arg>``
|
|
Appends ``<arg>`` to the remote compiler's command line.
|
|
Can be specified multiple times.
|
|
|
|
Options that introduce extra input/output files may cause miscompilation if
|
|
the distribution system does not automatically handle pushing/fetching them to
|
|
remote nodes. In such cases, configure the distributor - possibly using
|
|
``--thinlto-distributor-arg=`` - to manage these dependencies. See the
|
|
distributor documentation for details.
|
|
|
|
Some LLD LTO options (e.g., ``--lto-sample-profile=<file>``) are supported.
|
|
Currently, other options are silently accepted but do not have the intended
|
|
effect. Support for such options will be expanded in the future.
|
|
|
|
COFF LLD
|
|
--------
|
|
|
|
The command-line interface is as follows:
|
|
|
|
- ``/thinlto-distributor:<path>``
|
|
Specifies the file to execute as the distributor process. If specified,
|
|
ThinLTO backend compilations will be distributed.
|
|
|
|
- ``/thinlto-remote-compiler:<path>``
|
|
Specifies the path to the compiler that the distributor process will use for
|
|
backend compilations. The compiler invoked must match the version of LLD.
|
|
|
|
- ``/thinlto-distributor-arg:<arg>``
|
|
Specifies ``<arg>`` on the command line when invoking the distributor.
|
|
Can be specified multiple times.
|
|
|
|
- ``/thinlto-remote-compiler-arg:<arg>``
|
|
Appends ``<arg>`` to the remote compiler's command line.
|
|
Can be specified multiple times.
|
|
|
|
Options that introduce extra input/output files may cause miscompilation if
|
|
the distribution system does not automatically handle pushing/fetching them to
|
|
remote nodes. In such cases, configure the distributor - possibly using
|
|
``/thinlto-distributor-arg:`` - to manage these dependencies. See the
|
|
distributor documentation for details.
|
|
|
|
Some LLD LTO options (e.g., ``/lto-sample-profile:<file>``) are supported.
|
|
Currently, other options are silently accepted but do not have the intended
|
|
effect. Support for such options could be expanded in the future.
|
|
|
|
Currently, there is no DTLTO command line interface supplied for ``clang-cl``,
|
|
as users are expected to invoke LLD directly. |