This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in COFF LLD.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
Note: Bitcode members of non-thin archives are not currently supported.
This will be addressed in a future change. This patch is sufficient to
allow for self-hosting an LLVM build with DTLTO if thin archives are
used.
Testing:
- LLD `lit` test coverage has been added, using a mock distributor to
avoid requiring Clang.
- Cross-project `lit` tests cover integration with Clang.
For the design discussion of the DTLTO feature, see:
https://github.com/llvm/llvm-project/pull/126654
(cherry picked from commit bbbbc093febffcae262cde1baa429b950842d76e)
This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in Clang.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
Testing:
- `lit` test coverage has been added to Clang's Driver tests.
- The DTLTO cross-project tests will use this Clang support.
For the design discussion of the DTLTO feature, see:
https://github.com/llvm/llvm-project/pull/126654
(cherry picked from commit 5004c59803fdeb3389d30439a6cc8b1ff874df0c)
As a legacy of Dexter's role as a test runner, it selects a name for
result files based on the relative path from the test root to each
individual test. Since Dexter no longer takes a test directory as an
argument, only the basename for each test is ever used. This patch adds
an optional --test-root-dir argument, allowing for relative paths to be
used for result files again.
This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in ELF LLD.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
It is expected that users will invoke DTLTO through the compiler driver
(e.g., Clang) rather than calling LLD directly. A Clang-side interface
for DTLTO will be added in a follow-up patch.
Note: Bitcode members of archives (thin or non-thin) are not currently
supported. This will be addressed in a future change. As a consequence
of this lack of support, this patch is not sufficient to allow for
self-hosting an LLVM build with DTLTO. Theoretically,
--start-lib/--end-lib could be used instead of archives in a self-host
build. However, it's unclear how --start-lib/--end-lib can be easily
used with the LLVM build system.
Testing:
- ELF LLD `lit` test coverage has been added, using a mock distributor
to avoid requiring Clang.
- Cross-project `lit` tests cover integration with Clang.
For the design discussion of the DTLTO feature, see: #126654.
Following commit b8fc288, which changed some dexter test substitutions to
be specific to C and C++, some tests that had been added since the original
patch was written were still using the old substitution; this patch updates
them to use the new.
This patch replaces invocations of clang with clang++ for a set of
c++ files in the dexter cross-project tests. As a small additional change,
this patch removes -lstdc++ from a test that did not appear to require it.
This implements arm, armeb, thumb, thumbeb PLT entries parsing support
in ELF for llvm-objdump.
Implementation is similar to AArch64MCInstrAnalysis::findPltEntries. PLT
entry signatures are based on LLD code for PLT generation
(ARM::writePlt).
llvm-objdump tests are produced from lld/test/ELF/arm-plt-reloc.s,
lld/test/ELF/armv8-thumb-plt-reloc.s.
The cross-project-tests's debuginfo-tests don't rely on lldb being built
to run. While this is a good, a bug in the system lldb can cause a test
to fail with no way of fixing it. This patch makes it so the tests use
the built lldb instead if it's available.
Currently, the index of SWMMAC builtins is of type `short`, likely based
on the
assumption that K can only be up to 32, meaning there are only 16
non-zero
elements. However, this is not future-proof. This patch updates all of
them to
`int`.
The intrinsics themselves don't need to be updated since they accept any
integer
type, and in the backend, they are already extended to 32-bit.
Additionally, the
tests already use various kinds of integers.
Partially fixes SWDEV-518183.
The idea behind this tool was that you'd instrument a source file to
measure its debug-info quality, then watch as it progressively got worse
with more optimisations being enabled in clang. However we've since
stripped the "building" portions of Dexter out as they were ill placed,
which makes this tooling redundant. The lack of __init__.py adjacent to
it means it couldn't be run anyway.
The core idea behind this is still sound; just it's best placed on the
other side of the build system, something that Dexter shouldn't try to
solve.
Fixes issue added by: https://github.com/llvm/llvm-project/pull/111833
Following the previous commit that changed how Dexter imports modules,
the ComInterface module import became broken. This is because it had a
different directory structure to other modules, where we want to import
single file rather than a dir containing a __init__.py. For this case,
an optional extra arg has been added to load_module allowing a filename
to be specified, letting us import ComInterface.py directly and fixing
the issue.
Fixes: https://github.com/llvm/llvm-project/issues/111815
This patch replaces usage of the python `imp` library, which is
deprecated since python3.4 and removed in python3.12, with the
`importlib` library. As part of this update the repeated
find_module+load_module pattern is moved into a utility function, since
the importlib equivalent is much more verbose.
lldb will change how it reports stop reasons around breakpoints in the
near future. I landed an earlier version of this change and noticed
debuginfo test failures on the CI bots due to the changes. I'm
addressing the issues found by CI at
https://github.com/llvm/llvm-project/pull/105594 and will re-land once
I've done all of them.
Currently, when lldb stops at a breakpoint instruction -- but has not
yet executed the instruction -- it will overwrite the thread's Stop
Reason with "breakpoint-hit". This caused bugs when the original stop
reason was important to the user - for instance, a watchpoint on an
AArch64 system where we have to instruction-step past the watchpoint to
find the new value. Normally we would instruction step, fetch the new
value, then report the user that a watchpoint has been hit with the old
and new values. But if the instruction after this access is a breakpoint
site, we overwrite the "watchpoint hit" stop reason (and related
actions) with "breakpoint hit".
dexter sets breakpoints on all source lines, then steps line-to-line,
hitting the breakpoints. But with this new behavior, we see two steps
per source line: The first step gets us to the start of the next line,
with a "step completed" stop reason. Then we step again and we execute
the breakpoint instruction, stop with the pc the same, and report
"breakpoint hit". Now we can step a second time and move past the
breakpoint.
I've changed the `step` method in LLDB.py to check if we step to a
breakpoint site but have a "step completed" stop reason -- in which case
we have this new breakpoint behavior, and we need to step a second time
to actually hit the breakpoint like the debuginfo tests expect.
Getting this to work required a few additional changes:
- Add builtins for any instructions that can't be done with plain C
currently.
- Add support for the saturating version of fp_to_<s,i>_I16x8. Other
vector sizes supported this already.
- Support bitcast of f16x8 to v128. Needed to return a __f16x8 as
v128_t.
Currently the output of dexter --version contains the raw output of `git
remote get-url origin`, which may contain a username and password. This
patch adds a small change to remove these from the output string. A
similar patch for LLVM's default version string* also removes the git
URL altogether unless opted-in to; it's not clear whether this is a
necessary or desirable step yet, but if so we can trivially remove the
URL from Dexter as well.
*PR here: https://github.com/llvm/llvm-project/pull/105220
Python deprecated the distutils package in 3.10, and removed it in 3.12
causing problems when trying to run the lit tests with 3.12.
https://docs.python.org/3.10/library/distutils.html
Replace usage with the looseversion package which should be a drop-in
replacement for the original usage.
If your testing fails after this commit, you need to install the looseversion package.
Remove the testing for std::optional - it was originally for
llvm::Optional, but now that that doesn't exist and we use
std::optional, testing for that pretty printer should live, wherever the
pretty printer lives, not here in LLVM.
And the PointerIntPair pretty printer bit rotted due to changes in
PointerIntPair, 875391728c11339c8a6cd3338bcaa5ec0ffc2496.
Prior to this patch VisualStudio._get_step_info incorrectly identifies
the reason the debugger has stopped. e.g., stepping through a program
would be reported as a StopReason.Breakpoint rather than
StopReason.Step.
Fix. No test added as there are no VisualStudio tests (tested locally).
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
I am trying to bring up a MacOS buildbot targeting x86 and noticed that
two Dexter tests were failing,
cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp and
cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp.
Looking in the history for these tests, they were XFAILed for Apple
Silicon in 9c46606 and are failing similar on x86 for me, so we should extend
the XFAIL to all MacOS architectures.
Fix the bug where merge-fdata unconditionally outputs boltedcollection
line, regardless of whether input files have it set.
Test Plan:
Added bolt/test/X86/merge-fdata-nobat-mode.test which fails without this
fix.
Since Dexter no longer intends to build any code, the ShouldBuild
property in any Visual Studio project being run by Dexter should be
false to ensure that a build step is never invoked by Dexter, whether
the project has already been built or not.
Reviewed by: OCHyams
The purpose of this example is to provide a case where the debugger /
debug info experience could be improved. A recent commit by clang
(0d2860b795879f4dd152963b52f969b53b136899) changed codegen such that it
changes how "small structs" are initialized, in a way that the debugger
is now able to correctly display the variable being targeted by this
test.
In order to keep the example relevant, i.e. failing, this commit makes
it so that the struct is now "big enough" to not trigger the new
codegen.
Currently if Dexter encounters a parser error with a command, the resulting
error message will refer to the most recently declared file (i.e. the source
file it is testing) rather than the file containing the command itself. This
patch fixes this so that parser errors point towards the correct location.
The issue with these test failures is that the dSYM was not being found
by lldb, which is why setting breakpoints was failing and lldb quit
without performing any steps. This change copies the dSYM to the same
temp directory that the executable is copied to.
The patch in https://reviews.llvm.org/D151465 broke greendragon, because
it is hardcoding the use of libstdc++ which is not available on darwin.
XFAILing these tests till they can be fixed.
The optnone tests, following the removal of the Dexter builder in a
prior patch (45a40c16), are experiencing intermittent failures. This
patch marks them unsupported to be fixed in a new commit, rather than
reverting the Dexter patch.
This patch makes a further attempt to fix the tests broken by the
previous revision by ensuring that the command line for the modified
Dexter tests use -std=gnu++11, in keeping with the old build script.
This reverts commit 5647f2908de90fe07b0805e988cd2e91a1751928.
Re-application of the Dexter builder removal reversed due to continued
errors on the green dragon LLDB buildbot:
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/59716/
Cause of the error is unclear, but it looks as though there is some
unexpected non-determinism in the test failures.
This reverts commit 323270451d8db24f2c816f455b3d8f70f434286d.
Fixes a test which was broken on the green dragon buildbot, and further
failures on the SIE buildbot.
This reverts commit 8df9eff90ff8c79e3e508cce7fec1cd897487069.
The error message "Couldn't lookup symbols" emitted from IRExecutionUnit
is grammatically incorrect. "Lookup" is noun when spelled without a
space. Update the error message to use the verb "look up" instead.
& Revert "[Dexter] Fix incorrect substitution errors in clang-cl builder"
This reverts commits 262520a3c5450fd7f149438245b2aef12736347f,
and 0b72b71cd3c8ad824342c05bc6d9d64d87eeb81b.
Failures occurred on two buildbots, the SIE buildbot:
https://lab.llvm.org/buildbot/#/builders/216/builds/26006
And the green dragon buildbot:
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/59091
Errors appear to be related to incorrect tool substitution in the Dexter
test commands, and a currently unknown error with one of the general
debuginfo tests that uses Dexter.
