This reverts commit e1e312e6af34803d1686d9ce5a441446811f425d.
The original patch had some python3.10 syntax, which was incompatible with
the python3.8 minimum version in LLVM currently; this reapply brings
everything down to 3.8 compatibility. Separately, some incorrect arguments
were passed in LLDBDAP._get_launch_params, which have been removed.
This patch adds support for bitcode members of thin archives to DTLTO
(https://llvm.org/docs/DTLTO.html) in ELF LLD.
For DTLTO, bitcode identifiers must be valid paths to bitcode files on
disk. Clang does not support archive inputs for ThinLTO backend
compilations. This patch adjusts the identifier for bitcode members of
thin archives in DTLTO links so that it is the path to the member file
on disk, allowing such members to be supported in DTLTO.
This patch is sufficient to allow for self-hosting an LLVM build with
DTLTO when thin archives are used.
Note: Bitcode members of non-thin archives remain unsupported. This will
be addressed in a future change.
Testing:
- LLD lit test coverage has been added to check that the identifier is
adjusted appropriately.
- A cross-project lit test has been added to show that a DTLTO link can
succeed when linking bitcode members of thin archives.
For the design discussion of the DTLTO feature, see: #126654.
This reverts commit 83dfdd8f5485f6b50213c88f02878f86b3f53852.
Temporary revert, as the above patch contains some python code requiring at
least version 3.10, when the minimum required by LLVM is 3.8.
This patch adds a new variety of driver to Dexter, allowing it to work
with DAP-based interfaces for debuggers. The first concrete instance of
this is implemented in this patch, adding support for an `lldb-dap`
debugger. This is functionally very similar to the existing LLDB
debugger support*, but uses lldb-dap as its executable instead of lldb.
This has been tested successfully against the existing feature_test
suite, and manually tested against some other inputs; support is
essentially complete, although any further DAP-based debuggers may
require additional hooks inserted into the base class to deal with any
idiosyncrasies they exhibit (as with the several that have been inserted
for lldb-dap).
NB: There are some small differences resulting from differences between
lldb-dap's use of the lldb API and Dexter's use in its lldb driver; one
small example of this is when evaluating variables, lldb-dap will try to
first use `GetValueForVariablePath` and fallback to `EvaluateExpression`
if necessary, while Dexter will always use `EvaluateExpression`; these
can give slightly different results, resulting in different output from
Dexter for the same input.
This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in COFF LLD.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
Note: Bitcode members of non-thin archives are not currently supported.
This will be addressed in a future change. This patch is sufficient to
allow for self-hosting an LLVM build with DTLTO if thin archives are
used.
Testing:
- LLD `lit` test coverage has been added, using a mock distributor to
avoid requiring Clang.
- Cross-project `lit` tests cover integration with Clang.
For the design discussion of the DTLTO feature, see:
https://github.com/llvm/llvm-project/pull/126654
This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in Clang.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
Testing:
- `lit` test coverage has been added to Clang's Driver tests.
- The DTLTO cross-project tests will use this Clang support.
For the design discussion of the DTLTO feature, see:
https://github.com/llvm/llvm-project/pull/126654
As a legacy of Dexter's role as a test runner, it selects a name for
result files based on the relative path from the test root to each
individual test. Since Dexter no longer takes a test directory as an
argument, only the basename for each test is ever used. This patch adds
an optional --test-root-dir argument, allowing for relative paths to be
used for result files again.
This patch introduces support for Integrated Distributed ThinLTO (DTLTO)
in ELF LLD.
DTLTO enables the distribution of ThinLTO backend compilations via
external distribution systems, such as Incredibuild, during the
traditional link step: https://llvm.org/docs/DTLTO.html.
It is expected that users will invoke DTLTO through the compiler driver
(e.g., Clang) rather than calling LLD directly. A Clang-side interface
for DTLTO will be added in a follow-up patch.
Note: Bitcode members of archives (thin or non-thin) are not currently
supported. This will be addressed in a future change. As a consequence
of this lack of support, this patch is not sufficient to allow for
self-hosting an LLVM build with DTLTO. Theoretically,
--start-lib/--end-lib could be used instead of archives in a self-host
build. However, it's unclear how --start-lib/--end-lib can be easily
used with the LLVM build system.
Testing:
- ELF LLD `lit` test coverage has been added, using a mock distributor
to avoid requiring Clang.
- Cross-project `lit` tests cover integration with Clang.
For the design discussion of the DTLTO feature, see: #126654.
Following commit b8fc288, which changed some dexter test substitutions to
be specific to C and C++, some tests that had been added since the original
patch was written were still using the old substitution; this patch updates
them to use the new.
This patch replaces invocations of clang with clang++ for a set of
c++ files in the dexter cross-project tests. As a small additional change,
this patch removes -lstdc++ from a test that did not appear to require it.
This implements arm, armeb, thumb, thumbeb PLT entries parsing support
in ELF for llvm-objdump.
Implementation is similar to AArch64MCInstrAnalysis::findPltEntries. PLT
entry signatures are based on LLD code for PLT generation
(ARM::writePlt).
llvm-objdump tests are produced from lld/test/ELF/arm-plt-reloc.s,
lld/test/ELF/armv8-thumb-plt-reloc.s.
The cross-project-tests's debuginfo-tests don't rely on lldb being built
to run. While this is a good, a bug in the system lldb can cause a test
to fail with no way of fixing it. This patch makes it so the tests use
the built lldb instead if it's available.
Currently, the index of SWMMAC builtins is of type `short`, likely based
on the
assumption that K can only be up to 32, meaning there are only 16
non-zero
elements. However, this is not future-proof. This patch updates all of
them to
`int`.
The intrinsics themselves don't need to be updated since they accept any
integer
type, and in the backend, they are already extended to 32-bit.
Additionally, the
tests already use various kinds of integers.
Partially fixes SWDEV-518183.
The idea behind this tool was that you'd instrument a source file to
measure its debug-info quality, then watch as it progressively got worse
with more optimisations being enabled in clang. However we've since
stripped the "building" portions of Dexter out as they were ill placed,
which makes this tooling redundant. The lack of __init__.py adjacent to
it means it couldn't be run anyway.
The core idea behind this is still sound; just it's best placed on the
other side of the build system, something that Dexter shouldn't try to
solve.
Fixes issue added by: https://github.com/llvm/llvm-project/pull/111833
Following the previous commit that changed how Dexter imports modules,
the ComInterface module import became broken. This is because it had a
different directory structure to other modules, where we want to import
single file rather than a dir containing a __init__.py. For this case,
an optional extra arg has been added to load_module allowing a filename
to be specified, letting us import ComInterface.py directly and fixing
the issue.
Fixes: https://github.com/llvm/llvm-project/issues/111815
This patch replaces usage of the python `imp` library, which is
deprecated since python3.4 and removed in python3.12, with the
`importlib` library. As part of this update the repeated
find_module+load_module pattern is moved into a utility function, since
the importlib equivalent is much more verbose.
lldb will change how it reports stop reasons around breakpoints in the
near future. I landed an earlier version of this change and noticed
debuginfo test failures on the CI bots due to the changes. I'm
addressing the issues found by CI at
https://github.com/llvm/llvm-project/pull/105594 and will re-land once
I've done all of them.
Currently, when lldb stops at a breakpoint instruction -- but has not
yet executed the instruction -- it will overwrite the thread's Stop
Reason with "breakpoint-hit". This caused bugs when the original stop
reason was important to the user - for instance, a watchpoint on an
AArch64 system where we have to instruction-step past the watchpoint to
find the new value. Normally we would instruction step, fetch the new
value, then report the user that a watchpoint has been hit with the old
and new values. But if the instruction after this access is a breakpoint
site, we overwrite the "watchpoint hit" stop reason (and related
actions) with "breakpoint hit".
dexter sets breakpoints on all source lines, then steps line-to-line,
hitting the breakpoints. But with this new behavior, we see two steps
per source line: The first step gets us to the start of the next line,
with a "step completed" stop reason. Then we step again and we execute
the breakpoint instruction, stop with the pc the same, and report
"breakpoint hit". Now we can step a second time and move past the
breakpoint.
I've changed the `step` method in LLDB.py to check if we step to a
breakpoint site but have a "step completed" stop reason -- in which case
we have this new breakpoint behavior, and we need to step a second time
to actually hit the breakpoint like the debuginfo tests expect.
Getting this to work required a few additional changes:
- Add builtins for any instructions that can't be done with plain C
currently.
- Add support for the saturating version of fp_to_<s,i>_I16x8. Other
vector sizes supported this already.
- Support bitcast of f16x8 to v128. Needed to return a __f16x8 as
v128_t.
Currently the output of dexter --version contains the raw output of `git
remote get-url origin`, which may contain a username and password. This
patch adds a small change to remove these from the output string. A
similar patch for LLVM's default version string* also removes the git
URL altogether unless opted-in to; it's not clear whether this is a
necessary or desirable step yet, but if so we can trivially remove the
URL from Dexter as well.
*PR here: https://github.com/llvm/llvm-project/pull/105220
Python deprecated the distutils package in 3.10, and removed it in 3.12
causing problems when trying to run the lit tests with 3.12.
https://docs.python.org/3.10/library/distutils.html
Replace usage with the looseversion package which should be a drop-in
replacement for the original usage.
If your testing fails after this commit, you need to install the looseversion package.
Remove the testing for std::optional - it was originally for
llvm::Optional, but now that that doesn't exist and we use
std::optional, testing for that pretty printer should live, wherever the
pretty printer lives, not here in LLVM.
And the PointerIntPair pretty printer bit rotted due to changes in
PointerIntPair, 875391728c11339c8a6cd3338bcaa5ec0ffc2496.
Prior to this patch VisualStudio._get_step_info incorrectly identifies
the reason the debugger has stopped. e.g., stepping through a program
would be reported as a StopReason.Breakpoint rather than
StopReason.Step.
Fix. No test added as there are no VisualStudio tests (tested locally).
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
I am trying to bring up a MacOS buildbot targeting x86 and noticed that
two Dexter tests were failing,
cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp and
cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp.
Looking in the history for these tests, they were XFAILed for Apple
Silicon in 9c46606 and are failing similar on x86 for me, so we should extend
the XFAIL to all MacOS architectures.
Fix the bug where merge-fdata unconditionally outputs boltedcollection
line, regardless of whether input files have it set.
Test Plan:
Added bolt/test/X86/merge-fdata-nobat-mode.test which fails without this
fix.
Since Dexter no longer intends to build any code, the ShouldBuild
property in any Visual Studio project being run by Dexter should be
false to ensure that a build step is never invoked by Dexter, whether
the project has already been built or not.
Reviewed by: OCHyams
The purpose of this example is to provide a case where the debugger /
debug info experience could be improved. A recent commit by clang
(0d2860b795879f4dd152963b52f969b53b136899) changed codegen such that it
changes how "small structs" are initialized, in a way that the debugger
is now able to correctly display the variable being targeted by this
test.
In order to keep the example relevant, i.e. failing, this commit makes
it so that the struct is now "big enough" to not trigger the new
codegen.
Currently if Dexter encounters a parser error with a command, the resulting
error message will refer to the most recently declared file (i.e. the source
file it is testing) rather than the file containing the command itself. This
patch fixes this so that parser errors point towards the correct location.
The issue with these test failures is that the dSYM was not being found
by lldb, which is why setting breakpoints was failing and lldb quit
without performing any steps. This change copies the dSYM to the same
temp directory that the executable is copied to.
