This PR introduces support for the DWARF64 format, enabling handling of
64-bit DWARF sections as defined by the DWARF specification. The update
includes adjustments to header parsing and modification of form values
to accommodate 64-bit offsets and values.
Also Added the testcase to verify the DWARF64 format.
They are left over from our previous attempt at DWARF64. The new attempt
is not using them, and they also don't have equivalents in the llvm
DWARFDataExtractor class.
This relands changes in #144424 for adding a count of DWO files
parsed/loaded and the total number of DWO files.
The previous PR was reverted in #145494 due to the newly added unit
tests failing on Windows and MacOS CIs since these platforms don't
support DWO. This change add an additional
`@add_test_categories(["dwo"])` to the new tests to
[skip](cd46354dbd/lldb/packages/Python/lldbsuite/test/test_categories.py (L56))
these tests on Windows/MacOS.
Original PR: #144424
### Testing
Ran unit tests
```
$ bin/lldb-dotest -p TestStats.py llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 24 tests in 211.391s
OK (skipped=3)
```
## Summary
A new `totalLoadedDwoFileCount` and `totalDwoFileCount` counters to
available statisctics when calling "statistics dump".
1. `GetDwoFileCounts ` is created, and returns a pair of ints
representing the number of loaded DWO files and the total number of DWO
files, respectively. An override is implemented for `SymbolFileDWARF`
that loops through each compile unit, and adds to a counter if it's a
DWO unit, and then uses `GetDwoSymbolFile(false)` to check whether the
DWO file was already loaded/parsed.
3. In `Statistics`, use `GetSeparateDebugInfo` to sum up the total
number of loaded/parsed DWO files along with the total number of DWO
files. This is done by checking whether the DWO file was already
successfully `loaded` in the collected DWO data, anding adding to the
`totalLoadedDwoFileCount`, and adding to `totalDwoFileCount` for all CU
units.
## Expected Behavior
- When binaries are compiled with split-dwarf and separate DWO files,
`totalLoadedDwoFileCount` would be the number of loaded DWO files and
`totalDwoFileCount` would be the total count of DWO files.
- When using a DWP file instead of separate DWO files,
`totalLoadedDwoFileCount` would be the number of parsed compile units,
while `totalDwoFileCount` would be the total number of CUs in the DWP
file. This should be similar to the counts we get from loading separate
DWO files rather than only counting whether a single DWP file was
loaded.
- When not using split-dwarf, we expect both `totalDwoFileCount` and
`totalLoadedDwoFileCount` to be 0 since no separate debug info is
loaded.
## Testing
**Manual Testing**
On an internal script that has many DWO files, `statistics dump` was
called before and after a `type lookup` command. The
`totalLoadedDwoFileCount` increased as expected after the `type lookup`.
```
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 29,
}
(lldb) type lookup folly::Optional<unsigned int>::Storage
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
...
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 2160,
}
```
**Unit test**
Added three unit tests that build with new "third.cpp" and "baz.cpp"
files. For tests with w/ flags `-gsplit-dwarf -gpubnames`, this
generates 2 DWO files. Then, the test incrementally adds breakpoints,
and does a type lookup, and the count should increase for each of these
as new DWO files get loaded to support these.
```
$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 20 tests in 211.738s
OK (skipped=3)
```
This reverts commit 877511920dcf36463e06746d626e8876583a6abd.
This fixes the `TestObjCInBlockVars.py` LLDB API test.
The issue was that `GetCXXObjectParameter` wouldn't deduce the object
parameter of Objective-C method definitions correctly. In DWARF those
don't have a `DW_AT_specification` (so no link back to a DeclContext
that is a class type). The fix is to only check the validity of the
DeclContext DIE *if* no `DW_AT_object_pointer` exists on the DIE. If
`DW_AT_object_pointer` does exist, we should just always use that as the
object_parameter.
`DWARFFormValue::ExtractValue` has nothing to extract for
`DW_FORM_implicit_const` since the value is stored in the abbreviation.
`DWARFFormValue` expects the user to have set the value of the
implicit_const. This patch does so in `GetAttributeValue`.
Some inter-plugin dependencies are okay, others are not. Yet others not,
but we're sort of stuck with them. The idea is to be able to prevent
backsliding while making sure that acceptable dependencies are..
accepted. For context, see
https://github.com/llvm/llvm-project/pull/139170 and the attached
changes to the documentation.
We can just always use `GetCXXObjectParameter` instead. We've only used
this attribute to set the object parameter name on ClangASTMetadata,
which doesn't seem like good enough justification to keep it around.
Depends on https://github.com/llvm/llvm-project/pull/144879
My goal is to remove the `object_pointer` member on
`ParsedDWARFTypeAttributes` since it's duplicating information that we
retrieve with `GetCXXObjectParameter` anyway. To continue having
coverage for the `DW_AT_object_pointer` code-paths, instead of checking
the
`attrs.object_pointer` I'm now calling `GetCXXObjectParameter` directly.
We could find some very roundabout way to go via the Clang AST to check
that the object parameter was parsed correctly, but that quickly became
quite painful.
Depends on https://github.com/llvm/llvm-project/pull/144876
I'm trying to call `GetCXXObjectParameter` from unit-tests in a
follow-up patch and taking a `DWARFDIE` instead of `clang::DeclContext`
makes that much simpler. These should be equivalent, since all we're
trying to check is that the parent context is a record type.
This fixes the error reported in
https://github.com/llvm/llvm-project/pull/144037.
When computing the aranges table of a CU, LLDB would currently visit all
`DW_TAG_subprogram` DIEs and check their
`DW_AT_low_pc`/`DW_AT_high_pc`/`DW_AT_ranges` attributes. If those don't
exist it would error out and spam the console. Some subprograms
(particularly forward declarations) don't have low/high pc attributes,
so it's not really an "error". See DWARFv5 spec section `3.3.3
Subroutine and Entry Point Locations`:
```
A subroutine entry may have either a DW_AT_low_pc and DW_AT_high_pc
pair of attributes or a DW_AT_ranges attribute whose values encode the
contiguous or non-contiguous address ranges, respectively, of the machine
instructions generated for the subroutine (see Section 2.17 on page 51).
...
A subroutine entry representing a subroutine declaration that is not also a
definition does not have code address or range attributes.
```
We should just ignore those DIEs.
This reverts commit 3096f8768676bd64123270cc59b7cc904a72d875.
Reverting this commit because it depends on another PR
that was reverted, https://github.com/llvm/llvm-project/pull/142704
Both can be reapplied once we find a correct fix for that.
# Change
`SymbolFileDWARFDebugMap::CreateInstance()` will return `nullptr` if the
file is not a Mach-O.
# Benefit
This may improve **Linux** debugger launch time by skipping the creation
of `SymbolFileDWARFDebugMap` during the [`SymbolFile::FindPlugin()`
call](https://fburl.com/hi1w8dil), which loops through a list of
`SymbolFile` plugins and tries to find the one that provides the best
abilities. If the `SymbolFileDWARFDebugMap` is created during this loop,
it will load the symbol table of the file in question and loop through
all the compile units in the debug map (the OSO entries) to calculate
the abilities.
# Tests
See PR.
If we're not touching them, we don't need to do anything special to pass
them along -- with one important caveat: due to how cmake arguments
work, the implicitly passed arguments need to be specified before
arguments that we handle.
This isn't particularly nice, but the alternative is enumerating all
arguments that can be used by llvm_add_library and the macros it calls
(it also relies on implicit passing of some arguments to
llvm_process_sources).
Depends on https://github.com/llvm/llvm-project/pull/142163
This patch makes the `-ast-dump-filter` Clang option available to the
`target modules dump ast` command. This allows us to selectively dump
parts of the AST by name.
The AST can quickly grow way too large to skim on the console. This will
aid in debugging AST related issues.
Example:
```
(lldb) target modules dump ast --filter func
Dumping clang ast for 48 modules.
Dumping func:
FunctionDecl 0xc4b785008 <<invalid sloc>> <invalid sloc> func 'void (int)' extern
|-ParmVarDecl 0xc4b7853d8 <<invalid sloc>> <invalid sloc> x 'int'
`-AsmLabelAttr 0xc4b785358 <<invalid sloc>> Implicit "_Z4funcIiEvT_"
Dumping func<int>:
FunctionDecl 0xc4b7850b8 <<invalid sloc>> <invalid sloc> func<int> 'void (int)' implicit_instantiation extern
|-TemplateArgument type 'int'
| `-BuiltinType 0xc4b85b110 'int'
`-ParmVarDecl 0xc4b7853d8 <<invalid sloc>> <invalid sloc> x 'int'
```
The majority of this patch is adjust the `Dump` API. The main change in
behaviour is in `TypeSystemClang::Dump`, where we now use the
`ASTPrinter` for dumping the `TranslationUnitDecl`. This is where the
`-ast-dump-filter` functionality lives in Clang.
Inferred submodule declarations are emitted in DWARF as `DW_TAG_module`s
without `DW_AT_LLVM_include_path`s. Instead the parent DIE will have the
include path. This patch adds support for such setups. Without this, the
`ClangModulesDeclVendor` would fail to `AddModule` the submodules (i.e.,
compile and load the submodules). This would cause errors such as:
```
note: error: Header search couldn't locate module 'TopLevel'
```
The test added here also tests
https://github.com/llvm/llvm-project/pull/141220. Without that patch
we'd fail with:
```
note: error: No module map file in /Users/jonas/Git/llvm-worktrees/llvm-project/lldb/test/API/lang/cpp/decl-from-submodule
```
Unfortunately the embedded clang instance doesn't allow us to use the
decls we find in the modules. But we'll try to fix that in a separate
patch.
rdar://151022173
Add an overloaded `GetTypeSystem` to specify the expected type system subclass. Changes code from `GetTypeSystem().dyn_cast_or_null<TypeSystemClang>()` to `GetTypeSystem<TypeSystemClang>()`.
Here we were initializing & locking a shared_mutex in a thread, while
releasing it in the parent which may/often turned out to be a different
thread (shared_mutex::unlock_shared is undefined behavior if called from
a thread that doesn't hold the lock).
Switch to counter to more simply keep track of number of readers and
simply lock/unlock rather than utilizing reader mutex to verify last
freed (and so requiring this matching thread init/destroy behavior).
This fixes a regression caused by us starting to parse types from
declarations. The code in TypeSystemClang was assuming that the value
held in ClangASTMetadata was authoritative, but this isn't (and cannot)
be the case when the type is parsed from a forward-declaration.
For the fix, I add a new "don't know" state to ClangASTMetadata, and
make sure DWARFASTParserClang sets it only when it encounters a forward
declaration. In this case, the type system will fall back to completing
the type.
This does mean that we will be completing more types than before, but
I'm hoping this will offset by the fact that we don't search for
definition DIEs eagerly. In particular, I don't expect it to make a
difference in -fstandalone-debug scenarios, since types will nearly
always be present as definitions.
To avoid this cost, we'd need to create some sort of a back channel to
query the DWARFASTParser about the dynamicness of the type without
actually completing it. I'd like to avoid that if it is not necessary.
Identical PR to: https://github.com/llvm/llvm-project/pull/134563
Previous PR was approved and landed but broke the build due to bad
merge.
Manually resolve the merge conflict and try to land again.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
This reverts commit 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6.
Multiple buildbot failures have been reported:
https://github.com/llvm/llvm-project/pull/134563
The build fails with:
lldb/source/Target/Statistics.cpp:75:39: error: use of undeclared
identifier 'num_symbols_loaded'
This is part of the work proposed in
<https://discourse.llvm.org/t/rfc-speeding-up-dwarf-indexing-again/83979>.
One of the change is that the there will be a different structure for
holding the partial indexes and the final (consolidated) index. To
prepare for this, I'm making the IndexSet structure a template. The
index cache encoding/decoding methods are changed into free functions,
as they only need to know how to work with the final index.
I've moved this functionality to a separate file as all this doesn't
really depend on the rest of the ManualDWARFIndex and it needs to be
public due to its use in the unit test (both of which indicate that it
could be a component of its own).
The check is not correct for discontinuous functions, as one of the
blocks could very well begin before the function entry point. To catch
dead-stripped ranges, I check whether the functions is after the first
known code address. I don't print any error in this case as that is a
common/expected situation.
This avoids many errors like:
```
error: ld-linux-x86-64.so.2 0x00085f3b: adding range [0x0000000000001ae8-0x0000000000001b07) which has a
base that is less than the function's low PC 0x000000000001cfb0. Please file a bug and attach the file at
the start of this error message
```
when debugging binaries on debian trixie because the dynamic linker
(ld-linux) contains discontinuous functions.
If the block ranges is not a subrange of the enclosing block then this
will range will currently be added to the outer block as well (i.e., we
get the same behavior that's currently possible for non-subrange blocks
larger than function_low_pc). However, this code path is buggy and I'd
like to change that (#117725).
Reverts llvm/llvm-project#132274
Broke a test on LLDB Widows on Arm:
https://lab.llvm.org/buildbot/#/builders/141/builds/7726
```
FAIL: test_dwarf (lldbsuite.test.lldbtest.TestExternCSymbols.test_dwarf)
<...>
self.assertTrue(self.res.Succeeded(), msg + output)
AssertionError: False is not true : Command 'expression -- foo()' did not return successfully
Error output:
error: Couldn't look up symbols:
int foo(void)
Hint: The expression tried to call a function that is not present in the target, perhaps because it was optimized out by the compiler.
```
Add ObjectFile::GetObjectName and SymbolFile::GetObjectName to retrieve
the name of the object file, including the `.a` for static libraries.
We currently do something similar in CommandObjectTarget, but the code
for dumping this is a lot more involved than what's being offered by the
new method. We have options to print he full path, the base name, and
the directoy of the path and trim it to a specific width.
This is motivated by #133211, where Greg pointed out that the old code
would print the static archive (the .a file) rather than the actual
object file inside of it.
In #133211, Greg suggested making the rate limit configurable through a
setting. Although adding the setting is easy, the two places where we
currently use rate limiting aren't tied to a particular debugger.
Although it'd be possible to hook up, given how few progress events
currently implement rate limiting, I don't think it's worth threading
this through, if that's even possible.
I still think it's a good idea to be consistent and make it easy to pick
the same rate limiting value, so I've moved it into a constant in the
Progress class.
Emit progress events from SymbolFileDWARFDebugMap. Because we know the
number of OSOs, we can show determinate progress. This is based on a
patch from Adrian, and part of what prompted me to look into improving
how LLDB shows progress events. Before the statusline, all these
progress events would get shadowed and never displayed on the command
line.
This patch addresses the issue #129543.
After this patch DWARFExpression does not call DWARFUnit directly and does not depend on
lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp and a lot of clang code.
After this patch the size of lldb-server binary (Linux Aarch64) is reduced from 47MB to 17MB.
Original PR: #130537
Originally reverted due to revert of dependent commit. Relanding with no
changes.
This changes the MemberPointerType representation to use a
NestedNameSpecifier instead of a Type to represent the base class.
Since the qualifiers are always parsed as nested names, there was an
impedance mismatch when converting these back and forth into types, and
this led to issues in preserving sugar.
The nested names are indeed a better match for these, as the differences
which a QualType can represent cannot be expressed syntatically, and
they represent the use case more exactly, being either dependent or
referring to a CXXRecord, unqualified.
This patch also makes the MemberPointerType able to represent sugar for
a {up/downcast}cast conversion of the base class, although for now the
underlying type is canonical, as preserving the sugar up to that point
requires further work.
As usual, includes a few drive-by fixes in order to make use of the
improvements.
To be able to describe discontinuous functions, this patch changes the
UnwindPlan to accept more than one address range.
I've also squeezed in a couple improvements/modernizations, for example
using the lower_bound function instead of a linear scan.
Original PR: #130537
Reland after updating lldb too.
This changes the MemberPointerType representation to use a
NestedNameSpecifier instead of a Type to represent the base class.
Since the qualifiers are always parsed as nested names, there was an
impedance mismatch when converting these back and forth into types, and
this led to issues in preserving sugar.
The nested names are indeed a better match for these, as the differences
which a QualType can represent cannot be expressed syntatically, and
they represent the use case more exactly, being either dependent or
referring to a CXXRecord, unqualified.
This patch also makes the MemberPointerType able to represent sugar for
a {up/downcast}cast conversion of the base class, although for now the
underlying type is canonical, as preserving the sugar up to that point
requires further work.
As usual, includes a few drive-by fixes in order to make use of the
improvements.
In most places, the rows are copied anyway (because they are generated
by cumulating modifications) immediately after adding them to the unwind
plans. In others, they can be moved into the unwind plan. This lets us
remove some backflip copies and make `const UnwindPlan` actually mean
something.
I've split this patch into two (and temporarily left both APIs) as this
patch was getting a bit big. This patch covers all the interesting
cases. Part two all about converting "architecture default" unwind plans
from ABI and InstructionEmulation plugins.
This patch pushes the error handling boundary for the GetBitSize()
methods from Runtime into the Type and CompilerType APIs. This makes it
easier to diagnose problems thanks to more meaningful error messages
being available. GetBitSize() is often the first thing LLDB asks about a
type, so this method is particularly important for a better user
experience.
rdar://145667239
This reverts commit 6041c745f32e8fd60ed24e29e7d919d8d1c87ca6.
Relands the original patch with the test-case data fixed. Weirldy the PR CI
didn't seem to run the unit-tests? In any case, the problem was an
incorrect expectation in the test-case data. Since we have both public
and internal SDK in that test-case, we should `expect_mismatch` to be
`true`.
