This is all the tests and fixes I've had percolating since my first
attempt at this in January. After 6 months of trying, I've given up on
adding the ability to test DWP files in LLDB API tests. I've left both
the tests (disabled) and the changes to Makefile.rules in place, in the
hopes that someone who can configure the build bots will be able to
enable the tests once a non-borked dwp tool is widely available.
Other than disabling the DWP tests, this continues to be the same diff
that I've tried to land and
[not](https://github.com/llvm/llvm-project/pull/90622)
[revert](https://github.com/llvm/llvm-project/pull/87676)
[five](https://github.com/llvm/llvm-project/pull/86812)
[times](https://github.com/llvm/llvm-project/pull/85693)
[before](https://github.com/llvm/llvm-project/pull/96802). There are a
couple of fixes that the testing exposed, and I've abandoned the DWP
tests because I want to get those fixes finally upstreamed, as without
them DebugInfoD is less useful.
Reverts llvm/llvm-project#96802
Attempt #5 fails. It's been 6 months. I despise Makefile.rules and have
no ability to even *detect* these failures without _landing_ a diff. In
the mean time, we have no testing for DWP files at all (and a regression
that was introduced, that I fix with this diff) so I'm going to just
remove some of the tests and try to land it again, but with less testing
I guess.
This is the same diff I've put up at many times before. I've been trying
to add some brand new functionality to the LLDB test infrastucture
(create split-dwarf files!), and we all know that no good deed goes
unpunished. The last attempt was reverted because it didn't work on the
Fuchsia build.
There are no code differences between this and
[the](https://github.com/llvm/llvm-project/pull/90622)
[previous](https://github.com/llvm/llvm-project/pull/87676)
[four](https://github.com/llvm/llvm-project/pull/86812)
[diffs](https://github.com/llvm/llvm-project/pull/85693) landed &
reverted (due to testing infra failures). The only change in this one is
the way `dwp` is being identified in `Makefile.rules`.
Thanks to @petrhosek for helping me figure out how the fuchsia builders
are configured. I now prefer to use llvm-dwp and fall back to gnu's dwp
if the former isn't found. Hopefully this will work everywhere it needs
to.
Right now, ParseStructureLikeDIE begins the class definition (which
amounts to parsing the opening "{" of a class and promising to be able
to fill it in later) if it finds a definition DIE.
This makes sense in the current setup, where we eagerly search for the
definition die (so that we will either find it in the beginning or don't
find it at all), but with delayed definition searching (#92328), this
created an (in my view, undesirable) inconsistency, where the final
state of the type (whether it has begun its definition) depended on
whether we happened to start out with a definition DIE or not.
This patch attempts to pre-emptively rectify that by establishing a new
invariant: the definition is never started eagerly. It can only be
started in one of two ways:
- we're completing the type, in which case we will start the definition,
parse everything and immediately finish it
- we need to parse a member (typedef, nested class, method) of the class
without needing the definition itself. In this case, we just start the
definition to insert the member we need.
Besides the delayed definition search, I believe this setup has a couple
of other benefits:
- It treats ObjC and C++ classes the same way (we were never starting
the definition of those)
- unifies the handling of types that types that have a definition and
those that do. When adding (e.g.) a nested class we would previously be
going down a different code path depending on whether we've found a
definition DIE for that type. Now, we're always taking the
definition-not-found path (*)
- it reduces the amount of time a class spends in the funny "definition
started". Aside from the addition of stray addition of nested classes,
we always finish the definition right after we start it.
(*) Herein lies a danger, where if we're missing some calls to
PrepareContextToReceiveMembers, we could trigger a crash when
trying to add a member to the not-yet-started-to-be-defined classes.
However, this is something that could happen before as well (if we
did not have a definition for the class), and is something that
would be exacerbated by #92328 (because it could happen even if we
the definition exists, but we haven't found it yet). This way, it
will at least happen consistently, and the fix should consist of
adding a PrepareContextToReceiveMembers in the appropriate place.
This is a regression from #96484 caught by @ZequanWu.
Note that we will still create separate enum types for types parsed from
two definitions. This is different from how we handle classes, but it is
not a regression.
I'm also adding the DieToType check to the class type parsing code,
although in this case, the type uniqueness should be enforced by the
UniqueDWARFASTType map.
If ParseStructureLikeDIE (or ParseEnum) encountered a declaration DIE,
it would call FindDefinitionTypeForDIE. This returned a fully formed
type, which it achieved by recursing back into ParseStructureLikeDIE
with the definition DIE.
This obscured the control flow and caused us to repeat some work (e.g.
the UniqueDWARFASTTypeMap lookup), but it mostly worked until we tried
to delay the definition search in #90663. After this patch, the two
ParseStructureLikeDIE calls were no longer recursive, but rather the
second call happened as a part of the CompleteType() call. This opened
the door to inconsistencies, as the second ParseStructureLikeDIE call
was not aware it was called to process a definition die for an existing
type.
To make that possible, this patch removes the recusive type resolution
from this function, and leaves just the "find definition die"
functionality. After finding the definition DIE, we just go back to the
original ParseStructureLikeDIE call, and have it finish the parsing
process with the new DIE.
While this patch is motivated by the work on delaying the definition
searching, I believe it is also useful on its own.
This patch adds support for the new foreign type unit support in
.debug_names. Features include:
- don't manually index foreign TUs if we have info for them
- only use the type unit entries that match the .dwo files when we have
a .dwp file
- fix type unit lookups for .dwo files
- fix crashers that happen due to PeekDIEName() using wrong offsets where an entry had DW_IDX_comp_unit and DW_IDX_type_unit entries and when we had no type unit support, it would cause us to think it was a normal DIE in .debug_info from the main executable.
---------
Co-authored-by: paperchalice <liujunchang97@outlook.com>
- move type insertion from individual parse methods into
ParseTypeFromDWARF
- optimize sentinel (TYPE_IS_BEING_PARSED) insertion to avoid double map
lookup
- as this requires the map to not have nullptr values, I've replaced all
`operator[]` queries with calls to `lookup`.
Our dwarf parsing code treats structures and classes as interchangable.
CompilerContextKind is used when looking DIEs for types. This makes sure
we always they're treated the same way.
See also
[#95905#discussion_r1645686628](https://github.com/llvm/llvm-project/pull/95905#discussion_r1645686628).
With simple template names the template arguments aren't embedded in the
DW_AT_name attribute of the type. The code in
FindDefinitionTypeForDWARFDeclContext was comparing the synthesized
template arguments on the leaf (most deeply nested) DIE, but was not
sufficient, as the difference get be at any level above that
(Foo<T>::Bar vs. Foo<U>::Bar). This patch makes sure we compare the
entire context.
As a drive-by I also remove the completely unnecessary
ConstStringification of the GetDIEClassTemplateParams result.
…ARFDIE
This puts them closer to the other two functions doing something very
similar. I've tried to stick to the original logic of the functions as
much as possible, though I did apply some easy simplifications.
The changes in DWARFDeclContext.h are there to make the unit tests
produce more useful error messages.
This is an attempt at displaying the work that's being done by LLDB when waiting on type-completion events, e.g., when running an expression. This patch adds a single new progress event for cases where we search for the definition DIE of a forward declaration, which can be an expensive operation in the presence of many object files.
This patch moves some of the `is_cxx_method`/`objc_method` logic out of
`DWARFASTParserClang::ParseSubroutine` into their own functions. Mainly
the purpose of this is to (hopefully) make this function more readable
by turning the deeply nested if-statements into early-returns. This will
be useful in an upcoming change where we remove some of the branches of
said if-statement.
Considerations:
* Would be nice to make them into static helpers in
`DWARFASTParserClang.cpp`. That would require them take few more
arguments which seemed to get unwieldy.
* `HandleCXXMethod` can return three states: (1) found a `TypeSP` we
previously parsed (2) successfully set a link between the DIE and
DeclContext (3) failure. One could express this with
`std::optional<TypeSP>`, but then returning `std::nullopt` vs `nullptr`
becomes hard to reason about. So I opted to return `std::pair<bool,
TypeSP>`, where the `bool` indicates success and the `TypeSP` the cached
type.
* `HandleCXXMethod` takes `ignore_containing_context` as an output
parameter. Haven't found a great way to do this differently
The function that calculated the declaration context for a DIE was incorrectly transparently traversing acrosss DW_TAG_subprogram dies when climbing the parent DIE chain. This meant that types defined in functions would appear to have the declaration context of anything above the function. I fixed the GetTypeLookupContextImpl(...) function in DWARFDIE.cpp to not transparently skip over functions, lexical blocks and inlined functions and compile and type units. Added a test to verify things are working.
`GetDeclContextDIEs` and `DIEDeclContextsMatch` are unused (possibly
since we added support for simplified template names, but I haven't
checked). `GetDeclContextDIEs` is also very similar (but subtly
different) from `GetDeclContext` and `GetTypeLookupContext`.
I am keeping `GetParentDeclContextDIE` as that one still has some
callers, but I want to look into the possibility of merging it with at
least one of the functions mentioned above.
This makes sure we try to process declaration DIEs that are erroneously
present in the index. Until bd5c6367bd7, clang was emitting index
entries for declaration DIEs with DW_AT_signature attributes. This makes
sure to avoid returning those DIEs as the definitions of a type, but
also makes sure to pass through DIEs referring to static constexpr
member variables, which is a (probably nonconforming) extension used by
dsymutil.
It adds test cases for both of the scenarios. It is essentially a
recommit of #91808.
and two follow-up commits. The reason is the crash we've discovered when
processing -gsimple-template-names binaries. I'm committing a minimal
reproducer as a separate patch.
This reverts the following commits:
- 51dd4eaaa29683c16151f5168e7f8645acbd6e6c (#92328)
- 3d9d48523977af3590f7dd0edfd258454cb9e9cf (#93839)
- afe6ab7586f7078cc410f6162bd9851e48e2a286 (#94400)
Change the signature of `DWARFExpression::Evaluate` and
`DWARFExpressionList::Evaluate` to return an `llvm::Expected` instead of a
boolean. This eliminates the `Status` output parameter and generally improves
error handling.
This is a follow-up of
https://github.com/llvm/llvm-project/pull/92328#issuecomment-2145849441
Clang attaches `DW_AT_declaration` to static inline data members and
`dsymutil` indexes these constants. Skipping these caused the expression
evaluator to fail to find such constants when using DWARFv5.
Fixes `TestConstStaticIntegralMember.py` on DWARFv5.
This fixes
https://github.com/llvm/llvm-project/pull/92328#issuecomment-2139339444
by not differentiating `DW_TAG_class_type` and `DW_TAG_structure_type`
in `UniqueDWARFASTTypeList`, because it's possible that DIE for a type
is `DW_TAG_class_type` in one CU but is `DW_TAG_structure_type` in a
different CU.
---------
Co-authored-by: Michael Buch <michaelbuch12@gmail.com>
DWARFDebugInfo doesn't know how to resolve the "file_index" component of
a DIERef. This patch removes GetUnit (in favor of existing
GetUnitContainingDIEOffset) and changes GetDIE to take only the
components it actually uses.
After a bug (the bug is that the functions don't handle DW_AT_signature,
aka type units) led me to one of these similar-but-different functions,
I started to realize that most of the differences between these two
functions are actually bugs.
As a first step towards merging them, this patch rewrites both of them
to follow the same pattern, while preserving all of their differences.
The main change is that GetTypeLookupContext now also uses a `seen` list
to avoid reference loops (currently that's not necessary because the
function strictly follows parent links, but that will change with
DW_AT_signatures).
I've also optimized both functions to avoid recursion by starting contruction
with the deepest scope first (and then reversing it).
DebugNamesDWARFIndex was jumping through hoops to construct a DIERef
from an index entry only to jump through them back a short while later
to construct a DWARFDIE.
This used to be necessary as the index lookup was a two stage process,
where we first enumerated all matches, and then examined them (so it was
important that the enumeration was cheap -- does not trigger unnecessary
parsing). However, now that the processing is callback based, we are
always immediately examining the DWARFDIE right after finding the entry,
and the DIERef just gets in the way.
This reapplies
9a7262c260
(#90663) and added https://github.com/llvm/llvm-project/pull/91808 as a
fix.
It was causing tests on macos to fail because
`SymbolFileDWARF::GetForwardDeclCompilerTypeToDIE` returned the map
owned by this symol file. When there were two symbol files, two
different maps were created for caching from compiler type to DIE even
if they are for the same module. The solution is to do the same as
`SymbolFileDWARF::GetUniqueDWARFASTTypeMap`: inquery
SymbolFileDWARFDebugMap first to get the shared underlying SymbolFile so
the map is shared among multiple SymbolFileDWARF.
We currently cannot represent abbreviation codes with more than 16 bits,
and we were lldb-asserting if we ever ran into one. While I haven't seen
any real DWARF with these kinds of abbreviations, it is possible to hit
this with handcrafted evil dwarf, due some sort of corruptions, or just
bugs (the addition of PeekDIEName makes these bugs more likely, as the
function blindly dereferences offsets within the debug info section) .
Missing abbreviations were already reporting an error. This patch turns
sure that large abbreviations into an error as well, and adds a test for
both cases.
Here we go with attempt number five. Again, no changes to the LLDB code
diff, which has been reviewed several times.
For the tests, I added a `@skipIfCurlSupportMissing` annotation so that
the Debuginfod mocked server stuff won't run, and I also disabled
non-Linux/FreeBSD hosts altogether, as they fail for platform reasons on
macOS and Windows. In addition, I updated the process for extracting the
GNU BuildID to no create a target, per some feedback on the previous
diff.
For reference, previous PR's (landed, backed out after the fact for
various reasons) #90622, #87676, #86812, #85693
---------
Co-authored-by: Kevin Frei <freik@meta.com>
DWARFDebugInfo only knows how to resolve references in its own file, but
in split dwarf, the index entries will refer to DIEs in the separate
(DWO) file. To resolve the DIERef correctly we'd either need to go
through the SymbolFileDWARF to get the full logic for resolving a
DIERef, or use the fact that ToDIERef already looks up the correct unit
while computing its result.
This patch does the latter.
This bug manifested itself in not being able to find type definitions
for types in namespaces, so I've modified one of our type resolving test
cases to run with debug_names, and added a namespaced class into it (it
originally contained only a top-level class).
This is the implementation for
https://discourse.llvm.org/t/rfc-delay-definition-die-searching-when-parse-a-declaration-die-for-record-type/78526.
#### Motivation
Currently, lldb eagerly searches for definition DIE when parsing a
declaration DIE for struct/class/union definition DIE. It will search
for all definition DIEs with the same unqualified name (just
`DW_AT_name` ) and then find out those DIEs with same fully qualified
name. Then lldb will try to resolve those DIEs to create the Types from
definition DIEs. It works fine most time. However, when built with
`-gsimple-template-names`, the search graph expands very quickly,
because for the specialized-template classes, they don’t have template
parameter names encoded inside `DW_AT_name`. They have
`DW_TAG_template_type_parameter` to reference the types used as template
parameters. In order to identify if a definition DIE matches a
declaration DIE, lldb needs to resolve all template parameter types
first and those template parameter types might be template classes as
well, and so on… So, the search graph explodes, causing a lot
unnecessary searching/type-resolving to just get the fully qualified
names for a specialized-template class. This causes lldb stack overflow
for us internally on template-heavy libraries.
#### Implementation
Instead of searching for definition DIEs when parsing declaration DIEs,
we always construct the record type from the DIE regardless if it's
definition or declaration. The process of searching for definition DIE
is refactored to `DWARFASTParserClang::FindDefinitionTypeForDIE` which
is invoked when 1) completing the type on
`SymbolFileDWARF::CompleteType`. 2) the record type needs to start its
definition as a containing type so that nested classes can be added into
it in `PrepareContextToReceiveMembers`.
The key difference is `SymbolFileDWARF::ResolveType` return a `Type*`
that might be created from declaration DIE, which means it hasn't starts
its definition yet. We also need to change according in places where we
want the type to start definition, like `PrepareContextToReceiveMembers`
(I'm not aware of any other places, but this should be a simple call to
`SymbolFileDWARF::FindDefinitionDIE`)
#### Result
It fixes the stack overflow of lldb for the internal binary built with
simple template name. When constructing the fully qualified name built
with `-gsimple-template-names`, it gets the name of the type parameter
by resolving the referenced DIE, which might be a declaration (we won't
try to search for the definition DIE to just get the name).
I got rough measurement about the time using the same commands (set
breakpoint, run, expr this, exit). For the binary built without
`-gsimple-template-names`, this change has no impact on time, still
taking 41 seconds to complete. When built with
`-gsimple-template-names`, it also takes about 41 seconds to complete
wit this change.
Dwarf 5 says "There is no requirement that the entries be ordered in any
particular way" in 2.17.3 Non-Contiguous Address Ranges for rnglist.
Some places assume the ranges are already sorted but it's not.
For example, when [parsing function
info](bc8a427620/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp (L922-L927)),
it validates low and hi address of the function: GetMinRangeBase returns
the first range entry base and GetMaxRangeEnd returns the last range
end. If low >= hi, it stops parsing this function. This causes missing
inline stack frames for those functions.
This change fixes it and updates the test
`lldb/test/Shell/SymbolFile/DWARF/x86/debug_rnglists.s` so that two
ranges in `.debug_rnglists` are out of order and `image lookup -v -s
lookup_rnglists` is still able to produce sorted ranges for the inner
block.
The high level goal is to have 1 way of converting a DW_TAG value into a
human-readable string.
There are 3 ways this change accomplishes that:
1.) Changing DW_TAG_value_to_name to not create custom error strings.
The way it was doing this is error-prone: Specifically, it was using a
function-local static char buffer and handing out a pointer to it.
Initialization of this is thread-safe, but mutating it is definitely
not. Multiple threads that want to call this function could step on
each others toes. The implementation in this patch sidesteps the issue
by just returning a StringRef with no mention of the tag value in it.
2.) Changing all uses of DW_TAG_value_to_name to log the value of the
tag since the function doesn't create a string with the value in it
anymore.
3.) Removing `DWARFBaseDIE::GetTagAsCString()`. Callers should call
DW_TAG_value_to_name on the tag directly.
And "LLDB Debuginfod tests and a fix or two (#90622)".
f8fedfb6802173372ec923f99f31d4af810fbcb0 /
2d4acb086541577ac6ab3a140b9ceb9659ce7094
As it has caused a test failure on 32 bit Arm:
https://lab.llvm.org/buildbot/#/builders/17/builds/52580
Expr/TestStringLiteralExpr.test. The follow up did fix
lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
but not the other failure.
I'm taking yet another swing at getting these tests going, on the
hypothesis that the problems with buildbots & whatnot are because
they're not configured with CURL support, which I've confirmed would
cause the previous tests to fail. (I have no access to an ARM64 linux
system, but I did repro the failure on MacOS configured without CURL
support)
So, the only difference between this diff and
[previous](https://github.com/llvm/llvm-project/pull/85693)
[diffs](https://github.com/llvm/llvm-project/pull/87676) that have
already been approved is that I've added a condition to the tests to
only run if Debuginfod capabilities should be built into the binary. I
had done this for these tests when they were [Shell
tests](https://github.com/llvm/llvm-project/pull/79181) and not API
tests, but I couldn't find a direct analog in any API test, so I used
the "plugins" model used by the intel-pt tests as well.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Depends on #84384 and #90329
This adds support for `DW_TAG_LLVM_ptrauth_type` entries corresponding
to explicitly signed types (e.g. free function pointers) in lldb user
expressions. Applies PR https://github.com/apple/llvm-project/pull/8239
from Apple's downstream and also adds tests and related code.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Jonas upstreamed recognition of DW_TAG_LLVM_ptrauth_type
https://reviews.llvm.org/D130215 but it isn't recognized as a type
qualifier tag in DWARFASTParserClang::ParseTypeFromDWARF.
This removes `m_forward_decl_die_to_compiler_type` which is a map from
`const DWARFDebugInfoEntry *` to `lldb::opaque_compiler_type_t`. This
map is currently used in `DWARFASTParserClang::ParseEnum` and
`DWARFASTParserClang::ParseStructureLikeDIE` to avoid creating duplicate
CompilerType for the specific DIE. But before entering these two
functions in `DWARFASTParserClang::ParseTypeFromDWARF`, we already
checked with `SymbolFileDWARF::GetDIEToType()` if we have a Type created
from this DIE to avoid trying to parse the same DIE twice. So, this map
is unnecessary and not useful.
The couple other places that use the oso module's SymbolFile, they check
that it's non-null, so this is just an oversight in
ResolveSymbolContext.
I didn't add a test for this (but did add a log message for the error
case) because I can't see how this would actually happen. The .o file
had to have valid enough DWARF that the linker's parser could handle it
or it would not be in the debug map. If you delete the .o file, we just
leave that entry out of the debug map. If you strip it or otherwise mess
with it, we'll notice the changed mod time and refuse to read it...
This was based on a report from the field, and we don't have access to
the project. But if the logging tells me how this happened I can come
back and add a test with that example.
This reverts commit d6713ad80d6907210c629f22babaf12177fa329c.
This changed was reverted because of greendragon failures such
as
Unresolved Tests (2):
lldb-api :: debuginfod/Normal/TestDebuginfod.py
lldb-api :: debuginfod/SplitDWARF/TestDebuginfodDWP.py
I believe I've got the tests properly configured to only run on Linux
x86(_64), as I don't have a Linux AArch64/Arm device to diagnose what's
going wrong with the tests (I suspect there's some issue with generating
`.note.gnu.build-id` sections...)
The actual code fixes have now been reviewed 3 times:
https://github.com/llvm/llvm-project/pull/79181 (moved shell tests to
API tests), https://github.com/llvm/llvm-project/pull/85693 (Changed
some of the testing infra), and
https://github.com/llvm/llvm-project/pull/86812 (didn't get the tests
configured quite right). The Debuginfod integration for symbol
acquisition in LLDB now works with the `executable` and `debuginfo`
Debuginfod network requests working properly for normal, `objcopy
--only-keep-debug` stripped, split-dwarf, and `objcopy
--only-keep-debug` stripped *plus* split-dwarf symbols/binaries.
The reasons for the multiple attempts have been tests on platforms I
don't have access to (Linux AArch64/Arm + MacOS x86_64). I believe I've
got the tests properly disabled for everything except for Linux x86(_64)
now. I've built & tested on MacOS AArch64 and Linux x86_64.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
The previous diff (and it's subsequent fix) were reverted as the tests
didn't work properly on the AArch64 & ARM LLDB buildbots. I made a
couple more minor changes to tests (from @clayborg's feedback) and
disabled them for non Linux-x86(_64) builds, as I don't have the ability
do anything about an ARM64 Linux failure. If I had to guess, I'd say the
toolchain on the buildbots isn't respecting the `-Wl,--build-id` flag.
Maybe, one day, when I have a Linux AArch64 system I'll dig in to it.
From the reverted PR:
I've migrated the tests in my
https://github.com/llvm/llvm-project/pull/79181 from shell to API (at
@JDevlieghere's suggestion) and addressed a couple issues that were
exposed during testing.
The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when DebugInfoD returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
