… (#99444)
The previous version introduced a bug (caught by cross-project tests).
Explicit signature resolution is still necessary when one wants to
access the children (not attributes) of a given DIE.
The new version keeps just the findRecursively extension, and reverts
all the DWARFTypePrinter modifications.
This patch fixes .debug_names verification for split DWARF with no type
units. It will print out an error for any name entries where we can't
locate the .dwo file. It finds the non skeleton unit and correctly
figures out the DIE offset in the .dwo file. If the non skeleton unit is
found and yet the skeleton unit has a DWO ID, an error will be emitted
showing we couldn't access the non-skeleton compile unit.
This allows `llvm-dwarfdump` to decode the DWARF 5 opcode
`DW_OP_implicit_pointer` (0xa0). GCC makes use of this opcode in recent
versions. LLVM contains some (unfinished) support as well. With existing
usage in the ecosystem, adding decoding support here seems reasonable.
This sorts DWARF op descriptions in `DWARFExpression.cpp` by opcode and version, packing the standardised ops together. A few ops also had the wrong version listed, so this fixes those versions as well. (The version does not appear to actually be used currently.)
…_sig8
Splitting from #99495.
I've extended the type unit test case to feature more kinds of
references, including the gcc-style DW_AT_type[DW_FORM_ref_sig8]
reference, which this patch fixes.
LLVM Symbolizer attempt to symbolize addresses of optimized binaries
reports missing line numbers for some cases. It maybe due to compiler
which sometimes cannot map an instruction to line number due to
optimizations. Symbolizer should handle those cases gracefully.
Adding an option '--skip-line-zero' to symbolizer so as to report the
nearest non-zero line number.
---------
Co-authored-by: Amit Pandey <amit.pandey@amd.com>
This patch adds support for verifying local type units in .debug_names
section. It adds a test to test if the TU index is valid, and a test
that tests that an error is found inside the name entry for a type unit.
We don't need to test all other errors in the name entry because these
are essentially identical to compile unit entries, they just use a
different DWARF unit offset index.
findRecursively follows DW_AT_specification and DW_AT_abstract_origin
references, but not DW_AT_signature. As far as I can tell, there is no
fundamental difference between these attributes that would make this
behavior desirable, and this just seems like a consequence of the fact
that this attribute is newer. This patch aims to change that.
The motivation is some code in lldb, which assumes that it can construct
a qualified name of a type by just walking the parent chain and looking
at the name attribute. This works for "regular" debug info, even when
some of the DIEs are just forward declarations, but it breaks in the
presence of type units, because of the need to explicitly resolve the
signature reference.
While LLDB does not use the llvm's DWARFDie class (yet?), this seems
like a very important change in the overall API, and any divergence here
would complicate eventual reunification, which is why I am making the
change in the llvm API first. However, putting lldb aside, I think this
change is beneficial in llvm on its own, as it allows us to remove the
explicit DW_AT_signature resolution in the DWARFTypePrinter.
The result of the function cannot be correctly interpreted without
knowing the precise form type (a type signature needs to be looked up
very differently from a supplementary debug info reference). The
function sort of worked because the two reference types (unit-relative
and section-relative) that can be handled uniformly are also the most
common types of references, but this setup made it easy to write code
which does not support other kinds of reference (and if one tried to
support them, the result didn't look pretty --
https://github.com/llvm/llvm-project/pull/97423/files#r1676217081).
The split is based on the reference type classification from DWARFv5
(Section 7.5.5 Classes and Forms), and it should enable uniform (if
slightly more verbose) hadling. Note that this only affects users which
want more control of how (or if) the references are resolved. Users
which just want to access the referenced DIE can use the higher level
API (DWARFDie::GetAttributeValueAsReferencedDie) which returns (or will
return after #97423 is merged) the correct die for all reference types
(except for supplementary references, which we don't support right now).
llvm-gsymutil eats a lot of RAM. On some large binaries, it causes OOM's on smaller hardware, consuming well over 64GB of RAM. This change frees line tables once we're done with them, and frees DWARFUnits's DIE's when we finish processing each DU, though they may get reconstituted if there are references from other DU's during processing. Once the conversion is complete, all DIE's are freed. The reduction in peak memory usage from these changes showed between 7-12% in my tests.
The double-checked locking around the creation & freeing of the data structures was tested on a 166 core system. I validated that it trivially malfunctioned without the locks (and with stupid reordering of the locks) and worked reliably with them.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
DWARFTypePrinter::appendTemplateParameters already skips pointer type
parameters but didn't account for reference type parameters. This would
result in tripping up the RawName assertion below. This updates the
check for both `DW_TAG_pointer_type` and `DW_TAG_reference_type`.
Thanks to Dave Blaikie for helping with the test.
rdar://130297520
This patch adds support for the new foreign type unit support in
.debug_names. Features include:
- don't manually index foreign TUs if we have info for them
- only use the type unit entries that match the .dwo files when we have
a .dwp file
- fix type unit lookups for .dwo files
- fix crashers that happen due to PeekDIEName() using wrong offsets where an entry had DW_IDX_comp_unit and DW_IDX_type_unit entries and when we had no type unit support, it would cause us to think it was a normal DIE in .debug_info from the main executable.
---------
Co-authored-by: paperchalice <liujunchang97@outlook.com>
Motivation: LLDB is able to report errors about these scenarios whereas
LLVM's DWARF parser only gives a boolean success/fail. I want to migrate
LLDB to using LLVM's DWARFUnitHeader class, but I don't want to lose
some of the error reporting, so I'm adding it to the LLVM class first.
Part 1 of fix for issue
https://github.com/llvm/llvm-project/issues/54624
Split from PR #87623. Clang front end changes to follow.
Use DICompositeType to represent the template alias, using its extraData
field as a tuple of DITemplateParameter to describe the template
parameters.
Added template-alias.ll - Check DWARF emission.
Modified frame-types.s - Check llvm-symbolizer understands the DIE.
The parameter of `findDebugNamesOffsets` has been renamed to
`EndOfHeaderOffset` in #88064 to make it clear it is a section offset
instead of an offset relative to the current name index. Rename the call
site variable as well.
When dumping FDEs, `readelf` prints new location values after
`DW_CFA_advance_loc(*)` instructions, which looks quite convenient:
```
> readelf -wf test.o
...
... FDE ... pc=0000000000000030..0000000000000064
DW_CFA_advance_loc: 4 to 0000000000000034
...
DW_CFA_advance_loc: 4 to 0000000000000038
...
```
This patch makes `llvm-dwarfdump` and `llvm-readobj` do the same.
In order to make tooling around dwarf health easier, I've added an `--verify-json` option to `llvm-dwarfdump --verify` that will spit out error summary data with counts to a JSON file.
I've added the same capability to `llvm-gsymutil` in a [different PR.](https://github.com/llvm/llvm-project/pull/81763)
The format of the json is:
``` json
{
"error-categories": {
"<first category description>": {"count": 1234},
"<next category description>": {"count":4321}
},
"error-count": 5555
}
```
for a clean run:
``` json
{
"error-categories": {},
"error-count": 0
}
```
---------
Co-authored-by: Kevin Frei <freik@meta.com>
We recently ran into some bad DWARF where the `DW_AT_stmt_list` of many
compile units was randomly set to invalid values and was causing LLDB to
crash due to an assertion about address sizes not matching. Instead of
asserting, we should return an appropriate recoverable `llvm::Error`.
Refactor the code that calculates the offsets for the various pieces of
the DWARF .debug_names index section, to make it easier to share the
code with other tools, such as LLD.
The DWARF 5 debug_str_offsets section starts with a header, which must
be skipped in order to access the underlying `strp`s.
However, the verifier supports some pre-standardization version of this
section (with the same section name), which does not have a header. In
this case, the offsets start on the first byte of the section. More in
[1] and [2] about this legacy section.
How does The DWARF verifier figure out which version to use? It manually
reads the **first** header in debug_info and uses that. This is wrong
when multiple debug_str_offset sections have been linked together, in
particular it is wrong in the following two cases:
1. A standard DWARF 4 object file (i.e. no debug_str_offsets) linked
with a standard DWARF 5 object file.
2. A non-standard DWARF 4 object file (i.e. containing the header-less
debug_str_offsets section) linked with a standard DWARF 5 object file.
Based on discussions in https://github.com/llvm/llvm-project/pull/81210,
the legacy version is only possible with dwo files, and dwo files cannot
mix the legacy version with the dwarf 5 version. As such, we change the
verifier to only check the debug_info header in the case of dwo files.
If it sees a dwarf 4 version, it handles it the legacy way.
Note: the modified test was technically testing an unsupported
combination of dwarf version + non-dwo sections. To see why, simply note
that the test contained no `debug_info.dwo` sections, so the call to
DWARFObject::forEachInfoDWOSections was doing nothing. We were finding
the error through the "standard version", which shouldn't happen.
[1]: https://gcc.gnu.org/wiki/DebugFission
[2]: https://gcc.gnu.org/wiki/DebugFissionDWP
A line table whose sole entry is an end sequence should not have the
entry's file index verified, as that value corresponds to the initial
value of the state machine, not to a real file index. In DWARF 5, this
is particularly problematic as it uses 0-based indexing, and the state
machine specifies a starting index of 1; in other words, you'd need to
have _two_ files before such index became legal "by default".
A previous attempt to fix this problem was done [1], but it was too
specific in its condition, and did not capture all possible cases where
this issue can happen.
[1]: https://github.com/llvm/llvm-project/pull/77004
The new experimental calling convention preserve_none is the opposite
side of existing preserve_all. It tries to preserve as few general
registers as possible. So all general registers are caller saved
registers. It can also uses more general registers to pass arguments.
This attribute doesn't impact floating-point registers. Floating-point
registers still follow the c calling convention.
Currently preserve_none is supported on X86-64 only. It changes the c
calling convention in following fields:
* RSP and RBP are the only preserved general registers, all other
general registers are caller saved registers.
* We can use [RDI, RSI, RDX, RCX, R8, R9, R11, R12, R13, R14, R15, RAX]
to pass arguments.
It can improve the performance of hot tailcall chain, because many
callee saved registers' save/restore instructions can be removed if the
tail functions are using preserve_none. In my experiment in protocol
buffer, the parsing functions are improved by 3% to 10%.
This stemps from conversatin in:
https://github.com/llvm/llvm-project/pull/77457#discussion_r1457889792.
Right now Abbrev code for abbrev is combination of DIE TAG and other
attributes.
In the future it will be changed to be an index. Since DenseSet does not
preserve an order, added a sort based on abbrev code. Once change to
index is
made, it will print out abbrevs in the order they are stored.
The amount and format of output from `llvm-dwarfdump --verify` makes it
quite difficult to know if a change to a tool that produces or modifies
DWARF is causing new problems, or is fixing existing problems. This diff
adds a categorized summary of issues found by the DWARF verifier, on by
default, at the bottom of the error output.
The change includes a new `--error-display` option with 4 settings:
* `--error-display=quiet`: Only display if errors occurred, but no
details or summary are printed.
* `--error-display=summary`: Only display the aggregated summary of
errors with no error detail.
* `--error-display=details`: Only display the detailed error messages
with no summary (previous behavior)
* `--error-display=full`: Display both the detailed error messages and
the aggregated summary of errors (the default)
I changed a handful of tests that were failing due to new output, adding
the flag to use the old behavior for all but a couple. For those two I
added the new aggregated output to the expected output of the test.
The `OutputCategoryAggregator` is a pretty simple little class that
@clayborg suggested to allow code to only be run to dump detail if it's
enabled, while still collating counts of the category. Knowing that the
lambda passed in is only conditionally executed is pretty important
(handling errors has to be done *outside* the lambda). I'm happy to move
this somewhere else (and change/improve it) to be more broadly useful if
folks would like.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
TableEntry names are pointers into the string table section, and
accessing their
length requires a search for `\0`. However, 99% of the time we only need
to
compare the name against some other other, and such a comparison will
fail as
early as the first character.
This commit adds a method to the interface of TableEntry so that such a
comparison can be done without extracting the full name. It saves 10% in
the
time (1250ms -> 1100 ms) to evaluate the following expression.
```
lldb \
--batch \
-o "b CodeGenFunction::GenerateCode" \
-o run \
-o "expr Fn" \
-- \
clang++ -c -g test.cpp -o /dev/null &> output
```
The current implementation of DebugNames is _only_ using hashes to
compute the bucket number. Once inside the bucket, it reverts back to
string comparisons, even though not all hashes inside a bucket are
identical.
This commit changes the behavior so that we check the hash before
comparing strings. Such check is so important that it speeds up a simple
benchmark by 20%. In other words, the following expression evaluation
time goes from 1100ms to 850ms.
```
bin/lldb \
--batch \
-o "b CodeGenFunction::GenerateCode" \
-o run \
-o "expr Fn" \
-- \
clang++ -c -g test.cpp -o /dev/null &> output
```
(Note, these numbers are considering the usage of IDX_parent)
This commit introduces a helper function to DWARFAcceleratorTable::Entry
which follows DW_IDX_Parent attributes to returns the corresponding
parent Entry in the table.
It is tested by enhancing dwarfdump so that it now prints:
1. When data is corrupt.
2. When parent information is present, but the parent is not indexed.
3. The parent entry offset, when the parent is present and indexed. This
is printed in terms a real entry offset (the same that gets printed at
the start of each entry: "Entry @ 0x..."), instead of the encoded number
in the table (which is an offset from the start off the Entry list).
This makes it easy to visually inspect the dwarfdump and check what the
parent is.
This implements the ideas discussed in [1].
To summarize, this commit changes AsmPrinter so that it outputs
DW_IDX_parent information for debug_name entries. It will enable
debuggers to speed up queries for fully qualified types (based on a
DWARFDeclContext) significantly, as debuggers will no longer need to
parse the entire CU in order to inspect the parent chain of a DIE.
Instead, a debugger can simply take the parent DIE offset from the
accelerator table and peek at its name in the debug_info/debug_str
sections.
The implementation uses two types of DW_FORM for the DW_IDX_parent
attribute:
1. DW_FORM_ref4, which points to the accelerator table entry for the
parent.
2. DW_FORM_flag_present, when the entry has a parent that is not in the
table (that is, the parent doesn't have a name, or isn't allowed to be
in the table as per the DWARF spec). This is space-efficient, since it
takes 0 bytes.
The implementation works by:
1. Changing how abbreviations are encoded (so that they encode which
form, if
any, was used to encode IDX_Parent)
2. Creating an MCLabel per accelerator table entry, so that they may be
referred by IDX_parent references.
When all patches related to this are merged, we are able to show that
evaluating an expression such as:
```
lldb --batch -o 'b CodeGenFunction::GenerateCode' -o run -o 'expr Fn' -- \
clang++ -c -g test.cpp -o /dev/null
```
is far faster: from ~5000 ms to ~1500ms.
Building llvm-project + clang with and without this patch, and looking
at its impact on object file size:
```
ls -la $(find build_stage2_Debug_idx_parent_assert_dwarf5 -name \*.cpp.o) | awk '{s+=$5} END {printf "%\047d\n", s}'
11,507,327,592
-la $(find build_stage2_Debug_no_idx_parent_assert_dwarf5 -name \*.cpp.o) | awk '{s+=$5} END {printf "%\047d\n", s}'
11,436,446,616
```
That is, an increase of 0.62% in total object file size.
Looking only at debug_names:
```
$stage1_build/bin/llvm-objdump --section-headers $(find build_stage2_Debug_idx_parent_assert_dwarf5 -name \*.cpp.o) | grep __debug_names | awk '{s+="0x"$3} END {printf "%\047d\n", s}'
440,772,348
$stage1_build/bin/llvm-objdump --section-headers $(find build_stage2_Debug_no_idx_parent_assert_dwarf5 -name \*.cpp.o) | grep __debug_names | awk '{s+="0x"$3} END {printf "%\047d\n", s}'
369,867,920
```
That is an increase of 19%.
DWARF Linkers need to be changed in order to support this. This commit
already brings support to "base" linker, but it does not attempt to
modify the parallel linker. Accelerator entries refer to the
corresponding DIE offset, and this patch also requires the parent DIE
offset -- it's not clear how the parallel linker can access this. It may
be obvious to someone familiar with it, but it would be nice to get help
from its authors.
[1]:
https://discourse.llvm.org/t/rfc-improve-dwarf-5-debug-names-type-lookup-parsing-speed/74151/
Allow the dumping of .dwo files contents to show up when dumping an
executable with split DWARF.
Currently if you run llvm-dwarfdump on a binary that has skeleton
compile units, you only see the skeleton compile units. Since the main
binary has the linked addresses it would be nice to be able to dump
DWARF from the .dwo files and how the resolved addresses instead of
showing the address index and "<unresolved>" in the output. This patch
adds an option that can be specified to dump the non skeleton DIEs named
--dwo.
Added the ability to use the following options with split dwarf as well:
--name <name>
--lookup <addr>
--debug-info <die-offset>
If there is no file name in the prologue of a line table, the verifier
will try to verify the file index, which will be set to 1 by default.
This will cause the DWARF verifier to throw an error even if there is no
error.
rdar://114476503
rdar://114343624
This is a follow up patch after .debug_names can now emit local type
unit entries when we compile with type units + DWARF5 + .debug_names.
The pull request that added this functionality was:
https://github.com/llvm/llvm-project/pull/70515
This patch makes sure that the DebugNamesDWARFIndex in LLDB will not
manually need to parse type units if they have a valid index. It also
fixes the index to be able to correctly extract name entries that
reference type unit DIEs. Added a test to verify things work as
expected.
