This reverts commit 348374028970c956f2e49ab7553b495d7408ccd9 because #128619 doesn't handle the case when we have an empty frame from `getInliningInfoForAddress` because line num is 0 which makes it non-differentiable from missing debug info. So, we end up using the base filename from symtab again. Reverting for now until that issus is solved.
This PR makes verification of .debug_names acceleration table
multithreaded. In local testing it improves verification of clang
.debug_names from four minutes to under a minute.
This PR relies on a current mechanism of extracting DIEs into a vector.
Future improvements can include creating API to extract one DIE at a
time, or grouping Entires into buckets by CUs and extracting before
parallel step.
Single Thread
4:12.37 real, 246.88 user, 3.54 sys, 0 amem,10232004 mmem
Multi Thread
0:49.40 real, 612.84 user, 515.73 sys, 0 amem, 11226292 mmem
This land commits 23aca2f88dd5d2447e69496c89c3ed42a56f9c31 and
1b15a89a23c631a8e2d096dad4afe456970572c0.
https://github.com/llvm/llvm-project/pull/128619 makes symbolizer to
always use debug info when available so we can reland this chagnge.
To reland https://github.com/llvm/llvm-project/pull/124846, we need to
make symbolizer consistent with the case when line number is 0. Always
using filename and line from debug info even if the line number is 0
sounds like the reasonable path to go.
Currently, `DIContext::getLineInfoForAddress` and
`DIContext::getLineInfoForDataAddress` returns empty DILineInfo when the
debug info is missing for the given address. This is not differentiable
with the case when debug info is found for the given address but the
debug info is default value (filename:linenum is <invalid>:0).
This change wraps the return types of `DIContext::getLineInfoForAddress`
and `DIContext::getLineInfoForDataAddress` with `std::optional`.
There is a bug in `llvm/lib/DebugInfo/GSYM/DwarfTransformer.cpp` where
`StmtSeqVal` was being compared against `UINT32_MAX` rather than the
correct `UINT64_MAX` - thanks @nocchijiang for [pointing this
out](https://github.com/llvm/llvm-project/pull/129196#discussion_r1986203058).
We correct the issue with this patch. For testing - the issue would show
when we have a correct offset of value `UINT32_MAX` - but constructing
such a test is impractical.
To help make better use of dwarfdump verification for identifying and
fixing issues with debug information, the JSON will now emit details
(sub-categories) where relevant. First modification concerns missing
tags as those were recently missing for BOLT debug names.
Test:
test files for JSON output were previously added, so modify here to
expect the new JSON keys. One test has sub-categories and another is
empty.
ninja check-llvm-tools-llvm-dwarfdump
Also build the tool and run with a local executable to verify.
ninja llvm-dwarfdump
This patch introduces support for the `DW_AT_LLVM_stmt_sequence`
attribute in the GSYM DWARF transformer. With this change, the DWARF
GSYM creation process can now accurately associate debug information
with the correct functions, even when multiple functions have been
merged together.
The `macho-gsym-merged-callsites-dsym.yaml` test data is regenerated to
include the fixes in the DWARF linker
(https://github.com/llvm/llvm-project/pull/128953) and the test is
updated to check that debug data is correctly associated for merged
functions.
This commit creates an inconsistency on `__sanitizer_symbolize_pc` API. Before this change, this API always uses the filename from debug info when the line number is 0. After this change, the filename becomes invalid when line number is 0. The sanitizer might fall back to use base filename from symbol table. So, this API may return either `??:0` or `{filename from symbol table}:0` depending on if the symbol table has the filename for it. Make sure this inconsistency is resolved before relanding the commit.
It was discovered that BOLT had several distinct issues of missing debug
information by various tags for debug names (119493 & 119023 as
examples), but the verification of a DWARF with llvm-dwarfdump prior to
those fixes only gave one 'missing name' category.
```
{"error-categories":{"Name Index DIE entry missing name":{"count":36355210}},"error-count":36355210}
```
To more easily leverage dwarf verification for debug health, the JSON
output will be improved to allow having detailed counts by a
sub-category when it makes sense.
For now, this is only implemented on the missing tags, but can be
extended to more.
```
{"error-categories":{"Name Index DIE entry missing name":{"count":10,"details":{"DW_TAG_inlined_subroutine":1,"DW_TAG_label":1,"DW_TAG_namespace":2,"DW_TAG_subprogram":2,"DW_TAG_variable":4}}},"error-count":10}
```
This diff also modifies the tests created in pull request 124936 (not
yet landed) to ensure the JSON switches. Ideally this lands after that
but it did not correctly create a stack of pull requests.
This can occur due to linker ICF and stable sort will ensure the results
are stable.
No explicit/new test coverage, because nondeterminism is non-testable.
It should already be covered by the DWARFDebugLineTest that was failing
some internal testing on an ARM machine which might've been what changed
the sort order. But `llvm::sort` also deliberately randomizes the
contents (under EXPENSIVE_CHECKS) so I'd have expected failures to show
up in any EXPENSIVE_CHECKS Build...
**Summary**
Add support for filtering line table entries based on
`DW_AT_LLVM_stmt_sequence` attribute when looking up address ranges.
This ensures that line entries are correctly attributed to their
corresponding functions, even when multiple functions share the same
address range due to optimizations.
**Background**
In https://github.com/llvm/llvm-project/pull/110192 we added support to
clang to generate the `DW_AT_LLVM_stmt_sequence` attribute for
`DW_TAG_subprogram`'s. Corresponding RFC: [New DWARF Attribute for
Symbolication of Merged
Functions](https://discourse.llvm.org/t/rfc-new-dwarf-attribute-for-symbolication-of-merged-functions/79434)
The `DW_AT_LLVM_stmt_sequence` attribute allows accurate attribution of
line number information to their corresponding functions, even in
scenarios where functions are merged or share the same address space due
to optimizations like Identical Code Folding (ICF) in the linker.
**Implementation Details**
The patch modifies `DWARFDebugLine::lookupAddressRange` to accept an
optional DWARFDie parameter. When provided, the function checks if the
`DIE` has a `DW_AT_LLVM_stmt_sequence` attribute. This attribute
contains an offset into the line table that marks where the line entries
for this DIE's function begin.
If the attribute is present, the function filters the results to only
include line entries from the sequence that starts at the specified
offset. This ensures that even when multiple functions share the same
address range, we return only the line entries that actually belong to
the function represented by the DIE.
The implementation:
- Adds an optional DWARFDie parameter to lookupAddressRange
- Extracts the `DW_AT_LLVM_stmt_sequence` offset if present
- Modifies the address range lookup logic to filter sequences based on
their offset
- Returns only line entries from the matching sequence
Since line zero means "no line information", when symbolizing a location
(an address or an inline frame associated with the address) that has a
line zero location, we shouldn't include other irrelevant data (like
filename) in the result.
This update introduces the ability to filter merged functions during
lookups based on regex patterns derived from call site information in a
previous call to `llvm-gsymutil`. The regex patterns, extracted from
call sites, can then be passed to subsequent calls using the
`--merged-functions-filter` option along with `--merged-functions` and
`--address` (or `--addresses-from-stdin`). This allows for precise
filtering of functions during lookups, giving accurate results for call
stacks that contain merged functions.
We were seeing occasional test failures with expensive checks enabled.
The issue was tracked down to a `sort` which should instead be a
`stable_sort` to ensure determinism. Checked locally and the
non-determinism went away.
Adds the ability to lookup and display all merged functions for an
address in llvm-gsymutil.
Now, when `--merged-functions` is used in combination with
`--address/--addresses-from-stdin`, lookup results will contain
information about merged functions, if available.
To support printing merged function information when using the
`--verbose` option, the `LookupResult` data structure also had to be
extended with pointers to the raw function data and raw merged function
data. This is because merged functions share the same address range, so
it's not easy to look up the raw merged function data for a particular
`LookupResult` that is based on a merged function.
This patch makes sure YAML files are opened for reading as text file to
trigger auto-conversion from EBCDIC encoding into expected ASCII
encoding on z/OS platform. This is required to fix the following lit
tests:
```
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-exe.yaml
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-obj.test
LLVM :: tools/llvm-gsymutil/ARM_AArch64/macho-gsym-callsite-info-dsym.yaml
LLVM :: Transforms/PGOProfile/memprof_undrift_missing_leaf.ll
```
Previously, the test
`llvm/test/tools/llvm-gsymutil/ARM_AArch64/macho-gsym-merged-callsites-dsym.yaml`
[was disabled](https://github.com/llvm/llvm-project/pull/119957/files)
due to failing on Linux.
The issue is that the merged functions appear in a different order in
the final produced `gSYM`.
To ensure deterministic ordering of merged functions, we change the
sorting of functions to use the `stable_sort` algorithm. Before merged
functions, this was not an issue as the address is used as a comparator
in the sorting algorithm so there was no chance of two functions testing
as identical according to the comparator.
Confirmed that test now passes locally with
`-DLLVM_ENABLE_EXPENSIVE_CHECKS=ON`.
This patch modifies the DWARF verifier to handle a valid case where two
or more functions have identical address ranges due to being merged by
ICF (Identical Code Folding). Previously, the verifier would incorrectly
report these as errors, but functions merged via ICF (such as when using
LLD's --keep-icf-stabs option) can legitimately share the same address
range.
A new test case has been added to verify this behavior using YAML-based
DWARF data that simulates two DW_TAG_subprogram entries with identical
address ranges. The test ensures that the verifier correctly identifies
this as a valid case and doesn't emit any errors, while still
maintaining the existing verification for truly invalid overlapping
ranges in other scenarios. Before this change, the newly added test case
would have failed, with `llvm-dwarfdump` marking the overlapping address
ranges in the DWARF as an error.
We also modify the existing tests `llvm-dwarfutil/ELF/X86/verify.test` and
`llvm/test/tools/llvm-dwarfdump/X86/verify_parent_zero_length.yaml`
which rely on the existence of the error that we're trying to
suppress. We slightly change one offset so that the ranges don't
perfectly overlap and an error is still generated.
This change adds support for loading gSYM callsite information from
DWARF. Previously the only support was for loading callsites info from
YAML.
For testing, we add a pass where `macho-gsym-merged-callsites-dsym`
loads callsite info from DWARF rather than YAML.
[llvm-debuginfo-analyzer] Fix crash due to un-checked error in LVReaderHandler::handleArchive
method.
- Added README describing how to generated the binary files used for the test.
- A follow up patch to add extra ASSERT_NE
Committed on behalf of @aurelien35
Currently, when dumping the contents of a GSYM there are three issues:
- Callsite information is not displayed for merged functions - this is
because of a bug in `CallSiteInfoLoader::buildFunctionMap` where when
enumerating through `Func.MergedFunctions` - we enumerate by value
instead of by reference.
- There is no variable indent for printing callsite info - meaning that
when printing callsites for merged functions, the indent will be
different than the other info of the merged function. To address this we
add configurable indent for printing callsite info
- Callsite info is printed right after merged function info. Meaning
that if the merged function also has call site information, the parent's
callsite info will appear right after the merged function's callsite
info - leading to confusion. To address this we print the callsite info
first, then the merged functions info.
This change addresses all the above 3 issues.
Example of old vs new:
<img width="1074" alt="image"
src="https://github.com/user-attachments/assets/d039ad69-fa79-4abb-9816-eda9cc2eda53"
/>
