This patch a new verification mode called "auto" that runs the DWARF
verifier on the input and if the input is valid, also runs the DWARF
verifier on the output. The goal is to catch cases where dsymutil turns
valid DWARF into invalid DWARF. This patch makes this verification mode
the default when assertions or expensive checks are enabled.
Differential revision: https://reviews.llvm.org/D147203
Fix an inconsistency between input and output verification in dsymutil.
Previously, output verification would be controlled by the verbose flag,
while input verification would unconditionally dump to stdout. Make
input and output verification behave the same by printing verification
error to stderr in verbose mode only.
This patch adds support of DWARFv5 .debug_loclists table.
As DWARFLinker resolves relocations, it is able to always
use DW_FORM_addr instead of DW_FORM_addrx. DW_FORM_addrx
helps to minimize number of relocations, it is also used for
split DWARF. Both of these cases are not relevant for the
DWARFLinker. Thus, this patch converts all DW_FORM_addrx
forms into the DW_FORM_addr. And, as the result, it converts
location lists of DW_FORM_loclistx form into the DW_FORM_sec_offset.
For the --update case all DW_FORM_addrx, DW_FORM_loclistx
are preserved as is.
Depends On D145499
Differential Revision: https://reviews.llvm.org/D145680
This patch add support of DWARFv5 attribute forms: DW_FORM_addrx1,
DW_FORM_addrx2, DW_FORM_addrx3, DW_FORM_addrx4.
Differential Revision: https://reviews.llvm.org/D145805
This patch adds support of DWARFv5 .debug_rnglists table.
As DWARFLinker resolves relocations, it is able to always
use DW_FORM_addr instead of DW_FORM_addrx. DW_FORM_addrx
helps to minimize number of relocations, it is also used for
split DWARF. Both of these cases are not relevant for the
DWARFLinker. Thus, this patch converts all DW_FORM_addrx
forms into the DW_FORM_addr. And, as the result, it converts
range lists of DW_FORM_rnglistx form into the DW_FORM_sec_offset.
For the --update case all DW_FORM_addrx, DW_FORM_rnglistx
are preserved as is.
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D143903
As a preparation for implementing DWARFv5 address ranges generation,
this patch refactors cloneAddressAttribute() method. It has special
handling for addresses which can be relocated in some unrelated value,
for applying relocations twice, for indexed addresses. Instead of
all these special handlings this patch uses general handling:
Read attribute value from InputDIE and apply PCOffset.
Another thing is that current handling of DW_FORM_addrx misses the
fact that relocations might be applied twice in some cases. This
patch fixes this problem also.
Differential Revision: https://reviews.llvm.org/D143269
This relands the commit previously reverted in
`8570bee53a8ce0c5d04bc11f288e19a457474c4c` due to failures on linux.
The problem was that the test executable was built with absolute
OSO prefix paths. This re-commit adds a modified version of the
executable that strips the absolute OSO prefix paths and makes
sure the test appends the OSO prefix appropriately (via the appropriate
dsymutil flags).
Differential Revision: https://reviews.llvm.org/D143458
This reverts commit ccee9b7839a182fc692a65f563abf76f81dd436c.
This started failing on Linux buildbots. The executable wasn't generated
correctly.
Differential Revision: https://reviews.llvm.org/D143458
**Summary**
After this patch, `dsymutil` will preserve `DW_TAG_imported_declarations`
entries in accelerator tables.
This allows consumers to resolve imported declarations even on
executables processsed through dsymutil.
This helps consumers, particularly LLDB's expression evaluator,
to resolve imported declarations (i.e., useful for namespace aliases
in C++) more efficiently.
**Testing**
* Added unit-test
Differential Revision: https://reviews.llvm.org/D143458
This patch fixes#60307 issue. The 8bb4451 introduces the possibility
to unite overlapped or adjacent address ranges to keep address ranges
in an unambiguous state. The AddressRangesMap is used to normalize
address ranges. The AddressRangesMap keeps address ranges and the value
of the relocated address. For intersected range, it creates a united
range that keeps the last inserted mapping value. The same for adjusted ranges.
While it is OK to use the last inserted mapping value for intersected ranges
(as there is no way how to resolve ambiguity) It is not OK to use the
last inserted value for adjacent address ranges. Currently, two following
address ranges are united into a single one:
{0,24,17e685c} {24,d8,55afe20} -> {0,d8,55afe20}
To avoid the problem, the AddressRangesMap should not unite adjacent address ranges
with different relocated addresses. Instead, it should leave adjacent address ranges
as separate ranges. So, the ranges should look like this:
{0,24,17e685c} {24,d8,55afe20}
Differential Revision: https://reviews.llvm.org/D142936
As a preparation for implementing DWARFv5 address ranges generation,
this patch refactors existing address ranges generation code:
Split emitUnitRangesEntries into two functions emitDwarfDebugArangesTable
and emitDwarfDebugRangesTableFragment. Use AddressRanges to prepare linked
address ranges. Refactor Unit.getLowPc(), to use std::nullopt as undefined value.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D141943
generation code from DWARFLinker. It adds command line option:
--build-accelerator [none,DWARF]
Build accelerator tables(default: none)
=none - Do not build accelerators
=DWARF - Build accelerator tables according to the resulting DWARF version
DWARFv4: .debug_pubnames and .debug_pubtypes
DWARFv5: .debug_names
Differential Revision: https://reviews.llvm.org/D139638
The pruning property that's part of the DIE info is fully computing
during the analyzeContextInfo phase, but can be updated during the
lookForDIEsToKeep phase.
// Keep a module forward declaration if there is no definition.
if (!(isODRAttribute(AttrSpec.Attr) && Info.Ctxt &&
Info.Ctxt->hasCanonicalDIE()))
Info.Prune = false;
When the pruning property is updated during the lookForDIEsToKeep phase,
it's not propagated to the parent, unlike during the analyzeContextInfo
phase. This can result in an invalid keep chain with the child DIE being
marked as kept while its parent is still marked as pruned and therefore
never kept, a situation that's now caught by 1b79bed8f532.
This patch fixes this issue by updating the pruning properties of the
parent DIE during the parent walk.
Differential revision: https://reviews.llvm.org/D140930
This patch adds handling of debug_macinfo/debug_macro tables to the DWARFLinker.
It uses already existing code for reading tables from DWARFDebugMacro.h.
It adds new code writing tables into the DwarfStreamer::emitMacroTables.
Differential Revision: https://reviews.llvm.org/D140223
Verify that every DIE that's marked as kept, has a parent that's kept as
well. This invariant should always hold and is easy to verify when
asserts are enabled.
Differential revision: https://reviews.llvm.org/D140227
To improve deduplication across CUs within a single object file,
2b747241a6a0 changed the way we track ODR canonical candidates. The
result is that some assumptions no longer hold. Because of the
aforementioned change, the following condition
assert(Ref > InputDIE.getOffset());
in DWARFLinker::DIECloner::cloneDieReferenceAttribute is no longer an
invariant. An example of a situation where this assertion no longer
holds is when we have decided to replace a backward reference in the
current CU with a forward reference in a subsequent CU. The ODR
canonical DIE hasn't been emitted yet, but the references DIE has an
offset smaller than the current DIE. The assertion is only true if the
referenced DIE was the ODR canonical DIE, which is no longer guaranteed
to be part of the same CU.
Big thanks to Alexey for putting a test together.
Differential revision: https://reviews.llvm.org/D138176
Currently, DWARFLinker receives kind of accel tables as predefined sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Dwarf, ///< DWARF v5 .debug_names.
Default, ///< Dwarf for DWARF5 or later, Apple otherwise.
Pub, ///< .debug_pubnames, .debug_pubtypes
```
This patch removes implicit sets of tables(Default, Dwarf) and allows to ask for several sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Pub, ///< .debug_pubnames, .debug_pubtypes
DebugNames ///< .debug_names.
```
It allows seamlessness adding more accel tables in the future: .gdb_index, .debug_cu_index...
Doing things that way, DWARFLinker will be independent of consumers' requirements.
f.e. dsymutil and llvm-dwarfutil may have different variants for Default set
(so, instead of implementing these differencies inside DWARFLinker it could be
implemented in the corresponding module).
Differential Revision: https://reviews.llvm.org/D132371
Currently, DWARFLinker determines the target DWARF version internally.
It examines incoming object files, detects maximal
DWARF version and uses that version for the output file.
This patch allows explicitly setting output DWARF version by the consumer
of DWARFLinker. So that DWARFLinker uses a specified version instead
of autodetected one. It allows consumers to use different logic for
setting the target DWARF version. f.e. instead of the maximally used version
someone could set a higher version to convert from DWARFv4 to DWARFv5
(This possibility is not supported yet, but it would be good if
the interface will support it). Or another variant is to set the target
version through the command line. In this patch, the autodetection is moved
into the consumers(DwarfLinkerForBinary.cpp, DebugInfoLinker.cpp).
Differential Revision: https://reviews.llvm.org/D132755
Current implementation of registerModuleReference() function not only
"registers" module reference, but also clones referenced module
(inside loadClangModule()). That may lead to cloning the module with
incorrect options (registerModuleReference() examines module references
and additionally accumulates MaxDwarfVersion and accel tables info).
Since accumulated options may differ from the current values,
it is incorrect to clone modules before options are fully accumulated.
This patch separates "cloning" code from "registering" code. So,
that accumulating option is done in the "registering stage" and
"cloning" is done after all modules are registered and options accumulated.
It also adds a callback for loaded compile units which can be used for
D132755 and D132371(to allow doing options accumulation outside
of DWARFLinker).
Differential Revision: https://reviews.llvm.org/D133047
DWARFLinker::DIECloner::cloneAddressAttribute() contains call to
relocateIndexedAddr(StartOffset, EndOffset). StartOffset is
incorrectly calculated. Val.getRawUValue() is an index into the
.debug_addr table, so it should be multiplied
by Unit.getOrigUnit().getAddressByteSize().
Differential Revision: https://reviews.llvm.org/D132644
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
This patch is extracted from D86539.
Current implementation of lookForDIEsToKeep() function skips types
duplications basing on the getCanonicalDIEOffset() data:
```
if (AttrSpec.Form != dwarf::DW_FORM_ref_addr && (UseOdr || IsModuleRef) &&
Info.Ctxt &&
Info.Ctxt != ReferencedCU->getInfo(Info.ParentIdx).Ctxt &&
Info.Ctxt->getCanonicalDIEOffset() && isODRAttribute(AttrSpec.Attr)) <<<<<
continue;
```
But that field is set after all compile units inside object file are processed:
```
for (auto &CurrentUnit : OptContext.CompileUnits)
lookForDIEsToKeep(.., &CurrentUnit, ..); // check CanonicalDIEOffset
DIECloner.cloneAllCompileUnits(); // set CanonicalDIEOffset
```
Thus, if the object file contains several compilation units - types would
not be deduplicated. The above solution works well for the case when the object file
contains only one compilation unit. But if the object file contains several compilation
units then types would not be deduplicated between these compilation units.
This patch changes the algorithm so that types were deduplicated between
compilation units from the same object file.
It produces binary incompatible output for the cases when several compilation units
are located inside the same object file.
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D125469
this review is extracted from D86539.
1. Rename AccelTableKind to DwarfLinkerAccelTableKind
(to differentiate from AccelTableKind from CodeGen/AsmPrinter/DwarfDebug.h)
2. Add None value to the DwarfLinkerAccelTableKind.
3. added 'None' value for 'accelerator' option of dsymutil.
Differential Revision: https://reviews.llvm.org/D125474
Currently you can run the DWARF verifier on the linked dsymutil output.
This patch extends this functionality and makes it possible to
run the DWARF verifier on the input as well.
A new option --verify-dwarf allows you to specify input, output, all and
none. The existing --verify flag remains unchanged and acts and alias
for --verify-dwarf=output.
Input verification issues do not result in a non-zero exit code because
dsymutil is capable of taking invalid DWARF as input and producing valid
DWARF as output.
Differential revision: https://reviews.llvm.org/D89216
This makes a bunch of these call sites independent of a follow-up change
I'm making to have getAsCString return Expected<const char*> for more
descriptive error messages so that the failures there can be
communicated up to DWARFVerifier (or other callers who want to provide
more verbose diagnostics) so DWARFVerifier doesn't have to re-implement
the string lookup logic and error checking.
In debug info, we expect variables to have location info if they are used and we don't want location info for functions that are not used. However, if an unused function is inlined, we could have the scenario where a function is not in the final binary but its static variable is in the final binary. Ensure that variables in the final binary have location debug info even if their scope was inlined.
Also add `--implicit-check-not` to a test for clarity.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D115565
