This patch creates skeleton implementation for the DWARFLinkerParallel.
It also integrates DWARFLinkerParallel into dsymutil and llvm-dwarfutil,
so that empty DWARFLinker::link() can be called. To do this new command
line option is added "--linker apple/llvm". Additionally it changes
existing DWARFLinker interfaces/implementations to be compatible:
use Error for error reporting for the DWARFStreamer, make DWARFFile to
owner of referenced resources, other small refactorings.
Differential Revision: https://reviews.llvm.org/D147952
This patch creates skeleton implementation for the DWARFLinkerParallel.
It also integrates DWARFLinkerParallel into dsymutil and llvm-dwarfutil,
so that empty DWARFLinker::link() can be called. To do this new command
line option is added "--linker apple/llvm". Additionally it changes
existing DWARFLinker interfaces/implementations to be compatible:
use Error for error reporting for the DWARFStreamer, make DWARFFile to
owner of referenced resources, other small refactorings.
Differential Revision: https://reviews.llvm.org/D147952
The corresponding function definition was removed by:
commit 95a8e8a2557e487be6133475966255661a9f6567
Author: Jonas Devlieghere <jonas@devlieghere.com>
Date: Tue Dec 3 11:10:04 2019 -0800
With dsymutil's -u option, only the accelerator tables should be
updated, but with https://reviews.llvm.org/D150554 the -u option will
still re-generate the line table. If the line table was empty, that is,
it was a dummy line table, with no entries in it, dsymutil will always
generate a line table with a DW_LNE_end_sequence, a funky side effect of
this is that when the line table is re-generated, it will always emit a
DW_LNE_set_address first, which will change the line table total size.
This patch addresses this by making sure that if all the line table has
in it is a DW_LNE_end_sequence, it is the same as a dummy entry.
Differential Revision: https://reviews.llvm.org/D151579
This reverts commit 9bd3ff8bf9e05e7bf95a4a068aa3b50813a93975.
Reverting because of test failures:
TEST 'LLVM :: tools/dsymutil/ARM/fat-dylib-update.test' FAILED
Command Output (stderr):
--
+ : 'RUN: at line 2'
+ /b/ml-opt-rel-x86-64-b1/build/bin/dsymutil -oso-prepend-path /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/tools/dsymutil/ARM/.. /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/tools/dsymutil/ARM/../Inputs/fat-test.arm.dylib -o /b/ml-opt-rel-x86-64-b1/build/test/tools/dsymutil/ARM/Output/fat-dylib-update.test.tmp.dSYM
warning: /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/tools/dsymutil/ARM/../Inputs/fat-test.arm.o: timestamp mismatch between object file (2023-05-24 23:42:39.442778779) and debug map (2015-08-05 21:31:26.000000000)
warning: /b/ml-opt-rel-x86-64-b1/llvm-project/llvm/test/tools/dsymutil/ARM/../Inputs/fat-test.arm.o: timestamp mismatch between object file (2023-05-24 23:42:39.442778779) and debug map (2015-08-05 21:31:26.000000000)
error: lipo: No such file or directory
With dsymutil's -u option, only the accelerator tables should be
updated, but with https://reviews.llvm.org/D150554 the -u option will
still re-generate the line table. If the line table was empty, that is,
it was a dummy line table, with no entries in it, dsymutil will always
generate a line table with a DW_LNE_end_sequence, a funky side effect of
this is that when the line table is re-generated, it will always emit a
DW_LNE_set_address first, which will change the line table total size.
This patch addresses this by making sure that if all the line table has
in it is a DW_LNE_end_sequence, it is the same as a dummy entry.
Differential Revision: https://reviews.llvm.org/D151579
This patch fixes the problem introduced by D147066. As D147066 may change
the contents of location expression, it started to calculate final attribute
size. This patch uses more correct way to calculate size: DIEValue::sizeOf().
Differential Revision: https://reviews.llvm.org/D151348
This patch adds handling of DW_OP_addrx and DW_OP_constx expression operands.
In --update case these operands are preserved as is. Otherwise they are
converted into the DW_OP_addr and DW_OP_const[*]u correspondingly.
Differential Revision: https://reviews.llvm.org/D147066
This patch adds support for DWARFv5 .debug_line_str table.
It replaces code generating line table. Instead of copying original
table and patching certain places this patch implements full line table
generation.
Differential Revision: https://reviews.llvm.org/D150554
This patch makes interface of AddressManager from DWARFLinker
to be compatible with AddressesMap from DWARFLinkerParallel.
This makes both linkers to be interchangeable.
Differential Revision: https://reviews.llvm.org/D147455
This patch a new verification mode called "auto" that runs the DWARF
verifier on the input and if the input is valid, also runs the DWARF
verifier on the output. The goal is to catch cases where dsymutil turns
valid DWARF into invalid DWARF. This patch makes this verification mode
the default when assertions or expensive checks are enabled.
Differential revision: https://reviews.llvm.org/D147203
Fix an inconsistency between input and output verification in dsymutil.
Previously, output verification would be controlled by the verbose flag,
while input verification would unconditionally dump to stdout. Make
input and output verification behave the same by printing verification
error to stderr in verbose mode only.
This patch adds support of DWARFv5 .debug_loclists table.
As DWARFLinker resolves relocations, it is able to always
use DW_FORM_addr instead of DW_FORM_addrx. DW_FORM_addrx
helps to minimize number of relocations, it is also used for
split DWARF. Both of these cases are not relevant for the
DWARFLinker. Thus, this patch converts all DW_FORM_addrx
forms into the DW_FORM_addr. And, as the result, it converts
location lists of DW_FORM_loclistx form into the DW_FORM_sec_offset.
For the --update case all DW_FORM_addrx, DW_FORM_loclistx
are preserved as is.
Depends On D145499
Differential Revision: https://reviews.llvm.org/D145680
This patch add support of DWARFv5 attribute forms: DW_FORM_addrx1,
DW_FORM_addrx2, DW_FORM_addrx3, DW_FORM_addrx4.
Differential Revision: https://reviews.llvm.org/D145805
This patch adds support of DWARFv5 .debug_rnglists table.
As DWARFLinker resolves relocations, it is able to always
use DW_FORM_addr instead of DW_FORM_addrx. DW_FORM_addrx
helps to minimize number of relocations, it is also used for
split DWARF. Both of these cases are not relevant for the
DWARFLinker. Thus, this patch converts all DW_FORM_addrx
forms into the DW_FORM_addr. And, as the result, it converts
range lists of DW_FORM_rnglistx form into the DW_FORM_sec_offset.
For the --update case all DW_FORM_addrx, DW_FORM_rnglistx
are preserved as is.
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D143903
As a preparation for implementing DWARFv5 address ranges generation,
this patch refactors cloneAddressAttribute() method. It has special
handling for addresses which can be relocated in some unrelated value,
for applying relocations twice, for indexed addresses. Instead of
all these special handlings this patch uses general handling:
Read attribute value from InputDIE and apply PCOffset.
Another thing is that current handling of DW_FORM_addrx misses the
fact that relocations might be applied twice in some cases. This
patch fixes this problem also.
Differential Revision: https://reviews.llvm.org/D143269
This relands the commit previously reverted in
`8570bee53a8ce0c5d04bc11f288e19a457474c4c` due to failures on linux.
The problem was that the test executable was built with absolute
OSO prefix paths. This re-commit adds a modified version of the
executable that strips the absolute OSO prefix paths and makes
sure the test appends the OSO prefix appropriately (via the appropriate
dsymutil flags).
Differential Revision: https://reviews.llvm.org/D143458
This reverts commit ccee9b7839a182fc692a65f563abf76f81dd436c.
This started failing on Linux buildbots. The executable wasn't generated
correctly.
Differential Revision: https://reviews.llvm.org/D143458
**Summary**
After this patch, `dsymutil` will preserve `DW_TAG_imported_declarations`
entries in accelerator tables.
This allows consumers to resolve imported declarations even on
executables processsed through dsymutil.
This helps consumers, particularly LLDB's expression evaluator,
to resolve imported declarations (i.e., useful for namespace aliases
in C++) more efficiently.
**Testing**
* Added unit-test
Differential Revision: https://reviews.llvm.org/D143458
This patch fixes#60307 issue. The 8bb4451 introduces the possibility
to unite overlapped or adjacent address ranges to keep address ranges
in an unambiguous state. The AddressRangesMap is used to normalize
address ranges. The AddressRangesMap keeps address ranges and the value
of the relocated address. For intersected range, it creates a united
range that keeps the last inserted mapping value. The same for adjusted ranges.
While it is OK to use the last inserted mapping value for intersected ranges
(as there is no way how to resolve ambiguity) It is not OK to use the
last inserted value for adjacent address ranges. Currently, two following
address ranges are united into a single one:
{0,24,17e685c} {24,d8,55afe20} -> {0,d8,55afe20}
To avoid the problem, the AddressRangesMap should not unite adjacent address ranges
with different relocated addresses. Instead, it should leave adjacent address ranges
as separate ranges. So, the ranges should look like this:
{0,24,17e685c} {24,d8,55afe20}
Differential Revision: https://reviews.llvm.org/D142936
As a preparation for implementing DWARFv5 address ranges generation,
this patch refactors existing address ranges generation code:
Split emitUnitRangesEntries into two functions emitDwarfDebugArangesTable
and emitDwarfDebugRangesTableFragment. Use AddressRanges to prepare linked
address ranges. Refactor Unit.getLowPc(), to use std::nullopt as undefined value.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D141943
generation code from DWARFLinker. It adds command line option:
--build-accelerator [none,DWARF]
Build accelerator tables(default: none)
=none - Do not build accelerators
=DWARF - Build accelerator tables according to the resulting DWARF version
DWARFv4: .debug_pubnames and .debug_pubtypes
DWARFv5: .debug_names
Differential Revision: https://reviews.llvm.org/D139638
The pruning property that's part of the DIE info is fully computing
during the analyzeContextInfo phase, but can be updated during the
lookForDIEsToKeep phase.
// Keep a module forward declaration if there is no definition.
if (!(isODRAttribute(AttrSpec.Attr) && Info.Ctxt &&
Info.Ctxt->hasCanonicalDIE()))
Info.Prune = false;
When the pruning property is updated during the lookForDIEsToKeep phase,
it's not propagated to the parent, unlike during the analyzeContextInfo
phase. This can result in an invalid keep chain with the child DIE being
marked as kept while its parent is still marked as pruned and therefore
never kept, a situation that's now caught by 1b79bed8f532.
This patch fixes this issue by updating the pruning properties of the
parent DIE during the parent walk.
Differential revision: https://reviews.llvm.org/D140930
This patch adds handling of debug_macinfo/debug_macro tables to the DWARFLinker.
It uses already existing code for reading tables from DWARFDebugMacro.h.
It adds new code writing tables into the DwarfStreamer::emitMacroTables.
Differential Revision: https://reviews.llvm.org/D140223
Verify that every DIE that's marked as kept, has a parent that's kept as
well. This invariant should always hold and is easy to verify when
asserts are enabled.
Differential revision: https://reviews.llvm.org/D140227
To improve deduplication across CUs within a single object file,
2b747241a6a0 changed the way we track ODR canonical candidates. The
result is that some assumptions no longer hold. Because of the
aforementioned change, the following condition
assert(Ref > InputDIE.getOffset());
in DWARFLinker::DIECloner::cloneDieReferenceAttribute is no longer an
invariant. An example of a situation where this assertion no longer
holds is when we have decided to replace a backward reference in the
current CU with a forward reference in a subsequent CU. The ODR
canonical DIE hasn't been emitted yet, but the references DIE has an
offset smaller than the current DIE. The assertion is only true if the
referenced DIE was the ODR canonical DIE, which is no longer guaranteed
to be part of the same CU.
Big thanks to Alexey for putting a test together.
Differential revision: https://reviews.llvm.org/D138176
Currently, DWARFLinker receives kind of accel tables as predefined sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Dwarf, ///< DWARF v5 .debug_names.
Default, ///< Dwarf for DWARF5 or later, Apple otherwise.
Pub, ///< .debug_pubnames, .debug_pubtypes
```
This patch removes implicit sets of tables(Default, Dwarf) and allows to ask for several sets:
```
Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
Pub, ///< .debug_pubnames, .debug_pubtypes
DebugNames ///< .debug_names.
```
It allows seamlessness adding more accel tables in the future: .gdb_index, .debug_cu_index...
Doing things that way, DWARFLinker will be independent of consumers' requirements.
f.e. dsymutil and llvm-dwarfutil may have different variants for Default set
(so, instead of implementing these differencies inside DWARFLinker it could be
implemented in the corresponding module).
Differential Revision: https://reviews.llvm.org/D132371
Currently, DWARFLinker determines the target DWARF version internally.
It examines incoming object files, detects maximal
DWARF version and uses that version for the output file.
This patch allows explicitly setting output DWARF version by the consumer
of DWARFLinker. So that DWARFLinker uses a specified version instead
of autodetected one. It allows consumers to use different logic for
setting the target DWARF version. f.e. instead of the maximally used version
someone could set a higher version to convert from DWARFv4 to DWARFv5
(This possibility is not supported yet, but it would be good if
the interface will support it). Or another variant is to set the target
version through the command line. In this patch, the autodetection is moved
into the consumers(DwarfLinkerForBinary.cpp, DebugInfoLinker.cpp).
Differential Revision: https://reviews.llvm.org/D132755
Current implementation of registerModuleReference() function not only
"registers" module reference, but also clones referenced module
(inside loadClangModule()). That may lead to cloning the module with
incorrect options (registerModuleReference() examines module references
and additionally accumulates MaxDwarfVersion and accel tables info).
Since accumulated options may differ from the current values,
it is incorrect to clone modules before options are fully accumulated.
This patch separates "cloning" code from "registering" code. So,
that accumulating option is done in the "registering stage" and
"cloning" is done after all modules are registered and options accumulated.
It also adds a callback for loaded compile units which can be used for
D132755 and D132371(to allow doing options accumulation outside
of DWARFLinker).
Differential Revision: https://reviews.llvm.org/D133047
DWARFLinker::DIECloner::cloneAddressAttribute() contains call to
relocateIndexedAddr(StartOffset, EndOffset). StartOffset is
incorrectly calculated. Val.getRawUValue() is an index into the
.debug_addr table, so it should be multiplied
by Unit.getOrigUnit().getAddressByteSize().
Differential Revision: https://reviews.llvm.org/D132644
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
Current DWARFLinker implementation does not support some debug sections
(mainly DWARF v5 sections). This patch adds diagnostic for such sections.
The warning would be displayed for critical(such that could not be removed)
sections and the source file would be skipped. Other unsupported sections
would be removed and warning message should be displayed. The zero exit
status would be returned for both cases.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D123623
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
DWARF files may contain overlapping address ranges. f.e. it can happen if the two
copies of the function have identical instruction sequences and they end up sharing.
That looks incorrect from the point of view of DWARF spec. Current implementation
of DWARFLinker does not combine overlapped address ranges. It would be good if such
ranges would be handled in some useful way. Thus, this patch allows DWARFLinker
to combine overlapped ranges in a single one.
Depends on D86539
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D123469
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
This patch implements proposal https://lists.llvm.org/pipermail/llvm-dev/2020-August/144579.html
llvm-dwarfutil - is a tool that is used for processing debug info(DWARF) located in built binary files to improve debug info quality, reduce debug info size. The patch currently implements smaller set of command-line options(comparing to the proposal):
```
./llvm-dwarfutil [options] <input file> <output file>
--garbage-collection Do garbage collection for debug info(default)
-j <value> Alias for --num-threads
--no-garbage-collection Don`t do garbage collection for debug info
--no-odr-deduplication Don`t do ODR deduplication for debug types
--no-odr Alias for --no-odr-deduplication
--no-separate-debug-file
Create single output file, containing debug tables(default)
--num-threads <threads> Number of available threads for multi-threaded execution. Defaults to the number of cores on the current machine
--odr-deduplication Do ODR deduplication for debug types(default)
--odr Alias for --odr-deduplication
--separate-debug-file Create two output files: file w/o debug tables and file with debug tables
--tombstone [bfd,maxpc,exec,universal]
Tombstone value used as a marker of invalid address(default: universal)
=bfd - Zero for all addresses and [1,1] for DWARF v4 (or less) address ranges and exec
=maxpc - Minus 1 for all addresses and minus 2 for DWARF v4 (or less) address ranges
=exec - Match with address ranges of executable sections
=universal - Both: bfd and maxpc
```
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D86539
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
Currently, dsymutil treats the DW_OP_convert operand as absolute instead
of CU relative, as described by in the DWARFv5 spec, 2.5.1.6:
"[DW_OP_convert] takes one operand, which is an unsigned LEB128 integer
that represents the offset of a debugging information entry in the current
compilation unit"
This patch makes dsymutil correctly treat the offset as CU relative,
preventing a crash when there are multiple compilation units.
Big thanks to Akira Hatanaka for figuring out this issue and providing
both a reduced test case and a proposed fix.
