Store Swift mangled names in DW_AT_linkage_name. The Swift compiler
emits only the type mangled name in debug information, and LLDB uses
those mangled names as keys to look up size, alignment, fields, etc
from either reflection metadata or Swift modules.
Additionally, emit types linkage names for types into the accelerator
table if they exist and they're different from the display name.
Provide a option (--no-object-timestamp) to ignore object file timestamp
mismatches. We already have a similar option for Swift modules
(--no-swiftmodule-timestamp).
rdar://123975869
Several lit tests look for messages with text generated from strerror()
such as "no such file or directory". The value can change slightly from
system to system. Use the llvm-lit macro `%errc_<ERRCODE>` instead.
This was really noticable on z/OS because the generated text includes an
error code as well as the text.
According to the standard `DW_LNCT_directory_index` can be `data1`,
`data2`, or `udata` (see 6.2.4.1). The code was using `data1`, but this
limits the number of directories to 256, even if the variable holding
the directory index is a `uint64_t`. `dsymutil` was hitting an assertion
when trying to write directory indices higher than 255.
Modify the classic and the parallel DWARF linkers to use `udata` and
encode the directory indices as ULEB128 and provide a test that has more
than 256 directories to check the changes are working as expected.
For people that were using `dsymutil` with CUs that had between 128-256
directories, this will mean that for those indices 2 bytes will be used
now, instead of just one.
This fixes a verifier error uncovered as a result of
https://github.com/llvm/llvm-project/pull/101775.
When compiling with `-gmodules`, Clang will create a skeleton CU that
contains a `DW_AT_dwo_id` and a `DW_AT_dwo_name` corresponding to the
path of the `.pcm` that carries the type definitions referenced in the
non-skeleton CU (see the [gmodules LLDB
docs](https://lldb.llvm.org/resources/extensions.html) for more
details). The non-skeleton CU will also contain a `DW_AT_dwo_id` that
matches that of the skeleton.
`dsymutil` effectively undoes the `-gmodules` work, replacing all the
module type references with definitions. I.e., we no longer create a
skeleton `.dwo` CU.
Prior to this patch `dsymutil` did not strip out the `DW_AT_dwo_id` on
the non-skeleton CU. This now (since
https://github.com/llvm/llvm-project/pull/101775) causes verification
errors such as:
```
Verifying .debug_names...
error: Name Index @ 0x0: Entry @ 0x9a unable to load .dwo file "None"
for DWARF unit @ 0x0.
error: output verification failed for x86_64
make: *** [a.out.dSYM] Error 1
```
...because the verifier sees the DWO ID but can't find a matching `.dwo`
unit.
This patch simply strips the `DW_AT_dwo_id` from the main CU.
DWARF-4 introduced DWARF discriminators but both, the classic and the
parallel DWARF linkers discarded them so far.
After applying this patch dsymutil, which uses the DWARF linkers,
correctly preserves discriminator information.
Add a -q/--quiet flag to suppress dsymutil output. For now the flag is
limited to dsymutil, though there might be other places in the DWARF
linker that could be conditionalized by this flag.
The motivation is having a way to silence the "no debug symbols in
executable" warning. This is useful when we want to generate a dSYM for
a binary not containing debug symbols, but still want a dSYM that can be
indexed by spotlight.
rdar://127843467
Support generating dSYM companion files for (non-Darwin) firmware
environments by considering the binary component of the triple in
addition to the OS component.
rdar://125629792
Remove support for obfuscated bitcode in dsymutil and the DWARF linker.
We no longer support bitcode submissions and the obfuscation support has
been removed from the rest of the compiler.
rdar://123863918
Remove support for obfuscated bitcode in dsymutil and the DWARF linker.
We no longer support bitcode submissions and the obfuscation support has
been removed from the rest of the compiler.
rdar://123863918
Based on the discussion in
https://github.com/llvm/llvm-project/pull/80229
changed implementation to align with how .debug_abbrev is handled. So
that
.debug_names abbrev tag is a monotonically increasing index. This allows
for
tools like LLDB to access it in constant time using array like data
structure.
clang-19 debug build
before change
[41] .debug_names PROGBITS 0000000000000000 8f9e0350 137fdbe0 00 0 0 4
after change
[41] .debug_names PROGBITS 0000000000000000 8f9e0350 125bfdec 00 0 0 4
Reduction ~19.1MB
This commit introduces a helper function to DWARFAcceleratorTable::Entry
which follows DW_IDX_Parent attributes to returns the corresponding
parent Entry in the table.
It is tested by enhancing dwarfdump so that it now prints:
1. When data is corrupt.
2. When parent information is present, but the parent is not indexed.
3. The parent entry offset, when the parent is present and indexed. This
is printed in terms a real entry offset (the same that gets printed at
the start of each entry: "Entry @ 0x..."), instead of the encoded number
in the table (which is an offset from the start off the Entry list).
This makes it easy to visually inspect the dwarfdump and check what the
parent is.
When the MachODebugMapParser encounters an object file that cannot be
found on disk, it currently leaves the parser in an incoherent state,
resulting in spurious warnings that can in turn slow down dsymutil.
This fixes#78411.
rdar://117515153
This implements the ideas discussed in [1].
To summarize, this commit changes AsmPrinter so that it outputs
DW_IDX_parent information for debug_name entries. It will enable
debuggers to speed up queries for fully qualified types (based on a
DWARFDeclContext) significantly, as debuggers will no longer need to
parse the entire CU in order to inspect the parent chain of a DIE.
Instead, a debugger can simply take the parent DIE offset from the
accelerator table and peek at its name in the debug_info/debug_str
sections.
The implementation uses two types of DW_FORM for the DW_IDX_parent
attribute:
1. DW_FORM_ref4, which points to the accelerator table entry for the
parent.
2. DW_FORM_flag_present, when the entry has a parent that is not in the
table (that is, the parent doesn't have a name, or isn't allowed to be
in the table as per the DWARF spec). This is space-efficient, since it
takes 0 bytes.
The implementation works by:
1. Changing how abbreviations are encoded (so that they encode which
form, if
any, was used to encode IDX_Parent)
2. Creating an MCLabel per accelerator table entry, so that they may be
referred by IDX_parent references.
When all patches related to this are merged, we are able to show that
evaluating an expression such as:
```
lldb --batch -o 'b CodeGenFunction::GenerateCode' -o run -o 'expr Fn' -- \
clang++ -c -g test.cpp -o /dev/null
```
is far faster: from ~5000 ms to ~1500ms.
Building llvm-project + clang with and without this patch, and looking
at its impact on object file size:
```
ls -la $(find build_stage2_Debug_idx_parent_assert_dwarf5 -name \*.cpp.o) | awk '{s+=$5} END {printf "%\047d\n", s}'
11,507,327,592
-la $(find build_stage2_Debug_no_idx_parent_assert_dwarf5 -name \*.cpp.o) | awk '{s+=$5} END {printf "%\047d\n", s}'
11,436,446,616
```
That is, an increase of 0.62% in total object file size.
Looking only at debug_names:
```
$stage1_build/bin/llvm-objdump --section-headers $(find build_stage2_Debug_idx_parent_assert_dwarf5 -name \*.cpp.o) | grep __debug_names | awk '{s+="0x"$3} END {printf "%\047d\n", s}'
440,772,348
$stage1_build/bin/llvm-objdump --section-headers $(find build_stage2_Debug_no_idx_parent_assert_dwarf5 -name \*.cpp.o) | grep __debug_names | awk '{s+="0x"$3} END {printf "%\047d\n", s}'
369,867,920
```
That is an increase of 19%.
DWARF Linkers need to be changed in order to support this. This commit
already brings support to "base" linker, but it does not attempt to
modify the parallel linker. Accelerator entries refer to the
corresponding DIE offset, and this patch also requires the parent DIE
offset -- it's not clear how the parallel linker can access this. It may
be obvious to someone familiar with it, but it would be nice to get help
from its authors.
[1]:
https://discourse.llvm.org/t/rfc-improve-dwarf-5-debug-names-type-lookup-parsing-speed/74151/
This patch renames values of dsymutil/llvm-dwarfutil options:
--linker apple -> --linker classic
--linker llvm -> --linker parallel
The purpose to rename options is to avoid using vendor names and to
match with library names. It should be safe to rename options at current
stage as they are not seemed widely used(we may not preserve backward
compatibility).
There was a strange and seemingly unncessary empty string optimization
in NonRelocatableStringPool that I had to remove in order to support
empty strings in the line_str string table, without unconditionally
forcing an empty string to be added to every debug_line_str table.
The current warning emitted by dsymutil when it can't find a symbol in
an object file is worded rather poorly:
```
could not find object file symbol for symbol _foo
```
It's also lacking information that makes the warning actionable, such as
the object file it's looking at. This patch rewords the warning and adds
the object file path to the warning:
```
could not find symbol '_foo' in object file 'test.o'
```
rdar://119621065
This patch is extracted from D96035, it adds support for the type
deduplication mode. With this patch DWARFLinkerParallel handles --odr
option. It also processes clang modules.
Enable Type Units with DWARF5 accelerator tables for monolithic DWARF.
Implementation relies on linker to tombstone offset in LocalTU list to
-1 when
it deduplciates type units using COMDAT.
Dsymutil already avoids copying textual Swift interface files from the
SDK, since any consumer would have to have a matching SDK installed
anyway. It should also do the same thing with interfaces found in the
toolchain itself, which includes the compiler built-in libraries such as
Swift (=the standard library), and _Concurrency.
rdar://117881604
This reverts commit 122c89b271af30b86536cad7bac64ea9c56615ed. Change does not build, with errors such as:
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:60:17: error: declaration of ‘llvm::dsymutil::SymbolMapping llvm::dsymutil::ValidReloc::SymbolMapping’ changes meaning of ‘SymbolMapping’ [-fpermissive]
60 | SymbolMapping SymbolMapping;
| ^~~~~~~~~~~~~
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:36:8: note: ‘SymbolMapping’ declared here as ‘struct llvm::dsymutil::SymbolMapping’
36 | struct SymbolMapping {
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/DebugMap.h:198:32: error: declaration of ‘std::optional<llvm::dsymutil::RelocationMap> llvm::dsymutil::DebugMapObject::RelocationMap’ changes meaning of ‘RelocationMap’ [-fpermissive]
198 | std::optional<RelocationMap> RelocationMap;
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:76:7: note: ‘RelocationMap’ declared here as ‘class llvm::dsymutil::RelocationMap’
76 | class RelocationMap {
| ^~~~~~~~~~~~~
This adds support in dsymutil for mergeable libraries [1].
dsymutil reads a new stab emitted by ld, allowing it to operate on
dynamic libraries instead of object files. It also now loads the DWARF
files associated to the libraries, and build the debug map for each
binary from the list of symbols exported by the library. For each Debug
Map Object, there is a new associated Relocation Map which is serialized
from the information retrieved in the original debug_info (or
debug_addr) section of the .o file.
The final DWARF file has multiple compile units, so the offsets
information of the relocations are adjusted relatively to the compile
unit they will end up belonging to, inside the final linked DWARF file.
[1] https://developer.apple.com/documentation/xcode/configuring-your-project-to-use-mergeable-libraries
Differential revision: https://reviews.llvm.org/D158124
For LLVM_DIAGNOSTIC_DIR we want to create a subdirectory while for
DSYMUTIL_REPRODUCER_PATH we want to use the directory specified. The
latter makes writing a test a whole lot easier. Keep both to support
both scenarios.
This patch is extracted from D96035, it adds support for the accelerator
tables to the DWARFLinkerParallel functionality.
Differential Revision: https://reviews.llvm.org/D154793
Remove the paper trail warning from dsymutil and the DWARF linker. The
original purpose of this functionality was to leave a paper trail in the
output artifact about missing object files. The current implementation
however has diverged and is the source of a pretty serious bug:
- In the debug map parser, missing object files are not the only
warnings we emit. When paper trail warnings are enabled, all of them end
up in the dSYM which wasn't the goal.
- When warnings are associated with a object file in the debug map, it
is skipped by the DWARF linker. This only makes sense if the object file
is missing and is obviously incorrect for any other type of warning
(such as a missing symbol).
The combination of the two means that we can generate broken DWARF when
the feature is enabled. AFAIK it was only used by Apple and nobody I
spoke to has relied on it, so rather than fixing the broken behavior I
propose we remove it.
ObjectiveC has its own extra accelerator table entries that are helpful for the
debugger. This patch relaxes the DWARFVerifier so that it accepts those in DWARF
5's debug_names.
Differential Revision: https://reviews.llvm.org/D159471
The DWARFLinker library has code to identify ObjC selector names, which is used
by the debug linker to generate accelerator table entries. In the future, we
would like the DWARF verifier to also have access to such code, so that it can
identify these names when verifying accelerator tables (e.g. debug_names).
This patch follows the same intent of D155723, where we also moved code
generating simplified template names.
Since this is moving code around and changing the log, we also replace raw
pointer manipulation with the more expressive
StringRef::{drop_front,take_front,...} methods.
We also change a test so that it verifies its output, and that requires having
dsymutil not write to stdout.
Differential Revision: https://reviews.llvm.org/D158980
location-expression.test and tls-variable.test failed because wrong
endianness was used on powerpc machine. To fix the issue this patch
uses endianness from debug map as endianness for the whole target file.
Previosly, architecture endianness won over what is specified
in debug map.
Differential Revision: https://reviews.llvm.org/D159349
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Reland2: temporarily disabled call to "--linker llvm" for tls-variable.test
and location-expression.test as it does not work properly on bigendian
architecture.
Differential Revision: https://reviews.llvm.org/D153268
This reverts commit 0229dd0626b2538c78ebbd2b1bd44c31fbf1cdec.
This introduces two test failures on s390x.
tools/dsymutil/X86/location-expression.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/location-expression.test:20:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:34:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit [1] *
^
<stdin>:37:2: note: possible intended match here
DW_AT_name [DW_FORM_strp] ( .debug_str[0x09000000] = )
^
tools/dsymutil/X86/tls-variable.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/tls-variable.test:19:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:26:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit
^
<stdin>:29:2: note: possible intended match here
DW_AT_name ()
^
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
LLDB can benefit from having the base name of functions (i.e. without any
template parameters) as an entry into accelerator tables pointing back in the
DIE for the corresponding function specialization. In fact, some LLDB
functionality is only possible when those entries are present.
The DWARFLinker has been adding such entries for a while now, both with
apple_names and with debug_names. However, this has two side effects:
1. Some LLDB functionality is only possible when dsym bundles are present (i.e.
the linker touched the debug info).
2. The DWARFVerifier doesn't accept debug_name sections created by the linker,
as such names are (usually) neither the AT_name nor the AT_linkage_name of the
DIE.
Based on recent discussion [1], and because the DWARF 5 spec says that:
> A producer may choose to implement additional rules for what names are placed
> in the index
This patch relaxes the checks on the verifier to allow for simplified template
names in the accelerator table. To do so, we move some helper functions from
DWARFLinker into the core lib debug info. This addresses the point 2) above.
This patch also enables addressing point 1) in the future, since the helper
function is now visible to other parts of LLVM.
[1]: https://github.com/llvm/llvm-project/issues/58362
Differential Revision: https://reviews.llvm.org/D155723
instead of DW_ELE_start_length in debug_rnglists section
This patch tries to reduce the size of the debug_rnglist section by
replacing the DW_RLE_start_length opcodes currently emitted by dsymutil
in favor of using DW_RLE_base_addressx + DW_RLE_offset_pair instead.
The DW_RLE_start_length is one AddressSize followed by a ULEB per entry,
whereas, the DW_RLE_base_addressx + DW_RLE_offset_pair will use one ULEB
for the base address, and then the DW_RLE_offset_pair is a pair of
ULEBs. This will be more efficient.
Differential Revision: https://reviews.llvm.org/D156166
