When the MachODebugMapParser encounters an object file that cannot be
found on disk, it currently leaves the parser in an incoherent state,
resulting in spurious warnings that can in turn slow down dsymutil.
This fixes#78411.
rdar://117515153
The current warning emitted by dsymutil when it can't find a symbol in
an object file is worded rather poorly:
```
could not find object file symbol for symbol _foo
```
It's also lacking information that makes the warning actionable, such as
the object file it's looking at. This patch rewords the warning and adds
the object file path to the warning:
```
could not find symbol '_foo' in object file 'test.o'
```
rdar://119621065
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
This reverts commit 122c89b271af30b86536cad7bac64ea9c56615ed. Change does not build, with errors such as:
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:60:17: error: declaration of ‘llvm::dsymutil::SymbolMapping llvm::dsymutil::ValidReloc::SymbolMapping’ changes meaning of ‘SymbolMapping’ [-fpermissive]
60 | SymbolMapping SymbolMapping;
| ^~~~~~~~~~~~~
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:36:8: note: ‘SymbolMapping’ declared here as ‘struct llvm::dsymutil::SymbolMapping’
36 | struct SymbolMapping {
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/DebugMap.h:198:32: error: declaration of ‘std::optional<llvm::dsymutil::RelocationMap> llvm::dsymutil::DebugMapObject::RelocationMap’ changes meaning of ‘RelocationMap’ [-fpermissive]
198 | std::optional<RelocationMap> RelocationMap;
| ^~~~~~~~~~~~~
In file included from ../llvm-project/llvm/tools/dsymutil/DebugMap.h:24,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.h:13,
from ../llvm-project/llvm/tools/dsymutil/DwarfLinkerForBinary.cpp:9:
../llvm-project/llvm/tools/dsymutil/RelocationMap.h:76:7: note: ‘RelocationMap’ declared here as ‘class llvm::dsymutil::RelocationMap’
76 | class RelocationMap {
| ^~~~~~~~~~~~~
This adds support in dsymutil for mergeable libraries [1].
dsymutil reads a new stab emitted by ld, allowing it to operate on
dynamic libraries instead of object files. It also now loads the DWARF
files associated to the libraries, and build the debug map for each
binary from the list of symbols exported by the library. For each Debug
Map Object, there is a new associated Relocation Map which is serialized
from the information retrieved in the original debug_info (or
debug_addr) section of the .o file.
The final DWARF file has multiple compile units, so the offsets
information of the relocations are adjusted relatively to the compile
unit they will end up belonging to, inside the final linked DWARF file.
[1] https://developer.apple.com/documentation/xcode/configuring-your-project-to-use-mergeable-libraries
Differential revision: https://reviews.llvm.org/D158124
Remove the paper trail warning from dsymutil and the DWARF linker. The
original purpose of this functionality was to leave a paper trail in the
output artifact about missing object files. The current implementation
however has diverged and is the source of a pretty serious bug:
- In the debug map parser, missing object files are not the only
warnings we emit. When paper trail warnings are enabled, all of them end
up in the dSYM which wasn't the goal.
- When warnings are associated with a object file in the debug map, it
is skipped by the DWARF linker. This only makes sense if the object file
is missing and is obviously incorrect for any other type of warning
(such as a missing symbol).
The combination of the two means that we can generate broken DWARF when
the feature is enabled. AFAIK it was only used by Apple and nobody I
spoke to has relied on it, so rather than fixing the broken behavior I
propose we remove it.
Static archives can contain multiple files with the same file name, in
which case the timestamp is used to disambiguate. Because timestamps are
expressed in seconds since epoch timestamp collisions are far from
impossible. Furthermore, to facilitate reproducible builds, the static
linker can be told to emit no timestamps at all.
dsymutil already detects timestamp mismatches between the debug map and
the object files. However, it does not handle timestamp collisions
within the debug maps (STABS). Currently, we arbitrarily pick the first
debug map entry and ignore the rest. This is incorrect: if a symbol
exists in multiple object files, the linker might not have picked the
one from the first object file. This also results in missing symbol
warnings for all the symbols not defined in the first object file.
Given that in this scenario, dsymutil does not have enough information
to disambiguate, it should print a single informative warning and skip
the ambiguous debug map objects.
rdar://110374836
Differential revision: https://reviews.llvm.org/D152585
Fix a crash when a warning is emitted while loading the symbols from the
main binary. The warning helper assumes that the resulting debug map is
initialized, but this happened after loading the main binary. Since
there's no dependency between the two the initialization can be moved
up.
rdar://107298776
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
I saw the TODOs while reading this file and figured I'd do them.
I haven't seen these happen in practice.
No expected behavior change.
Differential Revision: https://reviews.llvm.org/D123215
STABS information consists of a list of records in the linked binary
that look like this:
OSO: path/to/some.o
SO: path/to/some.c
FUN: sym1
FUN: sym2
...
The linked binary has one such set of records for every .o file linked
into it.
When dsymutil processes the binary's STABS information, it:
1. Reads the .o file mentioned in the OSO line
2. For each FUN entry after it in the main executable's STABS info:
a) it looks up that symbol in the symbol of that .o file
b) if it doesn't find it there, it goes through all symbols in the
main binary at the same address and sees if any of those match
With ICF, ld64.lld's STABS output claims that all identical functions
that were folded are in the .o file of the one that's deemed the
canonical one. Many small functions might be folded into a single
function, so there are .o OSO entries that end up with many FUN lines,
but almost none of them exist in the .o file's symbol table.
Previously, dsymutil would do a full scan of all symbols in the main
executable _for every of these entries_.
This patch instead scans all aliases once and remembers them per name.
This reduces the alias resolution complexity from
O(number_of_aliases_in_o_file * number_of_symbols_in_main_executable) to
O(number_of_aliases_in_o_file * log(number_of_aliases_in_o_file)).
In practice, it reduces the time spent to run dsymutil on
Chromium Framework from 26 min (after https://reviews.llvm.org/D89444)
or 12 min (before https://reviews.llvm.org/D89444) to ~8m30s.
We probably want to change how ld64.lld writes STABS entries when ICF
is enabled, but making dsymutil not have pathological performance for
this input seems like a good change as well.
No expected behavior change (other than it's faster). I verified that
for Chromium Framework, the generated .dSYM is identical with and
without this patch.
Differential Revision: https://reviews.llvm.org/D123218
Fix spurious warnings for missing symbols with thinLTO. The latter
appends a unique suffix to avoid collisions for exported private
symbols, resulting in dsymutil complaining it couldn't find the symbol
in the object file.
rdar://75434058
Differential revision: https://reviews.llvm.org/D99125
In f9fb9da36c34d2745b74dc30e6c26f7d3f48466c we fixed spurious warnings
caused by aliases to private extern symbols. This patch generalizes that
to regular external symbols as well.
dsymutil was incorrectly ignoring aliases to private extern symbols in
the MachODebugMapParser. This resulted in spurious warnings about not
being able to find symbols.
rdar://49652389
Differential revision: https://reviews.llvm.org/D89444
This patch threads the virtual file system through dsymutil.
Currently there is no good way to find out exactly what files are
necessary in order to reproduce a dsymutil link, at least not without
knowledge of how dsymutil's internals. My motivation for this change is
to add lightweight "reproducers" that automatically gather the input
object files through the FileCollectorFileSystem. The files together
with the YAML mapping will allow us to transparently reproduce a
dsymutil link, even without having to mess with the OSO path prefix.
Differential revision: https://reviews.llvm.org/D79376
Summary:
In D77860, we have changed `getSymbolFlags()` return type to `Expected<uint32_t>`.
This change helps bubble the error further up the stack.
Reviewers: jhenderson, grimar, JDevlieghere, MaskRay
Reviewed By: jhenderson
Subscribers: hiraditya, MaskRay, rupprecht, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D79075
Summary: The wrong variable was being checked for an error, which mean a llvm::Error went unchecked and crashes dsymutil. Discovered this when trying to feed an ELF file to "dsymutil --update" and running into the crash.
Reviewers: aprantl, JDevlieghere
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D75777
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
The original patch got reverted because it hit a long-standing legacy
issue on Windows that prevents files from being named `com`. Thanks
Kristina & Jeremy for pointing this out.
llvm-svn: 374178
The added test files ("com", "com1.o", "com2.o") are reserved names on
Windows, and makes 'git checkout' fail with a filesystem error.
llvm-svn: 374144
For common symbols the linker emits only a single symbol entry in the
debug map. This caused dsymutil to not relocate common symbols when
linking DWARF coming form object files that did not have this entry.
This patch fixes that by keeping track of common symbols in the object
files and synthesizing a debug map entry for them using the address from
the main binary.
Differential revision: https://reviews.llvm.org/D68680
llvm-svn: 374139
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
llvm-svn: 369013
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
The unobufscation support for BCSymbolMaps was the last piece of code
that hasn't been upstreamed yet. This patch contains a reworked version
of the existing code and relevant tests.
Differential revision: https://reviews.llvm.org/D56346
llvm-svn: 350580
Usually local symbols will have their address described in the debug
map. Global symbols have to have their address looked up in the symbol
table of the main executable. By playing with 'ld -r' and export lists,
you can get a symbol described as global by the debug map while actually
being a local symbol as far as the link in concerned. By gathering the
address of local symbols, we fix this issue.
Also, we prefer a global symbol in case of a name collision to preserve
the previous behavior.
Note that using the 'ld -r' tricks, people can actually cause symbol
names collisions that dsymutil has no way to figure out. This fixes the
simple case where there is only one symbol of a given name.
rdar://problem/32826621
Differential revision: https://reviews.llvm.org/D54922
llvm-svn: 348021
This patch splits off some abstractions used by dsymutil's dwarf linker
and moves them into separate header and implementation files. This
almost halves the number of LOC in DwarfLinker.cpp and makes it a lot
easier to understand what functionality lives where.
Differential revision: https://reviews.llvm.org/D48647
llvm-svn: 335749
Create convenience functions for printing error, warning and note to
stdout. Previously we had similar functions being used in dsymutil, but
given that this pattern is so common it makes sense to make it available
globally.
llvm-svn: 330091
When running dsymutil as part of your build system, it can be desirable
for warnings to be part of the end product, rather than just being
emitted to the output stream. This patch upstreams that functionality.
Differential revision: https://reviews.llvm.org/D44639
llvm-svn: 328965
Make the architecture part of the warning in the DebugMapParser. This
makes things consistent with the Apple's internal version of dsymutil.
llvm-svn: 327485
This is a follow-up to r327137 where we unified error handling for the
DwarfLinker. This replaces calls to errs() and outs() with the
appropriate ostream wrapper everywhere in dsymutil.
llvm-svn: 327411
Summary:
swiftc emits symbols without flags set, which led dsymutil to ignore
them when searching for global symbols, causing dwarf location data
to be omitted. Xcode's dsymutil handles this case correctly, and emits
valid location data. Add this functionality to llvm-dsymutil by
allowing parsing of symbols with no flags set.
Reviewers: aprantl, friss, JDevlieghere
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38587
llvm-svn: 315218
Summary:
swiftc emits symbols without flags set, which led dsymutil to ignore
them when searching for global symbols, causing dwarf location data
to be omitted. Xcode's dsymutil handles this case correctly, and emits
valid location data. Add this functionality to llvm-dsymutil by
allowing parsing of symbols with no flags set.
Reviewers: aprantl, friss, JDevlieghere
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38587
llvm-svn: 315082
Summary:
Xcode's dsymutil emits a __swift_ast DWARF section, which is required for debugging,
and which contains a byte-for-byte dump of the swiftmodule file.
Add this feature to llvm-dsymutil.
Tested with `gobjdump --dwarf=info -s`, by verifying that the contents of
`__DWARF.__swift_ast` match between Xcode's dsymutil and llvm-dsymutil
(Xcode's dwarfdump and llvm-dwarfdump don't currently recognize the
__swift_ast section).
Reviewers: aprantl, friss
Subscribers: llvm-commits, JDevlieghere
Differential Revision: https://reviews.llvm.org/D38504
llvm-svn: 315066
Summary:
Xcode's dsymutil emits a __swift_ast DWARF section, which is required for debugging,
and which contains a byte-for-byte dump of the swiftmodule file.
Add this feature to llvm-dsymutil.
Tested with `gobjdump --dwarf=info -s`, by verifying that the contents of
`__DWARF.__swift_ast` match between Xcode's dsymutil and llvm-dsymutil
(Xcode's dwarfdump and llvm-dwarfdump don't currently recognize the
__swift_ast section).
Reviewers: aprantl, friss
Subscribers: llvm-commits, JDevlieghere
Differential Revision: https://reviews.llvm.org/D38504
llvm-svn: 315014
Summary:
Xcode's dsymutil emits a __swift_ast DWARF section, which is required for debugging,
and which contains a byte-for-byte dump of the swiftmodule file.
Add this feature to llvm-dsymutil.
Tested with `gobjdump --dwarf=info -s`, by verifying that the contents of
`__DWARF.__swift_ast` match between Xcode's dsymutil and llvm-dsymutil
(Xcode's dwarfdump and llvm-dwarfdump don't currently recognize the
__swift_ast section).
Reviewers: aprantl, friss
Subscribers: llvm-commits, JDevlieghere
Differential Revision: https://reviews.llvm.org/D38504
llvm-svn: 315004
