Note that PointerUnion::{is,get} have been soft deprecated in
PointerUnion.h:
// FIXME: Replace the uses of is(), get() and dyn_cast() with
// isa<T>, cast<T> and the llvm::dyn_cast<T>
I'm not touching PointerUnion::dyn_cast for now because it's a bit
complicated; we could blindly migrate it to dyn_cast_if_present, but
we should probably use dyn_cast when the operand is known to be
non-null.
This patch enhances the robustness of lld's Objective-C category
merging. Currently, the category merger assumes it can fully parse and
understand the format of all categories in the input, triggering an
assert if any invalid category data is encountered.
This will end up causing asserts in certain rare corner cases that are
difficult to reproduce in small test cases. The proposed changes modify
the behavior so that if invalid category data is detected, category
merging is skipped for that specific class and all other categories
sharing the same base class. This approach allows the linker to continue
processing other categories without failing entirely due to a single
problematic input.
We also add a LIT test to where we corrupt category data and check that
category merging for that class was skipped but the link was successful.
A crash was happening when both ObjC Category Merging and Relative
method lists were enabled.
ObjC Category Merging creates new data sections and adds them by calling
`addInputSection`. `addInputSection` uses the symbols within the added
section to determine which container to actually add the section to.
The issue is that ObjC Category merging is calling `addInputSection`
before actually adding the relevant symbols the the added section. This
causes `addInputSection` to add the `InputSection` to the wrong
container, eventually resulting in a crash.
To fix this, we ensure that ObjC Category Merging calls
`addInputSection` only after the symbols have been added to the
`InputSection`.
Local data is referenced in Objective-C metadata via section + offset
relocations on x86-64 rather than via symbols. Without this change, we
would crash on incorrect casts of the referents to `Defined`.
A basic test based on the existing `objc-relative-method-lists-simple.s`
adopted to x86-64 is added.
We were already not deleting category names for Swift classes as those
names can be reused by other parts. However, I have come across a corner
case where this also happens for ObjC categories - so we can't delete
category names for them either. TODO remains to optimize this behavior
for both ObjC and Swift.
This section contains metadata that's only relevant for Identical Code
Folding at link time, we should not include it in the output.
We still treat it like a regular section during input file parsing (e.g.
create a `ConcatInputSection` for it), as we want its relocations to be
parsed. But it should not be passed to `addInputSection`, as that's what
assigns it to an `OutputSection` and adds it to the `inputSections`
vector which specifies the inputs to dead-stripping and relocation
scanning.
This fixes a "__DATA,__llvm_addrsig, offset 0: fixups overlap" error
when using `--icf=safe` alongside `-fixup_chains`. This occurs because
all `__llvm_addrsig` sections are 8 bytes large, and the relocations
which signify functions whose addresses are taken are all at offset 0.
This makes the fix in 5fa24ac2 ("Category Merger: add support for
addrsig references") obsolete, as we no longer try to resolve symbols
referenced in `__llvm_addrsig` when writing the output file. When we do
iterate its relocations in `markAddrSigSymbols`, we do not try to
resolve their addresses.
Currently we rely on detecting the source language of a class by the
name of the symbol pointing to it. This generally works, but there are
scenarios where Swift generates both an ObjC name and a Swift name for
the symbol - by aliasing the ObjC name to the Swift name, as follows:
```
.globl _OBJC_CLASS_$__TtC11MyTestClass11MyTestClass
.private_extern _OBJC_CLASS_$__TtC11MyTestClass11MyTestClass
.set _OBJC_CLASS_$__TtC11MyTestClass11MyTestClass, _$s11MyTestClassAACN
```
So to correctly detect the source language in all cases, we need to
handle scenarios where both an ObjC symbol and a Swift symbol is defined
for a class - in this case the symbol is always a Swift class.
Currently the `tryFindDefinedOnIsec` takes in an `InputSection` and an
`offset` and is supposed to return the target symbol that is referenced
on that `InputSection` at the given offset.
However, it does not deal with the reloc `addend` and might return the
incorrect symbol.
Here we add support for handling the reloc's `addend`.
This change adds tracking for the source language of the various input
structs used by the category merger. Identification is based on expected
symbol names. It also adds checks to ensure we're dealing with the
expected data in known scenarios.
Currently ObjC category merger only supports categories defined in ObjC.
But swift supports generating ObjC categories also. This change adds
supports for the later categories.
Without this change, the build will fail like so:
llvm-project/lld/MachO/ObjC.cpp:1387:75: error: ISO C++20 considers use of overloaded operator '==' (with operand types 'ObjcCategoryMerger::PointerListInfo' and 'ObjcCategoryMerger::PointerListInfo') to be ambiguous despite there being a unique best viable function [-Werror,-Wambiguous-reversed-operator]
1387 | parseProtocolListInfo(classIsec, roClassLayout.baseProtocolsOffset) ==
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
1388 | parseProtocolListInfo(metaIsec, roClassLayout.baseProtocolsOffset) &&
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/assert.h💯27: note: expanded from macro 'assert'
100 | (static_cast <bool> (expr) \
| ^~~~
llvm-project/lld/MachO/ObjC.cpp:391:17: note: ambiguity is between a regular call to this operator and a call with the argument order reversed
391 | inline bool operator==(const PointerListInfo &cmp) {
| ^
llvm-project/lld/MachO/ObjC.cpp:391:17: note: mark 'operator==' as const or add a matching 'operator!=' to resolve the ambiguity
1 error generated.
Currently category merging only supports merging multiple categories
into one. With this commit we add the ability to fully merge categories
into the base class, if the base class is included in the current
module. This is the optimal approach for defined classes.
FIxing the address sanitizer issue reported in
https://github.com/llvm/llvm-project/pull/91548 .
The problem comes from the assignment `auto bodyData = newSectionData`
which defaults to `SmallVector<uint8_t> data = newSectionData` - which
actually creates a copy of the data, placed on the stack.
By explicitly using `ArrayRef` instead, we make sure that the original
copy is used.
We also change the assignment in `ObjcCategoryMerger::newStringData`
from `auto` to `SmallVector<uint8_t> &` to make it explicit.
In ObjC we can have categories that define a `+load` method that is
called when the category is loaded. In such cases, we shouldn't optimize
the category. These categories are present in the `__objc_nlcatlist`
section. So we scan these section for such categories and ignore them
from optimization.
Currently in `ObjcCategoryMerger::doMerge` and
`generateCatListForNonErasedCategories` we use maps of pointers which
leads to non-determinism. Switch instead to using `MapVector` which
preserves determinism.
In some cases we see strings from categories being part of "data"
sections (Ex:`__objc_const`), not part of of sections marked as
`cstring_literals`. Since lld treats these sections differently we need
to explicitly implement support for reading strings from the
non-`cstring_literals` sections.
Adding a test that previously would result in an assert.
When generating categories, clang sometimes will generate references in
the `.addrsig` section to the various category data items. Since we may
erase such items after merging them, we also need to remove them from
the `.addrsig` section - otherwise this will cause runtime asserts with
the `.addrsig` section trying to access invalid data.
Implementation wise, we use a hashset to keep track of all erased
`InputSection`'s and then go through all `.addrsig` sections and remove
references to any erased `InputSection`.
When generating symbols for various category constructs, make sure the
symbol names match the format of those generated by ld64 when it does
category merging.
We modify category merging code to simplify it, as follows:
- We can simplify InfoWriteSection to not be templated - this is not
really necessary.
- We remove PointerListInfo::categoryOffset as it is not used.
Currently, when moving symbols from one `InputSection` to another (like
in ICF) we directly update the symbol's `isec`, `unwindEntry` and
`size`. By doing this we lose the original information. This information
will be needed in a future change. Since when moving symbols we always
set the symbol's `wasCoalesced` and `isec-> replacement`, we can just
use this info to conditionally get the information we need at access
time.
Fixing gcc warning regarding creating non-null-terminated string:
```
../../lld/MachO/ObjC.cpp:1226:10: warning: 'char* strncpy(char*, const char*, size_t)' output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
1226 | strncpy(strData, str, len);
| ~~~~~~~^~~~~~~~~~~~~~~~~~~
../../lld/MachO/ObjC.cpp: In member function 'void {anonymous}::ObjcCategoryMerger::emitAndLinkPointerList(lld::macho::Defined*, uint32_t, const {anonymous}::ObjcCategoryMerger::ClassExtensionInfo&, const {anonymous}::ObjcCategoryMerger::PointerListInfo&)':
../../lld/MachO/ObjC.cpp:1223:24: note: length computed here
1223 | uint32_t len = strlen(str);
| ~~~~~~^~~~~
```
This is not actually a bug, as `newSectionData` returns a
zero-initialized memory region, so the null terminator will be there.
Before this change, after `InputSection` objects are created, they need
to be added to the appropriate container for tracking.
The logic for selecting the appropriate container lives in `Driver.cpp`
/ `gatherInputSections`, where the `InputSection` is added to the
matching container depending on the input config and the type of
`InputSection`.
Also, multiple other locations also insert directly into `inputSections`
array - assuming that that is the appropriate container for the
`InputSection`'s they create. Currently this is the correct assumption,
however an upcoming feature will change this.
For an upcoming feature (relative method lists), we need to route
`InputSection`'s either to `inputSections` array or to a synthetic
section, depending on weather the relative method list optimization is
enabled or not.
We can achieve the above either by duplicating some of the logic or
refactoring the routing and `InputSection`'s and reusing that.
The refactoring & code sharing approach seems the correct way to go - as
such this diff performs the refactoring while not introducing any
functional changes. Later on we can just call `addInputSection` and not
have to worry about routing logic.
---------
This change adds a flag to lld to enable category merging for MachoO +
ObjC.
It adds the '-objc_category_merging' flag for enabling this option and
uses the existing '-no_objc_category_merging' flag for disabling it.
In ld64, this optimization is enabled by default, but in lld, for now,
we require explicitly passing the '-objc_category_merging' flag in order
to enable it.
Behavior: if in the same link unit, multiple categories are extending
the same class, then they get merged into a single category.
Ex: Cat1(method1+method2,protocol1) + Cat2(method3+method4,protocol2,
property1) = Cat1_2(method1+method2+method3+method4,
protocol1+protocol2, property1)
Notes on implementation decisions made in this diff:
There is a possibility to further improve the current implementation by
directly merging the category data into the base class (if the base
class is present in the link unit) - this improvement may be done as a
follow-up. This improved functionality is already present in ld64.
We do the merging on the raw inputSections - after dead-stripping
(categories can't be dead stripped anyway).
The changes are mostly self-contained to ObjC.cpp, except for adding a
new flag (linkerOptimizeReason) to ConcatInputSection and StringPiece to
mark that this data has been optimized away. Another way to do it would
have been to just mark the pieces as not 'live' but this would cause the
old symbols to show up in the linker map as being dead-stripped - even
if dead-stripping is disabled. This flag allows us to match the ld64
behavior.
Note: This is a re-land of
https://github.com/llvm/llvm-project/pull/82928 after fixing using
already freed memory in `generatedSectionData`. This issue was detected
by ASAN build.
---------
Co-authored-by: Alex B <alexborcan@meta.com>
This change adds a flag to lld to enable category merging for MachoO +
ObjC.
It adds the '-objc_category_merging' flag for enabling this option and
uses the existing '-no_objc_category_merging' flag for disabling it.
In ld64, this optimization is enabled by default, but in lld, for now,
we require explicitly passing the '-objc_category_merging' flag in order
to enable it.
Behavior: if in the same link unit, multiple categories are extending
the same class, then they get merged into a single category.
Ex: `Cat1(method1+method2,protocol1) + Cat2(method3+method4,protocol2,
property1) = Cat1_2(method1+method2+method3+method4,
protocol1+protocol2, property1)`
Notes on implementation decisions made in this diff:
1. There is a possibility to further improve the current implementation
by directly merging the category data into the base class (if the base
class is present in the link unit) - this improvement may be done as a
follow-up. This improved functionality is already present in ld64.
2. We do the merging on the raw inputSections - after dead-stripping
(categories can't be dead stripped anyway).
3. The changes are mostly self-contained to ObjC.cpp, except for adding
a new flag (linkerOptimizeReason) to ConcatInputSection and StringPiece
to mark that this data has been optimized away. Another way to do it
would have been to just mark the pieces as not 'live' but this would
cause the old symbols to show up in the linker map as being
dead-stripped - even if dead-stripping is disabled. This flag allows us
to match the ld64 behavior.
---------
Co-authored-by: Alex B <alexborcan@meta.com>
When printing category conflicts in the ObjC category checker, also
print the source file name of the problematic categories. Currently we
only print the object file name. This change is mostly useful only for
thinLTO builds, where the object file name will be of form
999.arm64.lto.o and thus does not reveal any information about the
original source file.
---------
Co-authored-by: Alex Borcan <alexborcan@meta.com>
+load methods are static initializers and treated specially by the
runtime: all +load methods for a class & its categories are called when
loading that class, unlike regular methods where only one definition
will get called per message. Thus, there is no need to check for
duplicates.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D147230
We were previously assuming that to hold, but it appears that `swiftc`
may generate classes that link against `__objc_empty_cache` for their
rodata pointer.
This should allow us to re-land {D142916} (as a stack).
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D147229
This implements ld64's checks for duplicate method names in categories &
classes.
In addition, this sets us up for implementing Obj-C category merging.
This diff handles the most of the parsing work; what's left is rewriting
those category / class structures.
Numbers for chromium_framework:
base diff difference (95% CI)
sys_time 2.182 ± 0.027 2.200 ± 0.047 [ -0.2% .. +1.8%]
user_time 6.451 ± 0.034 6.479 ± 0.062 [ -0.0% .. +0.9%]
wall_time 6.841 ± 0.048 6.885 ± 0.105 [ -0.1% .. +1.4%]
samples 33 22
Fixes https://github.com/llvm/llvm-project/issues/54912.
Issues seen with the previous land will be fixed in the next commit.
Reviewed By: #lld-macho, thevinster, oontvoo
Differential Revision: https://reviews.llvm.org/D142916
This implements ld64's checks for duplicate method names in categories &
classes.
In addition, this sets us up for implementing Obj-C category merging.
This diff handles the most of the parsing work; what's left is rewriting
those category / class structures.
Numbers for chromium_framework:
base diff difference (95% CI)
sys_time 2.182 ± 0.027 2.200 ± 0.047 [ -0.2% .. +1.8%]
user_time 6.451 ± 0.034 6.479 ± 0.062 [ -0.0% .. +0.9%]
wall_time 6.841 ± 0.048 6.885 ± 0.105 [ -0.1% .. +1.4%]
samples 33 22
Fixes https://github.com/llvm/llvm-project/issues/54912.
Reviewed By: #lld-macho, thevinster, oontvoo
Differential Revision: https://reviews.llvm.org/D142916
This is an NFC diff that prepares for pruning & relocating `__eh_frame`.
Along the way, I made the following changes to ...
* clarify usage of `section` vs. `subsection`
* remove `map` & `vec` from type names
* disambiguate class `Section` from template parameter `SectionHeader`.
Differential Revision: https://reviews.llvm.org/D113241
The previous logic was duplicated between symbol-initiated
archive loads versus flag-initiated loads (i.e. `-force_load` and
`-ObjC`). This resulted in code duplication as well as redundant work --
we would create Archive instances twice whenever we had one of those
flags; once in `getArchiveMembers` and again when we constructed the
ArchiveFile.
This was motivated by an upcoming diff where we load archive members
containing ObjC-related symbols before loading those containing
ObjC-related sections, as well as before performing symbol resolution.
Without this refactor, it would be difficult to do that while avoiding
loading the same archive member twice.
Differential Revision: https://reviews.llvm.org/D108780
@thakis pointed out that `mach_header` and `mach_header_64`
actually have the same set of (used) fields, with the 64-bit version
having extra padding. So we can access the fields we need using the
single `mach_header` type instead of using templates to switch between
the two.
I also spotted a potential issue where hasObjCSection tries to parse a
file w/o checking if it does indeed match the target arch... As such,
I've added a quick magic number check to ensure we don't access invalid
memory during `findCommand()`.
Addresses PR50180.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D101724
As preparation for a subsequent diff that implements builtin section renaming, define more `constexpr` strings in namespaces `lld::macho::segment_names` and `lld::macho::section_names`, and use them to replace string literals.
Differential Revision: https://reviews.llvm.org/D101393
The main challenge was handling the different on-disk structures (e.g.
`mach_header` vs `mach_header_64`). I tried to strike a balance between
sprinkling `target->wordSize == 8` checks everywhere (branchy = slow, and ugly)
and templatizing everything (causes code bloat, also ugly). I think I struck a
decent balance by judicious use of type erasure.
Note that LLD-ELF has a similar architecture, though it seems to use more templating.
Linking chromium_framework takes about the same time before and after this
change:
N Min Max Median Avg Stddev
x 20 4.52 4.67 4.595 4.5945 0.044423204
+ 20 4.5 4.71 4.575 4.582 0.056344803
No difference proven at 95.0% confidence
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D99633
