Currently, WebAssembly/WASI target does not provide direct support for
code coverage.
This patch set fixes several issues to unlock the feature. The main
changes are:
1. Port `compiler-rt/lib/profile` to WebAssembly/WASI.
2. Adjust profile metadata sections for Wasm object file format.
- [CodeGen] Emit `__llvm_covmap` and `__llvm_covfun` as custom sections
instead of data segments.
- [lld] Align the interval space of custom sections at link time.
- [llvm-cov] Copy misaligned custom section data if the start address is
not aligned.
- [llvm-cov] Read `__llvm_prf_names` from data segments
3. [clang] Link with profile runtime libraries if requested
See each commit message for more details and rationale.
This is part of the effort to add code coverage support in Wasm target
of Swift toolchain.
Instead of always generating __wasm_apply_data_relocs when relevant
options like -pie and -shared are specified, generate it only when the
relevant relocations are actually necessary.
Note: omitting empty __wasm_apply_data_relocs is not a problem because
the export is optional in the spec (DynamicLinking.md) and all runtime
linker implementations I'm aware of implement it that way. (emscripten,
toywasm, wasm-tools)
Motivations:
* This possibly reduces the module size
* This is also a preparation to fix
https://github.com/llvm/llvm-project/issues/107387, for which it isn't
obvious if we need these relocations at the time of
createSyntheticSymbols. (unless we introduce a new explicit option like
--non-pie-dynamic-link.)
Followup to #104926.
We ran into issues on the emscripten waterfall where relocation against
`__dso_handle` were being reported as errors even though
`-r/--relocatable` was being used to generate object file output rather
than executable output.
`WASM_MEMORY_ADDR_REL_` and `WASM_TABLE_INDEX_REL_` relocations against
**undefined symbols** are not supported and, except for
`UnresolvedPolicy::ReportError`, lead to incorrect Wasm code, such as
invalid data address or invalid table index that cannot be patched
during later dynamic Wasm linking with modules declaring those symbols.
This is different to other relocations that support undefined symbols by
declaring correspond Wasm imports.
For more robust behavior, `wasm-ld` should probably report an error for
such unsupported PIC relocations, independent of the `UnresolvedPolicy`.
* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Strip calls to raw_string_ostream::str(), to avoid excess layer of indirection.
Add `allow-multiple-definition` flag to `wasm-ld`. This follows the ELF
linker logic. In case of duplication, the first symbol met is used.
This PR resolves the #97543
This change is enough to allow `--strip-debug` to work on object files,
without breaking the relocation information or symbol table.
A more complete version of this change would instead reconstruct the
symbol table and relocation sections, but that is much larger change.
Bug: #102002
This reverts commit 740161a9b98c9920dedf1852b5f1c94d0a683af5.
I moved the `ISD` dependencies into the CodeGen portion of the handling,
it's a little awkward but it's the easiest solution I can think of for
now.
We don't currently have a great way to detect the architecture of shared
object files under wasm. The currently method involves checking if the
imported or exported memory is 64-bit. However some shared libraries
don't use linear memory at all.
See https://github.com/llvm/llvm-project/issues/98778
Summary:
The LTO pass and LLD linker have logic in them that forces extraction
and prevent internalization of needed runtime calls. However, these
currently take all RTLibcalls into account, even if the target does not
support them. The target opts-out of a libcall if it sets its name to
nullptr. This patch pulls this logic out into a class in the header so
that LTO / lld can use it to determine if a symbol actually needs to be
kept.
This is important for targets like AMDGPU that want to be able to use
`lld` to perform the final link step, but does not want the overhead of
uncalled functions. (This adds like a second to the link time trivially)
Previously we would ignore all undefined symbols when using
`-shared` or `-pie`. All undefined symbols would be treated as imports
regardless of whether those symbols we defined in any shared library.
With this change we now track symbol in shared libraries and report
undefined symbols in the main program by default.
The old behavior is still available via the
`--unresolved-symbols=import-dynamic` command line flag.
This rationale for allowing this type of breaking change is that `-pie`
and `-shared` are both still experimental will warn as such, unless
`--experimental-pic` is passed.
As part of this change the linker now models shared library symbols
via new SharedFunctionSymbol and SharedDataSymbol types.
I've also added a new `--no-shlib-sigcheck` option that bypassed the
checking of functions signature in shared libraries. This is
specifically required by emscripten the case where the imports/exports
of shared libraries have been modified by via JS type legalization (this
is only needed when targeting old JS engines where bigint is not yet
available
See https://github.com/emscripten-core/emscripten/issues/18198
When an undefined symbol is referenced from more than one file we were
reporting all undefined symbols as originating from just one of them.
This came up while working on
https://github.com/WebAssembly/tool-conventions/issues/158 where
undefined symbols in one object file were being reported as coming from
another.
When reference types are enabled clang will generate call_indirect
instructions that explicitly reference the global
`__indirect_function_table` symbol.
In this case the resulting global symbol was not being correctly marked
with explicit import name/module, resulting in the linker reporting
errors when it was referenced.
This issue was reported in
https://github.com/WebAssembly/tool-conventions/issues/158
When two or more functions' signatures differ, one of them is selected
and for other signatures `unreachable` stubs are generated:
57778ec36c/lld/wasm/SymbolTable.cpp (L975)57778ec36c/lld/wasm/SymbolTable.cpp (L852-L870)
And when these `SyntheticFunction`s are generated, this constructor is
used,
57778ec36c/lld/wasm/InputChunks.h (L266-L269)
which does not set its `function` field:
57778ec36c/lld/wasm/InputChunks.h (L304)
As a result, the `function` field contains a garbage value for these
stub functions.
`InputFunction::getFunctionCodeOffset()` is called when relocations are
resolved for `.debug_info` section to get functions' PC locations. But
because these stub functions don't have their `function` field set, this
function segfaults:
57778ec36c/lld/wasm/InputChunks.h (L282)
This bug seems to be triggered when these conditions are met:
- There is a signature mismatch warning with multiple different
definitions (one definition with other declarations is not sufficient)
with weak linkage with the same name
- The 'stub' function containing unreachable has a callsite, meaning it
isn't DCE'd
- .debug_info section is generated (i.e., DWARF is used)
This PR initializes the field with `nullptr`, and in
`InputFunction::getFunctionCodeOffset`, checks if `function` is
`nullptr`, and if so, just returns 0. This function is called only for
resolving relocations in the `.debug_info` section, and addresses of
these stub functions, which are not the functions users wrote in the
first place, are not really meaningful anyway.
Search *.so libraries regardless of -pie to make it a bit more
straightforward to build non-pie dynamic-linked executables.
Flip the default to -Bstatic (unless -pie or -shared is specified) as I
think it's what most users expect for the default as of today.
The assumption here is that, because dynamic-linking is not widely used
for WebAssembly, the most users do not specify -Bdynamic or -Bstatic,
expecting static link.
Although the recent wasi-sdk ships *.so files, there are not many wasm
runtimes providing the support of dynamic-linking. (only emscripten and
toywasm as far as i know.)
I'm planning to remove StringRef::equals in favor of
StringRef::operator==.
- StringRef::operator==/!= outnumber StringRef::equals by a factor of
276 under llvm-project/ in terms of their usage.
- The elimination of StringRef::equals brings StringRef closer to
std::string_view, which has operator== but not equals.
- S == "foo" is more readable than S.equals("foo"), especially for
!Long.Expression.equals("str") vs Long.Expression != "str".
And use it to print the correct default OpenMP version for flang and
flang -fc1.
This change adds an optional `HelpTextsForVariants` to options. This
allows you to change the help text that gets shown in documentation and
`--help` based on the program its being generated for.
As `OptTable` needs to be constexpr compatible, I have used a std::array
of help text variants. Each entry is:
(list of visibilities) - > help text string
So for the OpenMP version we have (flang, fc1) -> "OpenMP version for
flang is...".
So you can have multiple visibilities use the same string. The number of
entries is currently set to 1, and the number of visibilities per entry
is 2, because that's the maximum we need for now. The code is written so
we can increase these numbers later, and the unused elements will be initialised.
I have not applied this to group descriptions just because I don't know
of one that needs changing. It could easily be enabled for those too if
needed. There are minor changes to them just to get it all to compile.
This approach of storing many help strings per option in the 1 driver
library seemed preferable to making a whole new library for Flang (even
if that would mostly be including stuff from Clang).
We recently added `--initial-heap` - an option that allows one to up the
initial memory size without the burden of having to know exactly how
much is needed.
However, in the process of implementing support for this in Emscripten
(https://github.com/emscripten-core/emscripten/pull/21071), we have
realized that `--initial-heap` cannot support the use-case of
non-growable memories by itself, since with it we don't know what to set
`--max-memory` to.
We have thus agreed to move the above work forward by introducing
another option to the linker (see
https://github.com/emscripten-core/emscripten/pull/21071#discussion_r1491755616),
one that would allow users to explicitly specify they want a
non-growable memory.
This change does this by introducing `--no-growable-memory`: an option
that is mutally exclusive with `--max-memory` (for simplicity - we can
also decide that it should override or be overridable by `--max-memory`.
In Emscripten a similar mix of options results in `--no-growable-memory`
taking precedence). The option specifies that the maximum memory size
should be set to the initial memory size, effectively disallowing memory
growth.
Closes#81932.
In WebAssembly, we have `WASM_SYMBOL_NO_STRIP` symbol flag to mark the
referenced content as retained. However, the flag is not enough to
express retained data that is not referenced by any symbol. This patch
adds a new segment flag`WASM_SEG_FLAG_RETAIN` to support "private"
linkage data that is retained by llvm.used.
This kind of data that is not referenced but must be retained is usually
used with encapsulation symbols (__start/__stop). Swift runtime uses
this technique and depends on the fact "all metadata sections in live
objects are retained", which was not guaranteed with `--gc-sections`
before this patch.
This is a revised version of https://reviews.llvm.org/D126950 (has been
reverted) based on @MaskRay's comments
Move the WasmSymbolInfos from their own vector on the WasmLinkingData
directly into the WasmSymbol object. Removing the const-ref to an
external object allows the vector of WasmSymbols to be safely
expanded/reallocated; generating symbol info from the name section will
require this, as the numbers of function and data segment names are
stored separately.
This is a step toward generating symbol information from name sections
for #76107
This change allows a WasmObjectFile to be created from a wasm file even
if it uses typed funcrefs and GC types. It does not significantly change how
lib/Object models its various internal types (e.g. WasmSignature,
WasmElemSegment), so LLVM does not really "support" or understand such
files, but it is sufficient to parse the type, global and element sections, discarding
types that are not understood. This is useful for low-level binary tools such as
nm and objcopy, which use only limited aspects of the binary (such as function
definitions) or deal with sections as opaque blobs.
This is done by allowing `WasmValType` to have a value of `OTHERREF`
(representing any unmodeled reference type), and adding a field to
`WasmSignature` indicating it's a placeholder for an unmodeled reference
type (since there is a 1:1 correspondence between WasmSignature objects
and types in the type section).
Then the object file parsers for the type and element sections are expanded
to parse encoded reference types and discard any unmodeled fields.
Followup to #78658, which caused a regression in emscripten.
When a lazy symbol is added, which resolved and existing undefined
symbol, we don't need/want to replace the undefined symbol with the lazy
one. Instead we called extract, which replaces the undefined symbol with
the defined one.
The fact that we were first replacing the undefined symbol with a lazy
one before extracting the archive member doesn't normally matter but, in
the case of the function symbol, replacing the undefined symbol with a
lazy symbol means that `addDefinedFunction` sees the existing symbol as
lazy and simply replaces it.
Note that this is consistent with both the ELF code in
`Symbol::resolve(const LazySymbol &other)` and the wasm code prior to
#78658, neither of which replace the existing symbol with the lazy one
in this case.
The ELF linker transitioned away from archive indexes in
https://reviews.llvm.org/D117284.
This paves the way for supporting `--start-lib`/`--end-lib` (See #77960)
The ELF linker unified library handling with `--start-lib`/`--end-lib` and removed
the ArchiveFile class in https://reviews.llvm.org/D119074.
This mirrors how the ELF linker works. I wasn't able to find anywhere
where this is currently tested.
Followup to #78640, which triggered a regression.
When undefined functions exist in the final link we need to create
stub functions (otherwise direct calls to those functions could
not be generated). We were creating those stub when
`--unresolved-symbols=ignore-all` was passed but overlooked the fact
that `--warn-unresolved-symbols` essentially has the same effect (i.e.
undefined function can exist in the final link).
Fixes: #53987
Also convert from std::vector to SmallVector.
This matches the ELF linker where these were moved into the ctx object
in 9a572164d592e and converted to SmallVector in ba948c5a9c524b.
LLVM models some features found in the binary format with raw integers
and others with nested or enumerated types. This PR switches modeling of
tables and segments to use wasm::ValType rather than uint32_t. This NFC
change is in preparation for modeling more reference types, but IMO is
also cleaner and closer to the spec.
It is beneficial to preallocate a certain number of pages in the linear
memory (i. e. use the "minimum" field of WASM memories) so that fewer
"memory.grow"s are needed at startup.
So far, the way to do that has been to pass the "--initial-memory"
option to the linker. It works, but has the very significant downside of
requiring the user to know the size of static data beforehand, as it
must not exceed the number of bytes passed-in as "--initial-memory".
The new "--initial-heap" option avoids this downside by simply appending
the specified number of pages to static data (and stack), regardless of
how large they already are.
Ref: https://github.com/emscripten-core/emscripten/issues/20888.
When doing LTO on multiple archives, the order with which bitcodes are
linked to the LTO module is hard to control, given that processing
undefined symbols can lead to parsing of an object file, which in turn
lead to parsing of another object file before finishing parsing of the
previous file. This can result in encountering a non-prevailing comdat
first when linking, which can make the the symbol undefined, and the
real definition is added later with an additional prefix to avoid
duplication (e.g. `__cxx_global_var_init` and `__cxx_global_var_init.2`)
So this one-line fix ensures we compile bitcodes in the order that we
process comdats, so that when multiple archived bitcode files have the
same variable with the same comdat, we make sure that the prevailing
comdat will be linked first in the LTO.
Fixes#62243.
