767 Commits

Author SHA1 Message Date
SingleAccretion
b2cdf3cc4c
[lld][WebAssembly] Add an --initial-heap option (#75594)
It is beneficial to preallocate a certain number of pages in the linear
memory (i. e. use the "minimum" field of WASM memories) so that fewer
"memory.grow"s are needed at startup.

So far, the way to do that has been to pass the "--initial-memory"
option to the linker. It works, but has the very significant downside of
requiring the user to know the size of static data beforehand, as it
must not exceed the number of bytes passed-in as "--initial-memory".

The new "--initial-heap" option avoids this downside by simply appending
the specified number of pages to static data (and stack), regardless of
how large they already are.

Ref: https://github.com/emscripten-core/emscripten/issues/20888.
2023-12-15 10:16:38 -08:00
Sam Clegg
97b25d91df
[lld][WebAssembly] Don't set importUndefined when -shared is used. NFC (#75241)
`importUndefined` is only used a couple of places and both of those
already handle `isPIC` separately.
2023-12-12 13:19:42 -08:00
Heejin Ahn
83305faeb5
[lld][WebAssembly] Fix bitcode LTO order in archive parsing (#73095)
When doing LTO on multiple archives, the order with which bitcodes are
linked to the LTO module is hard to control, given that processing
undefined symbols can lead to parsing of an object file, which in turn
lead to parsing of another object file before finishing parsing of the
previous file. This can result in encountering a non-prevailing comdat
first when linking, which can make the the symbol undefined, and the
real definition is added later with an additional prefix to avoid
duplication (e.g. `__cxx_global_var_init` and `__cxx_global_var_init.2`)

So this one-line fix ensures we compile bitcodes in the order that we
process comdats, so that when multiple archived bitcode files have the
same variable with the same comdat, we make sure that the prevailing
comdat will be linked first in the LTO.

Fixes #62243.
2023-11-28 17:44:32 -08:00
Sam Clegg
89d5635f0a [lld][WebAssembly] Add --keep-section flag
This flag causes wasm-ld preserve a section even in the face of
`--strip-all`.  This is useful, for example, to preserve the
target_features section in the ase of clang (which can run wasm-opt
after linking), and emcc (which performs a bunch of post-link work).

Fixes: https://github.com/llvm/llvm-project/issues/60613
Fixes: https://github.com/llvm/llvm-project/issues/55781

Differential Revision: https://reviews.llvm.org/D149917
2023-11-02 14:23:45 -07:00
Kazu Hirata
4a0ccfa865 Use llvm::endianness::{big,little,native} (NFC)
Note that llvm::support::endianness has been renamed to
llvm::endianness while becoming an enum class as opposed to an
enum. This patch replaces support::{big,little,native} with
llvm::endianness::{big,little,native}.
2023-10-12 21:21:45 -07:00
Sam Clegg
afe957ea95
[WebAssembly] Allow absolute symbols in the linking section (symbol table) (#67493)
Fixes a crash in `-Wl,-emit-relocs` where the linker was not able to
write linker-synthetic absolute symbols to the symbol table.

This change adds a new symbol flag (`WASM_SYMBOL_ABS`), which means that
the symbol's offset is absolute and not relative to a given segment.
Such symbols include `__stack_low` and `__stack_low`.

Note that wasm object files never contains such symbols, only binaries
linked with `-Wl,-emit-relocs`.

Fixes: #67111
2023-10-03 13:16:16 -07:00
Sam Clegg
7bac0bc115
[lld][WebAssembly] Improve error message on adding LTO object post-LTO. NFC (#66688)
We have been getting errors from emscripten users where including the
name of the symbol that triggered the inclusion would be useful in the
diagnosis. e.g:
https://github.com/emscripten-core/emscripten/issues/20275
2023-09-18 14:11:49 -07:00
Arthur Eubanks
0a1aa6cda2
[NFC][CodeGen] Change CodeGenOpt::Level/CodeGenFileType into enum classes (#66295)
This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.

This matches other nearby enums.

For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::
2023-09-14 14:10:14 -07:00
Sam Clegg
93adcb770b [lld][WebAssembly] Add --table-base setting
This is similar to `--global-base` but determines where to place the
table segments rather than that data segments.

See https://github.com/emscripten-core/emscripten/issues/20097

Differential Revision: https://reviews.llvm.org/D158892
2023-08-25 16:10:38 -07:00
Sam Clegg
8e44f037dc [lld][WebAssembly] Add support for -soname
This change writes the module name to the name section of the wasm
binary.  We use the `-soname` argument to determine the name and we
default the output file basename if this option is not specified.

In the future we will likely want to embed the soname in the dylink
section too, but this the first step in supporting `-soname`.

Differential Revision: https://reviews.llvm.org/D158001
2023-08-15 18:33:45 -07:00
Sam Clegg
cb5bc75680 [lld][WebAssembly] Always include bss segments when --emit-relocs is used
If we don't do this then we end up with symbols that refer to
non-existent segments.

Differential Revision: https://reviews.llvm.org/D158025
2023-08-15 17:03:40 -07:00
Justin Bogner
dcb6d212fd Reapply "[Option] Add "Visibility" field and clone the OptTable APIs to use it"
This reverts commit 4e3b89483a6922d3f48670bb1c50a37f342918c6, with
fixes for places I'd missed updating in lld and lldb. I've also
renamed OptionVisibility::Default to "DefaultVis" to avoid ambiguity
since the undecorated name has to be available anywhere Options.inc is
included.

Original message follows:

This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149
2023-08-15 01:16:58 -07:00
Justin Bogner
4e3b89483a Revert "[Option] Add "Visibility" field and clone the OptTable APIs to use it"
this is failing on bots, reverting to investigate.

This reverts commit a16104e6da6f36f3d72dbf53d10ba56495a0d65a.
2023-08-14 13:31:02 -07:00
Justin Bogner
a16104e6da [Option] Add "Visibility" field and clone the OptTable APIs to use it
This splits OptTable's "Flags" field into "Flags" and "Visibility",
updates the places where we instantiate Option tables, and adds
variants of the OptTable APIs that use Visibility mask instead of
Include/Exclude flags.

We need to do this to clean up a bunch of complexity in the clang
driver's option handling - there's a whole slew of flags like
CoreOption, NoDriverOption, and FlangOnlyOption there today to try to
handle all of the permutations of flags that the various drivers need,
but it really doesn't scale well, as can be seen by things like the
somewhat recently introduced CLDXCOption.

Instead, we'll provide an additive model for visibility that's
separate from the other flags. For things like "HelpHidden", which is
used as a "subtractive" modifier for option visibility, we leave that
in "Flags" and handle it as a special case.

Note that we don't actually update the users of the Include/Exclude
APIs here or change the flags that exist in clang at all - that will
come in a follow up that refactors clang's Options.td to use the
increased flexibility this change allows.

Differential Revision: https://reviews.llvm.org/D157149
2023-08-14 13:24:54 -07:00
Sam Clegg
7a8ee92fa8 [lld][WebAssembly] Process stub libraries in a loop
When stub libraries trigger the fetching of new object files we can
potentially introduce new undefined symbols so process the stub in
loop until no new objects are pulled in.

Differential Revision: https://reviews.llvm.org/D153466
2023-08-11 11:07:40 -07:00
Jan Svoboda
3f092f37b7 [llvm] Extract common OptTable bits into macros
All command-line tools using `llvm::opt` create an enum of option IDs and a table of `OptTable::Info` object. Most of the tools use the same ID (`OPT_##ID`), kind (`Option::KIND##Class`), group ID (`OPT_##GROUP`) and alias ID (`OPT_##ALIAS`). This patch extracts that common code into canonical macros. This results in fewer changes when tweaking the `OPTION` macros emitted by the TableGen backend.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157028
2023-08-04 13:57:13 -07:00
YAMAMOTO Takashi
47ba908a5e [lld][WebAssembly] Fix func reloc for internal GOT with extended-const
Differential Revision: https://reviews.llvm.org/D155542
2023-07-27 00:46:00 -07:00
Fangrui Song
8db61ed29d [WebAssembly] Stabilize custom section order
It currently depends on the StringMap iteration order, which is not
guaranteed to be deterministic. Use MapVector to stabilize the order.
2023-07-22 09:37:22 -07:00
Fangrui Song
a3622ac80b [wasm-ld] Switch to xxh3_64bits
Similar to recent changes to ELF (e.g., D154813), Mach-O, and COFF to
improve hashing performance.

Reviewed By: dschuff

Differential Revision: https://reviews.llvm.org/D155752
2023-07-20 10:47:47 -07:00
Brendan Dahl
220fe00a7c [WebAssembly] Support annotate clang attributes for marking functions.
Annotation attributes may be attached to a function to mark it with
custom data that will be contained in the final Wasm file. The
annotation causes a custom section named
"func_attr.annotate.<name>.<arg0>.<arg1>..." to be created that will
contain each function's index value that was marked with the annotation.

A new patchable relocation type for function indexes had to be created so
the custom section could be updated during linking.

Reviewed By: sbc100

Differential Revision: https://reviews.llvm.org/D150803
2023-07-11 15:17:26 -07:00
Elliot Goodrich
b0abd4893f [llvm] Add missing StringExtras.h includes
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
2023-06-25 15:42:22 +01:00
Alexandre Ganea
6f2e92c10c Re-land [LLD] Allow usage of LLD as a library
This reverts commit aa495214b39d475bab24b468de7a7c676ce9e366.

As discussed in https://github.com/llvm/llvm-project/issues/53475 this patch
allows for using LLD-as-a-lib. It also lets clients link only the drivers that
they want (see unit tests).

This also adds the unit test infra as in the other LLVM projects. Among the
test coverage, I've added the original issue from @krzysz00, see:
https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

Important note: this doesn't allow (yet) linking in parallel. This will come a
bit later hopefully, in subsequent patches, for COFF at least.

Differential revision: https://reviews.llvm.org/D119049
2023-06-19 07:35:11 -04:00
Leonard Chan
aa495214b3 Revert "[LLD] Allow usage of LLD as a library"
This reverts commit 2700da5fe28d8b17c66e5c960d2188276a6ced39.

Reverting since this causes some test failures on our builders: https://ci.chromium.org/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8778372807208184913/overview
2023-06-14 20:36:27 +00:00
Alexandre Ganea
2700da5fe2 [LLD] Allow usage of LLD as a library
As discussed in https://github.com/llvm/llvm-project/issues/53475 this patch allows using LLD-as-a-lib. It also lets clients link only the drivers that they want (see unit tests).

This also adds the unit test infra as in the other LLVM projects. Among the test coverage, I've added the original issue from @krzysz00, see: https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

Important note: this doesn't allow (yet) linking in parallel. This will come a bit later, in subsequent patches, for COFF at last.

Differential revision: https://reviews.llvm.org/D119049
2023-06-13 16:22:59 -04:00
Nick Desaulniers
8abbc17ff3 reland: [Demangle] make llvm::demangle take std::string_view rather than const std::string&
As suggested by @erichkeane in
https://reviews.llvm.org/D141451#inline-1429549

There's potential for a lot more cleanups around these APIs. This is
just a start.

Callers need to be more careful about sub-expressions producing strings
that don't outlast the expression using `llvm::demangle`. Add a
release note.

Differential Revision: https://reviews.llvm.org/D149104
2023-06-06 10:18:06 -07:00
Fangrui Song
8d85c96e0e [lld] StringRef::{starts,ends}with => {starts,ends}_with. NFC
The latter form is now preferred to be similar to C++20 starts_with.
This replacement also removes one function call when startswith is not inlined.
2023-06-05 14:36:19 -07:00
Nico Weber
2e2e110464 fix typos to cycle bots 2023-05-10 19:53:55 +02:00
Nick Desaulniers
3e3c6f24ff Revert "[Demangle] make llvm::demangle take std::string_view rather than const std::string&"
This reverts commit c117c2c8ba4afd45a006043ec6dd858652b2ffcc.

itaniumDemangle calls std::strlen with the results of
std::string_view::data() which may not be NUL-terminated. This causes
lld/test/wasm/why-extract.s  to fail when "expensive checks" are enabled
via -DLLVM_ENABLE_EXPENSIVE_CHECKS=ON. See D149675 for further
discussion. Back this out until the individual demanglers are converted
to use std::string_view.
2023-05-02 15:54:09 -07:00
Nick Desaulniers
c117c2c8ba [Demangle] make llvm::demangle take std::string_view rather than const std::string&
As suggested by @erichkeane in
https://reviews.llvm.org/D141451#inline-1429549

There's potential for a lot more cleanups around these APIs. This is
just a start.

Callers need to be more careful about sub-expressions producing strings
that don't outlast the expression using ``llvm::demangle``. Add a
release note.

Reviewed By: MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D149104
2023-05-02 11:20:15 -07:00
Sam Clegg
d43f0889dd [lld][WebAssembly] stub objects: Fix handling of LTO libcall dependencies
This actually simplifies the code by performs a pre-pass of the stub
objects prior to LTO.

This should be the final change needed before we can make the switch
on the emscripten side: https://github.com/emscripten-core/emscripten/pull/18905

Differential Revision: https://reviews.llvm.org/D148287
2023-04-13 20:32:05 -07:00
Sam Clegg
aca110f9dd [lld][WebAssembly] Trace export of symbols when specified with --trace-symbol. NFC
Differential Revision: https://reviews.llvm.org/D148190
2023-04-13 10:36:27 -07:00
Kazu Hirata
d5a50b02e6 Fix warnings
This patch fixes:

  mlir/lib/Dialect/Linalg/TransformOps/LinalgTransformOps.cpp:1334:51:
  warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]

  lld/wasm/Writer.cpp:250:39: warning: suggest parentheses around ‘&&’
  within ‘||’ [-Wparentheses]
2023-04-10 10:11:41 -07:00
Sam Clegg
d9d840cdaf [lld][WebAssembly] Fix stub library parsing with windows line endings
Also, fix checking of first line in ::parse.  We can't use the
::getLines helper here since that already does comment stripping
internally.

Differential Revision: https://reviews.llvm.org/D147548
2023-04-04 11:55:01 -07:00
Sam Clegg
2ea8a3a56a [lld][WebAssembly] Process stub libraries before performing LTO
There are cases where stub library processing can trigger new exports
which might require them to be included at LTO time.

Specifically `processStubLibraries` marks symbols as `forceExports`
which even effect the LTO process.

And since the LTO process can generate new undefined symbols
(specifically libcall function) we need to also process the stub
libraries after LTO.

Differential Revision: https://reviews.llvm.org/D147190
2023-03-30 14:27:19 -07:00
Sam Clegg
3111784ff7 [lld][WebAssembly] Initial support for stub libraries
See the docs in lld/docs/WebAssembly.rst for more on this.

This feature unlocks a lot of simplification in the emscripten toolchain
since we can represent the JS libraries to wasm-ld as stub libraries.

See https://github.com/emscripten-core/emscripten/issues/18875

Differential Revision: https://reviews.llvm.org/D145308
2023-03-23 14:26:27 -07:00
Congcong Cai
467cf15428 [NFC] Fix typo lld::wasm in comment 2023-03-23 15:40:55 +08:00
Heejin Ahn
4e844a1498 [WebAssembly] Replace Bugzilla links with Github issues
Reviewed By: dschuff, asb

Differential Revision: https://reviews.llvm.org/D145966
2023-03-17 20:13:00 -07:00
Alexandre Ganea
7adaefbaa8 [wasm] Silence 'not all control paths return a value' warning when
building with MSVC on Windows
2023-03-15 14:20:18 -04:00
Sam Clegg
d32f71a91a [lld][WebAssembly] Use C++17 nested namespace syntax in most places. NFC
Like D131405, but for wasm-ld.

Differential Revision: https://reviews.llvm.org/D145399
2023-03-09 18:37:32 -08:00
Sam Clegg
8aef04fa69 [lld][WebAssembly] Implement --why-extract flag from the ELF backend
See https://reviews.llvm.org/D109572 for the original ELF version.

Differential Revision: https://reviews.llvm.org/D145431
2023-03-06 17:52:49 -08:00
Derek Schuff
c7af9ae577 Reapply: [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a
build_id section according to the WebAssembly tool convention
specified in https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the
output and (unlike ELF) generates a v5 UUID based on the hash (using a
random namespace). It also supports generating a random v4 UUID, a
sha1 hash, and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662

Fix MSVC build by std::copy on the underying buffer rather than
directly from std::array to llvm::MutableArrayRef
2023-03-03 17:15:55 -08:00
Derek Schuff
c8e5354f00 Revert "[WebAssembly] Implement build-id feature"
This reverts commit 41e31466af6a7feab82742bb01af43f4f3ae4ede
due to a build failure on Windows.
2023-03-02 15:28:56 -08:00
Derek Schuff
41e31466af [WebAssembly] Implement build-id feature
Implement the --build-id flag similarly to ELF, and generate a build_id
section according to the WebAssembly tool convention specified in
https://github.com/WebAssembly/tool-conventions/pull/183

The default style ("fast" aka "tree") hashes the contents of the output
and (unlike ELF) generates a v5 UUID based on the hash (using a random
namespace).
It also supports generating a random v4 UUID, a sha1 hash,
and a user-specified string (as ELF does).

Differential Revision: https://reviews.llvm.org/D107662
2023-03-02 14:19:06 -08:00
Sam Clegg
d65ed8cde0 [lld][WebAssembly] Fix handling of mixed strong and weak references
When adding a undefined symbols to the symbol table, if the existing
reference is weak replace the symbol flags with (potentially) non-weak
binding.

Fixes: https://github.com/llvm/llvm-project/issues/60829

Differential Revision: https://reviews.llvm.org/D144747
2023-02-27 14:20:01 -08:00
Scott Linder
45ee0a9afc [LLD] Add --lto-CGO[0-3] option
Allow controlling the CodeGenOpt::Level independent of the LTO
optimization level in LLD via new options for the COFF, ELF, MachO, and
wasm frontends to lld. Most are spelled as --lto-CGO[0-3], but COFF is
spelled as -opt:lldltocgo=[0-3].

See D57422 for discussion surrounding the issue of how to set the CG opt
level. The ultimate goal is to let each function control its CG opt
level, but until then the current default means it is impossible to
specify a CG opt level lower than 2 while using LTO. This option gives
the user a means to control it for as long as it is not handled on a
per-function basis.

Reviewed By: MaskRay, #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D141970
2023-02-15 17:34:35 +00:00
Sam Clegg
9369b7d307 [lld][WebAssembly] Limit size of shared 64-bit memories of 2^^34
This is current limit in v8.  See
https://github.com/WebAssembly/memory64/issues/33 how we might change
this in the future.

Differential Revision: https://reviews.llvm.org/D143783
2023-02-13 12:21:32 -08:00
Archibald Elliott
d768bf994f [NFC][TargetParser] Replace uses of llvm/Support/Host.h
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
2023-02-10 09:59:46 +00:00
Archibald Elliott
62c7f035b4 [NFC][TargetParser] Remove llvm/ADT/Triple.h
I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.
2023-02-07 12:39:46 +00:00
Nico Weber
a2ae9e3a24 [lld] fix comment typos to cycle bots 2023-02-04 15:23:30 -05:00
Sam Clegg
a0495e6b00 [lld][WebAssembly] Apply relocations to TLS data
This is only needed in the case of dynamic linking and pthreads.
Previously these relocations were simply not being applied.

Verified that this works using a more real world emscripten test:
https://github.com/emscripten-core/emscripten/pull/18641

Differential Revision: https://reviews.llvm.org/D143020
2023-01-31 15:50:44 -08:00