1269 Commits

Author SHA1 Message Date
George Rimar
777f96304e Recommit of r263252, [ELF] - Change all messages to lowercase to be consistent.
which was reverted because included
unrelative changes by mistake.

Original commit message:

[ELF] - Change all messages to lowercase to be consistent.

That is directly opposite to http://reviews.llvm.org/D18045,
which was reverted.

This patch changes all messages to start from lowercase letter if
they were not before.

That is done to be consistent with clang.

Differential revision: http://reviews.llvm.org/D18085

llvm-svn: 263337
2016-03-12 08:31:34 +00:00
Rui Ueyama
634ddf0bec ELF: Implement --build-id.
This patch implements --build-id. After the linker creates an output file
in the memory buffer, it computes the FNV1 hash of the resulting file
and set the hash to the .note section as a build-id.

GNU ld and gold have the same feature, but their default choice of the
hash function is different. Their default is SHA1.

We made a deliberate choice to not use a secure hash function for the
sake of performance. Computing a secure hash is slow -- for example,
MD5 throughput is usually 400 MB/s or so. SHA1 is slower than that.

As a result, if you pass --build-id to gold, then the linker becomes about
10% slower than that without the option. We observed a similar degradation
in an experimental implementation of build-id for LLD. On the other hand,
we observed only 1-2% performance degradation with the FNV hash.

Since build-id is not for digital certificate or anything, we think that
a very small probability of collision is acceptable.

We considered using other signals such as using input file timestamps as
inputs to a secure hash function. But such signals would have an issue
with build reproducibility (if you build a binary from the same source
tree using the same toolchain, the build id should become the same.)

GNU linkers accepts --build-id=<style> option where style is one of
"MD5", "SHA1", or an arbitrary hex string. That option is out of scope
of this patch.

http://reviews.llvm.org/D18091

llvm-svn: 263292
2016-03-11 20:51:53 +00:00
Rui Ueyama
f714955402 Revert r263252: "[ELF] - Change all messages to lowercase to be consistent."
This reverts commit r263252 because the change contained unrelated changes.

llvm-svn: 263272
2016-03-11 18:46:51 +00:00
George Rimar
96bcdae1a5 [ELF] - Change all messages to lowercase to be consistent.
That is directly opposite to http://reviews.llvm.org/D18045,
which was reverted.

This patch changes all messages to start from lowercase letter if
they were not before.

That is done to be consistent with clang.

Differential revision: http://reviews.llvm.org/D18085

llvm-svn: 263252
2016-03-11 16:40:55 +00:00
George Rimar
5761042db7 This reverts the r263125
It was discussed to make all messages be 
lowercase to be consistent with clang.
(also reverts the r263128 which fixed 
build bot fail after r263125)

Original commit message:
[ELF] - Consistent spelling for error/warning messages

Previously error and warnings were not consistent in lld.
Some of them started from lowercase letter, others from
uppercase. Also there was one or two which had a dot at the end.
This patch changes all messages to start from uppercase letter if
they were not before.

Differential revision: http://reviews.llvm.org/D18045

llvm-svn: 263240
2016-03-11 14:43:02 +00:00
George Rimar
56e0d53e92 [ELF] - Move initSymbols() to Driver.cpp. NFC.
That is followup for http://reviews.llvm.org/D18047
patch. initSymbols() moved to Driver.cpp and made static.

llvm-svn: 263214
2016-03-11 10:07:18 +00:00
Rui Ueyama
e98095026f ELF: Add --thread option and partially parallelize writeTo().
This patch adds --thread option and use parallel_for_each to write
sections in regular OutputSections.

This is the first patch to use more than one threads.

Note that --thread is off by default because it is experimental.
At this moment I still want to focus on single thread performance
because multi-threading is not a magic wand to fix performance
problems after all. It is generally very hard to make a slow program
faster by threads. Therefore, I want to make the linker as efficient
as possible first and then look for opportunity to make it even faster
using more than one core.

Here are some numbers to link programs with and without --threads
and using GNU gold. Numbers are in seconds.

Clang
  w/o --threads 0.697
  w --threads   0.528
  gold          1.643

Scylla
  w/o --threads 5.032
  w --threads   4.935
  gold          6.791

GNU gold
  w/o --threads 0.550
  w --threads   0.551
  gold          0.737

I limited the number of cores these processes can use to 4 using
perf command, so although my machine has 20 physical cores, the
performance gain I observed should be reproducible with a machine
which is not as beefy as mine.

llvm-svn: 263190
2016-03-11 04:23:12 +00:00
George Rimar
5a3dcf4edf [ELF] - Do not call doInitSymbols for all ELFTs
It looks a bit wierd that we have to initialize symbols for all ELFT 
types when we use only one ELFT for link. We can only init those
that we need. Patch fixes it.

Differential revision: http://reviews.llvm.org/D18047

llvm-svn: 263133
2016-03-10 17:38:49 +00:00
George Rimar
e094388861 [ELF] - Consistent spelling for error/warning messages
Previously error and warnings were not consistent in lld.
Some of them started from lowercase letter, others from
uppercase. Also there was one or two which had a dot at the end.
This patch changes all messages to start from uppercase letter if
they were not before.

Differential revision: http://reviews.llvm.org/D18045

llvm-svn: 263125
2016-03-10 16:58:34 +00:00
Sean Silva
03e41ee6a7 [lto] Initialize asmparsers.
Summary:
They are needed for inline asm during LTO.

In particular we hit the report_fatal_error on
llvm/lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp:138

    LLVM ERROR: Inline asm not supported by this streamer because we don't have an asm parser for this target

Reviewers: ruiu, rafael

Subscribers: Bigcheese, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18027

llvm-svn: 263094
2016-03-10 04:58:52 +00:00
Rui Ueyama
2b4df612ec Simplify. NFC.
In this context, MBRef.getBufferIdentifier() == Path.

llvm-svn: 263061
2016-03-09 21:15:17 +00:00
Sean Silva
35ef3d95eb [lto] Add beginning of -save-temps option.
Summary:
This is useful for debugging issues with LTO.
The option follows the analogous option in ld64 and the gold plugin (per
Rafael's suggestion).

For starters, this only dumps the combined bitcode file.
In a future patch I will add dumping for the .o file.

The naming of the output follows ld64's convention which is slightly more
consistent IMO (consistent `.lto.<extension>` for all the files).

Reviewers: rafael, ruiu

Subscribers: joker.eph, Bigcheese, llvm-commits

Differential Revision: http://reviews.llvm.org/D18006

llvm-svn: 263055
2016-03-09 20:01:08 +00:00
Rafael Espindola
0650192094 Revert "ELF: Add /lib and /usr/lib as default search paths."
It was causing errors like

/lib/libc.so.6 is incompatible with elf_x86_64

when linking on Fedora.

Every system has different default paths. It seems better to just trust
the driver to pass the correct -L options.

This reverts commit 262910.

llvm-svn: 262941
2016-03-08 17:13:12 +00:00
Rui Ueyama
3f9f09265b ELF: Rename NoInhibitExec -> NoinhibitExec.
The variables corresponding to command line options are named mechanically.
Because the option for the variable is -noinhibit-exec and not -no-inhibit-exec,
it should be name this way.

llvm-svn: 262911
2016-03-08 04:06:29 +00:00
Rui Ueyama
b9883d55d8 ELF: Add /lib and /usr/lib as default search paths.
GNU ld and gold have these paths as default search paths.
If you don't want these directories, pass -nostdlib.

llvm-svn: 262910
2016-03-08 04:06:27 +00:00
Rafael Espindola
75714f618c Rename 'fatal' to 'check' when it doesn't always fail.
llvm-svn: 262666
2016-03-03 22:24:39 +00:00
Rafael Espindola
1130935c4a Simplify error handling.
This makes fatal return T when there is no error. This avoids the need
for quite a few temporaries.

llvm-svn: 262626
2016-03-03 16:21:44 +00:00
Rui Ueyama
a453c0a5ad Merge DarwinLdDriver and Driver.
Now that DarwinLdDriver is the only derived class of Driver.
This patch merges them and actually removed the class because
they can now just be non-member functions. This change simplifies
a common header, Driver.h.

http://reviews.llvm.org/D17788

llvm-svn: 262502
2016-03-02 19:08:05 +00:00
George Rimar
262b927d36 [ELF] - handle of command line options incompatible with -r
Patch handles the command line options
incompatible with relocatable output.

Differential revision: http://reviews.llvm.org/D17758

llvm-svn: 262377
2016-03-01 19:38:51 +00:00
Rui Ueyama
7c82dbf333 Remove unused #include.
llvm-svn: 262169
2016-02-28 03:27:09 +00:00
Rui Ueyama
1eb9f44118 ELF: Add --help option.
llvm-svn: 262168
2016-02-28 03:18:09 +00:00
Rui Ueyama
1abcf370fd ELF: Add --version option.
llvm-svn: 262167
2016-02-28 03:18:07 +00:00
Rafael Espindola
e0df00b91f Rename elf2 to elf.
llvm-svn: 262159
2016-02-28 00:25:54 +00:00
Rui Ueyama
10bd283041 ELF: Define log() to print out message if --verbose is given.
llvm-svn: 261919
2016-02-25 18:56:01 +00:00
Rui Ueyama
0b28952993 ELF: Implement ICF.
This patch implements the same algorithm as LLD/COFF's ICF. I'm
not going to repeat the same description about how it works, so you
want to read the comment in ICF.cpp in this patch if you want to know
the details. This algorithm should be more powerful than the ICF
algorithm implemented in GNU gold. It can even merge mutually-recursive
functions (which is harder than one might think).

ICF is a fairly effective size optimization. Here are some examples.

 LLD:   37.14 MB -> 35.80 MB (-3.6%)
 Clang: 59.41 MB -> 57.80 MB (-2.7%)

The lacking feature is "safe" version of ICF. This merges all
identical sections. That is not compatible with a C/C++ language
requirement that two distinct functions must have distinct addresses.

But as long as your program do not rely on the pointer equality
(which is in many cases true), your program should work with the
feature. LLD works fine for example.

GNU gold implements so-called "safe ICF" that identifies functions
that are safe to merge by heuristics -- for example, gold thinks
that constructors are safe to merge because there is no way to
take an address of a constructor in C++. We have a different idea
which David Majnemer suggested that we add NOPs at beginning of
merged functions so that two or more pointers can have distinct
values. We can do whichever we want, but this patch does not
include neither.

http://reviews.llvm.org/D17529

llvm-svn: 261912
2016-02-25 18:43:51 +00:00
George Rimar
58941ee12a [ELF2] - Basic implementation of -r/--relocatable
-r, -relocatable - Generate relocatable output

Currently does not have support for files containing 
relocation sections with entries that refer to local 
symbols (like rel[a].eh_frame which refer to sections
and not to symbols)

Differential revision: http://reviews.llvm.org/D14382

llvm-svn: 261838
2016-02-25 08:23:37 +00:00
Rui Ueyama
c89bff2ce1 Handle bitcode files in archive files with --whole-archive.
This patch moves BitcodeFile instantiation into createObjectFile.
Previously, we handle bitcode files outside that function and did
not handle for --whole-archive.

http://reviews.llvm.org/D17527

llvm-svn: 261663
2016-02-23 18:17:11 +00:00
Rafael Espindola
9f77ef0c08 Add initial LTO support.
llvm-svn: 260726
2016-02-12 20:54:57 +00:00
Rui Ueyama
e9c5806593 ELF: Remove use of MapVector from LinkerScript.
We don't have to use a MapVector here. Instead, just std::vector suffices.

llvm-svn: 260724
2016-02-12 20:41:43 +00:00
Rui Ueyama
f9de0d6904 ELF: Remove readLinkerScript and define LinkerScript::read instead.
llvm-svn: 260598
2016-02-11 21:38:55 +00:00
Rui Ueyama
c1c282a04e ELF: Release Target at end of link().
Previously, Target held a value until a new value is assigned to the
variable. That was a benign leak -- that was not an unbounded leak
and didn't grab any resources except a small amount of memory. But
it is better to fix than leaving as is.

llvm-svn: 260592
2016-02-11 21:18:01 +00:00
Rui Ueyama
717677af35 ELF: Create LinkerScript class to move code out of Writer.
Previously, we had code for linker scripts in Writer. This patch
separates that as LinkerScript class. The class provides a few
functions to query linker scripts and is also a container of some
linker-script-specific information.

Hopefully, Writer will only implement the default behavior and let
the new class handle gotchas regarding linker scripts.

llvm-svn: 260591
2016-02-11 21:17:59 +00:00
Simon Atanasyan
b76108a63a [ELF][MIPS] Add comments to describe __gnu_local_gp magic symbol
NFC. Follow-up to r259781.

llvm-svn: 260027
2016-02-07 12:09:40 +00:00
Simon Atanasyan
597df21eb2 [ELF][MIPS] Add handling for __gnu_local_gp symbol
This symbol is a "fake" symbol like "_gp_disp" and denotes
the GOT + 0x7FF0 value.

llvm-svn: 259781
2016-02-04 12:09:49 +00:00
Rui Ueyama
b6940115a8 ELF: Make link() to take an output stream to which error messages are written.
http://reviews.llvm.org/D16668

llvm-svn: 259597
2016-02-02 22:49:32 +00:00
Rui Ueyama
21eecb4f14 Re-submit ELF: Report multiple errors from the driver.
This reverts r259395 which reverted r259143.

llvm-svn: 259572
2016-02-02 21:13:09 +00:00
George Rimar
5c36e5938d [ELF] Implemented -Bsymbolic-functions command line option
-Bsymbolic-functions: 
When creating a shared library, bind references to global 
function symbols to the definition within the shared library, if any.

This patch also fixed behavior of already existent -Bsymbolic:
previously PLT entries were created even if -Bsymbolic was specified.

Differential revision: http://reviews.llvm.org/D16411

llvm-svn: 259481
2016-02-02 09:28:53 +00:00
Rui Ueyama
71c066d8cf ELF: Include archive names in error messages.
If object files are drawn from archive files, the error message should
be something like "conflict symbols in foo.a(bar.o) and baz.o" instead
of "conflict symbols in bar.o and baz.o". This patch implements that.

llvm-svn: 259475
2016-02-02 08:22:41 +00:00
Nico Weber
f07bd3b72d Revert r259143, it broke check-lld on Windows (see PR26388).
llvm-svn: 259395
2016-02-01 20:03:53 +00:00
Rui Ueyama
16ba669c87 ELF: Report duplicate symbols as many as possible instead of the first one.
http://reviews.llvm.org/D16647

llvm-svn: 259233
2016-01-29 19:41:13 +00:00
George Rimar
2c9b7bd326 Replace StringRef with std::string in LinkerDriver::addLibrary(StringRef Name) because std::string is destroyed after assigning what fills the buffer with garbage and StringRef is no more valid.
That fixes few failing tests under windows+msvs2015(debug).

llvm-svn: 259186
2016-01-29 10:52:52 +00:00
Rui Ueyama
1940424632 ELF: Report multiple errors from the driver.
This patch let the driver keep going until it parses all
command line options.

http://reviews.llvm.org/D16645

llvm-svn: 259143
2016-01-29 01:54:52 +00:00
Rui Ueyama
64cfffd333 ELF: Rename error -> fatal and redefine error as a non-noreturn function.
In many situations, we don't want to exit at the first error even in the
process model. For example, it is better to report all undefined symbols
rather than reporting the first one that the linker picked up randomly.

In order to handle such errors, we don't need to wrap everything with
ErrorOr (thanks for David Blaikie for pointing this out!) Instead, we
can set a flag to record the fact that we found an error and keep it
going until it reaches a reasonable checkpoint.

This idea should be applicable to other places. For example, we can
ignore broken relocations and check for errors after visiting all relocs.

In this patch, I rename error to fatal, and introduce another version of
error which doesn't call exit. That function instead sets HasError to true.
Once HasError becomes true, it stays true, so that we know that there
was an error if it is true.

I think introducing a non-noreturn error reporting function is by itself
a good idea, and it looks to me that this also provides a gradual path
towards lld-as-a-library (or at least embed-lld-to-your-program) without
sacrificing code readability with lots of ErrorOr's.

http://reviews.llvm.org/D16641

llvm-svn: 259069
2016-01-28 18:40:06 +00:00
Rafael Espindola
3a6a0a0109 Delete addIgnoredStrong.
It is not needed now that we resolve symbols is shared libraries
correctly.

llvm-svn: 258104
2016-01-19 00:05:54 +00:00
George Rimar
f6bc65a3b2 Reapply r257753 with fix:
Added check for terminator CIE/FDE which has zero data size.
void EHOutputSection<ELFT>::addSectionAux(
...
 // If CIE/FDE data length is zero then Length is 4, this
 // shall be considered a terminator and processing shall end.
    if (Length == 4)
      break;
...

After this "Bug 25923 - lld/ELF2 linked application crashes if exceptions were used." is fixed for me. Self link of clang also works.

Initial commit message:
[ELF] - implemented --eh-frame-hdr command line option.

--eh-frame-hdr
Request creation of ".eh_frame_hdr" section and ELF "PT_GNU_EH_FRAME" segment header.

Both gold and the GNU linker support an option --eh-frame-hdr which tell them to construct a header for all the .eh_frame sections. This header is placed in a section named .eh_frame_hdr and also in a PT_GNU_EH_FRAME segment. At runtime the unwinder can find all the PT_GNU_EH_FRAME segments by calling dl_iterate_phdr.
This section contains a lookup table for quick binary search of FDEs.
Detailed info can be found here:
http://www.airs.com/blog/archives/462

Differential revision: http://reviews.llvm.org/D15712

llvm-svn: 257889
2016-01-15 13:34:52 +00:00
Rui Ueyama
5f91ace828 Revert r257753: "[ELF] - implemented --eh-frame-hdr command line option."
This reverts commit r257753 because we cannot link Clang with this patch.

llvm-svn: 257797
2016-01-14 20:32:19 +00:00
George Rimar
28f4fbe480 [ELF] - implemented --eh-frame-hdr command line option.
--eh-frame-hdr
Request creation of ".eh_frame_hdr" section and ELF "PT_GNU_EH_FRAME" segment header.

Both gold and the GNU linker support an option --eh-frame-hdr which tell them to construct a header for all the .eh_frame sections. This header is placed in a section named .eh_frame_hdr and also in a PT_GNU_EH_FRAME segment. At runtime the unwinder can find all the PT_GNU_EH_FRAME segments by calling dl_iterate_phdr.
This section contains a lookup table for quick binary search of FDEs.
Detailed info can be found here:
http://www.airs.com/blog/archives/462

Differential revision: http://reviews.llvm.org/D15712

llvm-svn: 257753
2016-01-14 10:30:32 +00:00
Rui Ueyama
a4a628fb51 Demangle symbols when including them in error messages.
llvm-svn: 257647
2016-01-13 18:55:39 +00:00
Simon Atanasyan
188558e5eb [ELF][MIPS] Prevent substitution of _gp_disp symbol
On MIPS O32 ABI, _gp_disp is a magic symbol designates offset between
start of function and gp pointer into GOT. To make seal with such symbol
we add new method addIgnoredStrong(). It adds ignored symbol with global
binding to prevent the symbol substitution. The addIgnored call is not
enough here because this call adds a weak symbol which might be
substituted by symbol from shared library.

Differential Revision: http://reviews.llvm.org/D16084

llvm-svn: 257449
2016-01-12 06:23:57 +00:00
Davide Italiano
15414cd0a7 [ELF] FreeBSD has slightly different emulation strings for PPC.
Teach lld about them.

llvm-svn: 257437
2016-01-12 02:58:59 +00:00