11207 Commits

Author SHA1 Message Date
Michael J. Spencer
b842725c1d [ELF] Add profile guided section layout
This adds profile guided layout using the Call-Chain Clustering (C³) heuristic
from https://research.fb.com/wp-content/uploads/2017/01/cgo2017-hfsort-final1.pdf .

RFC: [llvm-dev] [RFC] Profile guided section layout
     http://lists.llvm.org/pipermail/llvm-dev/2017-June/114178.html

Pass `--call-graph-ordering-file <file>` to read a call graph profile where each
line has the format:

    <from symbol> <to symbol> <call count>

Differential Revision: https://reviews.llvm.org/D36351

llvm-svn: 330234
2018-04-17 23:30:05 +00:00
Peter Collingbourne
66f1c9a858 Reland r330223, "COFF: Merge .idata, .didat and .edata into .rdata by default.", which was reverted in r330228.
In this reland I removed an unnecessary use of /debug in the test
delayimports32.test and used the /pdbaltpath flag in the test
pdb-publics-import.test, both of which avoid embedding absolute PDB
paths in executables which could affect later RVAs.

Original commit message:
> COFF: Merge .idata, .didat and .edata into .rdata by default.
>
> This saves a little space and matches what link.exe does.
>
> Tested using the chromium Windows trybots:
> https://chromium-review.googlesource.com/c/chromium/src/+/1014784

Differential Revision: https://reviews.llvm.org/D45737

llvm-svn: 330233
2018-04-17 23:28:52 +00:00
Peter Collingbourne
94aa62e48a COFF: Implement /pdbaltpath flag.
I needed to revert r330223 because we were embedding an absolute PDB
path in the .rdata section, which ended up being laid out before the
.idata section and affecting its RVAs. This flag will let us control
the embedded path.

Differential Revision: https://reviews.llvm.org/D45747

llvm-svn: 330232
2018-04-17 23:28:38 +00:00
Peter Collingbourne
1254f3e77c Revert r330223, "COFF: Merge .idata, .didat and .edata into .rdata by default."
Seems to have uncovered some sort of non-determinism on the bots.

llvm-svn: 330228
2018-04-17 22:16:39 +00:00
Peter Collingbourne
09e0e2e656 COFF: Merge .idata, .didat and .edata into .rdata by default.
This saves a little space and matches what link.exe does.

Tested using the chromium Windows trybots:
https://chromium-review.googlesource.com/c/chromium/src/+/1014784

Differential Revision: https://reviews.llvm.org/D45737

llvm-svn: 330223
2018-04-17 21:44:31 +00:00
Peter Collingbourne
7c26663f58 llvm-pdbutil: Fix an off-by-one error.
Differential Revision: https://reviews.llvm.org/D45740

llvm-svn: 330222
2018-04-17 21:44:17 +00:00
Zachary Turner
bee6c22414 [llvm-pdbutil] Dump first section contribution for each module.
The DBI stream contains a list of module descriptors.  At the
beginning of each descriptor is a structure representing the first
section contribution in the output file for that module.  LLD
currently doesn't fill out this structure at all, but link.exe
does.  So as a precursor to emitting this data in LLD, we first
need a way to dump it so that it can be checked.

This patch adds support for the dumping, and verifies via a test
that LLD emits bogus information.

llvm-svn: 330208
2018-04-17 20:06:43 +00:00
Peter Collingbourne
2f6d00612d COFF: Make SectionChunk::Relocs field an ArrayRef. NFCI.
Differential Revision: https://reviews.llvm.org/D45714

llvm-svn: 330172
2018-04-17 01:54:34 +00:00
Reid Kleckner
9c86a31307 Revert r329960 "Do not keep shared symbols created from garbage-collected eliminated DSOs."
This is causing large numbers of Chromium test executables to crash on
shutdown. The relevant symbol seems to be __cxa_finalize, which gets
removed from the dynamic symbol table for some of the support libraries.

llvm-svn: 330164
2018-04-16 22:45:23 +00:00
Zachary Turner
d8d97de514 [PDB] Correctly use the target machine when writing DBI stream.
Using Config->is64() will treat ARM64 as Amd64, which is incorrect.
Furthermore, there are more esoteric architectures that could
theoretically be encountered.  Just set it directly to the machine
type, which we already know anyway.

llvm-svn: 330157
2018-04-16 20:42:06 +00:00
Zachary Turner
e3fe669855 Resubmit "Fix some incorrect fields in our generated PDBs."
This fixes the failing tests.  They simply hadn't been updated
to match the new output resulting from this patch.

llvm-svn: 330145
2018-04-16 18:17:13 +00:00
Zachary Turner
52c80e3860 Revert "Fix some incorrect fields in our generated PDBs."
There are a couple of failing tests which slipped under my radar
so I'm reverting this while I attempt to fix.

llvm-svn: 330133
2018-04-16 16:55:41 +00:00
Zachary Turner
1b06cc7817 Fix some incorrect fields in our generated PDBs.
Most of these are pretty trivial and obvious. Setting the toolchain
version to 14.11 is perhaps a little questionable, but we've been bitten
in the past where one of our version fields sidn't match MSVC's, and I
definitely don't want to go through that diagnosis again as it was
pretty time consuming and hard to track down.

I found all of these by using llvm-pdbutil export to dump the dbi and
pdb streams to a file, then using fc followed by llvm-pdbutil explain to
explain the mismatched bytes.

There are still some more, these are just the low hanging fruit.

Differential Revision: https://reviews.llvm.org/D45276

llvm-svn: 330130
2018-04-16 16:27:49 +00:00
Rui Ueyama
197194b6c9 Define InitLLVM to do common initialization all at once.
We have a few functions that virtually all command wants to run on
process startup/shutdown. This patch adds InitLLVM class to do that
all at once, so that we don't need to copy-n-paste boilerplate code
to each llvm command's main() function.

Differential Revision: https://reviews.llvm.org/D45602

llvm-svn: 330046
2018-04-13 18:26:06 +00:00
Rafael Espindola
ea2c78369c Reduce code duplication.
getVA was already implemented in the base class.

llvm-svn: 330036
2018-04-13 16:07:27 +00:00
Simon Atanasyan
1ba194212a [ELF][MIPS] Support linking of PIE for MIPS
MIPS ABI requires creation of the MIPS_RLD_MAP dynamic tag for non-PIE
executables only and MIPS_RLD_MAP_REL tag for both PIE and non-PIE
executables. The patch skips definition of the MIPS_RLD_MAP for PIE
files and defines MIPS_RLD_MAP_REL.

The MIPS_RLD_MAP_REL tag stores the offset to the .rld_map section
relative to the address of the tag itself.

Differential Revision: https://reviews.llvm.org/D43347

llvm-svn: 329996
2018-04-13 08:15:01 +00:00
Rui Ueyama
039d248778 Do not keep shared symbols created from garbage-collected eliminated DSOs.
If all references to a DSO happen to be weak, and if the DSO is
specified with --as-needed, the DSO is not added to DT_NEEDED.
If that happens, we also need to eliminate shared symbols created
from the DSO. Otherwise, they become dangling references that point
to non-exsitent DSO.

Fixes https://bugs.llvm.org/show_bug.cgi?id=36991

Differential Revision: https://reviews.llvm.org/D45536

llvm-svn: 329960
2018-04-12 21:57:04 +00:00
Sam Clegg
225c4697b4 [WebAssembly] Match llvm change to custom section size
Summary:
The content of custome sections no longer includes the
name itself.

See: https://reviews.llvm.org/D45579

Subscribers: jfb, dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Differential Revision: https://reviews.llvm.org/D45580

llvm-svn: 329948
2018-04-12 20:31:35 +00:00
Sam Clegg
d11b6edfc8 Remove references to old SymbolBody class
Differential Revision: https://reviews.llvm.org/D45400

llvm-svn: 329846
2018-04-11 19:52:53 +00:00
David Blaikie
ad191189f0 Rename to match an LLVM change
llvm-svn: 329841
2018-04-11 18:49:41 +00:00
George Rimar
c552619fc1 [ELF] - Reorder local symbols.
This fixes PR36716 (https://bugs.llvm.org/show_bug.cgi?id=36716),

Patch sorts local symbols to match the
following order: file1, local1, hidden1, file2, local2, hidden2 ...

Differential revision: https://reviews.llvm.org/D45325

llvm-svn: 329787
2018-04-11 09:24:27 +00:00
George Rimar
9f0b8e8025 [ELF] - Simplify. NFC.
llvm-svn: 329785
2018-04-11 09:03:02 +00:00
Rafael Espindola
262589db70 Don't warn on ICFed symbols, warn on synthetic ones.
Based on a patch for the ICF warning by Rui.

llvm-svn: 329757
2018-04-10 21:13:52 +00:00
Sam Clegg
80ba43872e [WebAssembly] Add support for custom sections
Copy user-defined custom sections into the output, concatenating
sections with the same name.

Differential Revision: https://reviews.llvm.org/D45340

llvm-svn: 329717
2018-04-10 16:12:49 +00:00
Nico Weber
fb64730005 s/LLVM_ON_WIN32/_WIN32/, lld
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in             
HandleLLVMOptions.cmake, which is where _WIN32 defined too.  Just use the        
default macro instead of a reinvented one.                                       
                                                                                 
See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.   
No intended behavior change. 

llvm-svn: 329696
2018-04-10 13:15:21 +00:00
George Rimar
e160473823 [ELF] - Do not crash when trying to order --defsym/linker script symbols.
Currently, we crash because File is null for
such symbols.

Differential revision: https://reviews.llvm.org/D45440

llvm-svn: 329678
2018-04-10 09:44:44 +00:00
Rui Ueyama
e4ba9044ad Add a test for nested --start-group.
llvm-svn: 329642
2018-04-09 23:30:21 +00:00
Rui Ueyama
1d92aa7380 Add --warn-backrefs to maintain compatibility with other linkers
I'm proposing a new command line flag, --warn-backrefs in this patch.
The flag and the feature proposed below don't exist in GNU linkers
nor the current lld.

--warn-backrefs is an option to detect reverse or cyclic dependencies
between static archives, and it can be used to keep your program
compatible with GNU linkers after you switch to lld. I'll explain the
feature and why you may find it useful below.

lld's symbol resolution semantics is more relaxed than traditional
Unix linkers. Therefore,

  ld.lld foo.a bar.o

succeeds even if bar.o contains an undefined symbol that have to be
resolved by some object file in foo.a. Traditional Unix linkers
don't allow this kind of backward reference, as they visit each
file only once from left to right in the command line while
resolving all undefined symbol at the moment of visiting.

In the above case, since there's no undefined symbol when a linker
visits foo.a, no files are pulled out from foo.a, and because the
linker forgets about foo.a after visiting, it can't resolve
undefined symbols that could have been resolved otherwise.

That lld accepts more relaxed form means (besides it makes more
sense) that you can accidentally write a command line or a build
file that works only with lld, even if you have a plan to
distribute it to wider users who may be using GNU linkers.  With
--check-library-dependency, you can detect a library order that
doesn't work with other Unix linkers.

The option is also useful to detect cyclic dependencies between
static archives. Again, lld accepts

  ld.lld foo.a bar.a

even if foo.a and bar.a depend on each other. With --warn-backrefs
it is handled as an error.

Here is how the option works. We assign a group ID to each file. A
file with a smaller group ID can pull out object files from an
archive file with an equal or greater group ID. Otherwise, it is a
reverse dependency and an error.

A file outside --{start,end}-group gets a fresh ID when
instantiated. All files within the same --{start,end}-group get the
same group ID. E.g.

  ld.lld A B --start-group C D --end-group E

A and B form group 0, C, D and their member object files form group
1, and E forms group 2. I think that you can see how this group
assignment rule simulates the traditional linker's semantics.

Differential Revision: https://reviews.llvm.org/D45195

llvm-svn: 329636
2018-04-09 23:05:48 +00:00
Peter Collingbourne
bdfa127c3e COFF: Remove dead code. NFCI.
Differential Revision: https://reviews.llvm.org/D45423

llvm-svn: 329609
2018-04-09 19:46:00 +00:00
Rumeet Dhindsa
682a417e51 Added support for LTO options: sample_profile, new_pass_manager and debug_pass_manager
Differential Revision: https://reviews.llvm.org/D45275

llvm-svn: 329598
2018-04-09 17:56:07 +00:00
George Rimar
aeb7e305b8 [ELF] - Simplify test case. NFC.
llvm-svn: 329563
2018-04-09 13:13:36 +00:00
George Rimar
0029f21482 [ELF] - Stop setting OutSecOff too early.
Currently LLD sets OutSecOff in addSection for input sections.
That is a fake offset (just a rude approximation to remember the order), 
used for sorting SHF_LINK_ORDER sections
(see resolveShfLinkOrder, compareByFilePosition).

There are 2 problems with such approach:

1. We currently change and reuse Size field as a value assigned. Changing size is
not good because leads to bugs. Currently, SIZEOF(.bss) for empty .bss returns 2
because we add two empty synthetic sections and increase size twice by 1. 
(See PR37011: https://bugs.llvm.org/show_bug.cgi?id=37011)

2. Such approach simply does not work when --symbol-ordering-file is involved,
because processing of the ordering file might break the initial section order.

This fixes PR37011.

Differential revision: https://reviews.llvm.org/D45368

llvm-svn: 329560
2018-04-09 13:01:50 +00:00
George Rimar
9bc20fff4a [ELF] - Fix cref.s test case.
The intention of -gc-sections flag was to check
that discarded is not in the output. It should be
specified in the executable command line invocation
and also, the symbol must be global as local symbols
are anyways not printed.

Differential revision: https://reviews.llvm.org/D45159

llvm-svn: 329559
2018-04-09 12:45:29 +00:00
George Rimar
7bf92be676 [ELF] - Allow LLD to produce file symbols.
This is for PR36716 and
this enables emitting STT_FILE symbols.

Output size affect is minor:
lld binary size changes from 52,883,408 to 52,949,400
clang binary size changes from 83,136,456 to 83,219,600

Differential revision: https://reviews.llvm.org/D45261

llvm-svn: 329557
2018-04-09 11:43:52 +00:00
Aaron Smith
c3b5515e51 [tests] Fix format-binary-non-ascii.s to work with Python 3 on Windows
Some platforms interpret the pound sign as one character. Platforms that use 
Python 2.x actually interpret it as two characters because in the Python 2.x 
version of lit, the string used for the file name is a byte string and the pound 
sign is two bytes.

Patch by Stella Stamenova!

llvm-svn: 329472
2018-04-07 00:55:26 +00:00
Peter Collingbourne
4902508934 COFF: Process /merge flag as we create output sections.
With this we can merge builtin sections.

Differential Revision: https://reviews.llvm.org/D45350

llvm-svn: 329471
2018-04-07 00:46:55 +00:00
Rafael Espindola
b4d3dfefb7 Avoid some temporary allocations.
Some system libraries have a lot of versioned symbols. When linking
scylla this brings the number of malloc calls from 49154 to 37944.

llvm-svn: 329453
2018-04-06 20:53:06 +00:00
Benjamin Kramer
d726d04f45 [ELF] Don't write to the source directory in test.
llvm-svn: 329384
2018-04-06 10:05:47 +00:00
Rafael Espindola
3bd98b1031 Change which file we print when a symbol cannot be ordered.
Currently there are a few odd things about the warning about symbols
that cannot be ordered. This patch fixes:

* When there is an undefined symbol that resolves to a shared file, we
  were printing the location of the undefined reference.

* If there are multiple comdats, we were reporting them all.

llvm-svn: 329371
2018-04-06 03:36:19 +00:00
Peter Collingbourne
f2c0f39b91 COFF: Create output sections early. NFCI.
With this, all output sections are created in one place. This will make
it simpler to implement merging of builtin sections.

Differential Revision: https://reviews.llvm.org/D45349

llvm-svn: 329370
2018-04-06 03:25:49 +00:00
Rafael Espindola
9b6a65b144 Don't ignore addend when a SHF_MERGE section is dead.
This is similar to r329219, but for the entire section. Like r329219 I
don't expect this to have any real impact, it is just more consistent
and simpler.

llvm-svn: 329367
2018-04-06 01:10:33 +00:00
Rui Ueyama
96ba8befa9 [WebAssembly] Remove StackPointerGlobal member variable from the driver.
Since InputGlobal makes a copy of a given object, we can use a temporary
object allocated on the stack here.

Differential Revision: https://reviews.llvm.org/D43924

llvm-svn: 329337
2018-04-05 19:37:48 +00:00
Rui Ueyama
ac95bb1d52 [WebAssembly] Remove another trivial accessor.
Differential Revision: https://reviews.llvm.org/D43725

llvm-svn: 329336
2018-04-05 19:37:31 +00:00
Fangrui Song
c43eff6208 [ELF] Add CHECK to test/ELF/undef-start.s
llvm-svn: 329333
2018-04-05 19:20:33 +00:00
Rui Ueyama
4e150c4c88 Make "Size" column in the map file one characters shorter.
Previously, "size" column is 9 characters long which is too long
at least for 32-bit (because at maximum it needs 8 columns). This
patch make it one column shorter than before. That's also a reasonable
default for 64-bit.

llvm-svn: 329317
2018-04-05 17:20:18 +00:00
Rui Ueyama
16a130bc78 Fix column size in the map file.
Size can be narrow, but LMA should be the same width as VMA.

llvm-svn: 329312
2018-04-05 16:45:37 +00:00
Rafael Espindola
5f8e77afb5 Initialize OutputOff to zero.
We have a dedicated Live bit, so we don't need a special value and we
were not accounting for in at least one place.

llvm-svn: 329307
2018-04-05 15:56:04 +00:00
Eugene Leviant
57225ac179 [ELF] Don't add NOLOAD sections to segment
Differential revision: https://reviews.llvm.org/D45264

llvm-svn: 329281
2018-04-05 13:23:59 +00:00
George Rimar
f9936e1fc9 [ELF] - Eliminate Target::isPicRel method.
As was mentioned in comments for D45158,
isPicRel's name does not make much sense,
because what this method does is checks if
we need to create the dynamic relocation or not.

Instead of renaming it to something different,
we can 'isPicRel' completely.

We can reuse the getDynRel method.
They are logically very close, getDynRel can just return
R_*_NONE in case no dynamic relocation should be produced
and that would simplify things and avoid functionality
correlation/duplication with 'isPicRel'.

The patch does this change.

Differential revision: https://reviews.llvm.org/D45248

llvm-svn: 329275
2018-04-05 12:07:20 +00:00
George Rimar
e88b76a989 [ELF] - Reveal more information in -Map file about assignments.
Currently, LLD print symbol assignment commands to the map file,
but it does not do that for assignments that are outside of the section
descriptions. Such assignments can affect the layout though.

The patch implements the following:

* Teaches LLD to print symbol assignments outside of section declaration.
* Teaches LLD to print PROVIDE/HIDDEN/PROVIDE hidden commands.

In case when symbol is not provided, nothing will be printed.

Differential revision: https://reviews.llvm.org/D44894

llvm-svn: 329272
2018-04-05 11:25:58 +00:00