908 Commits

Author SHA1 Message Date
Bob Haarman
fbc229dc8f added missing files for r293965
llvm-svn: 293966
2017-02-02 23:53:10 +00:00
Bob Haarman
6c8f736ba7 COFF: add error() and warn() to Error.{cpp,h}
Summary: This copies over some functionality we have in ELF/Error.{cpp,h} and makes it available in COFF/Error.{cpp,h}

Reviewers: pcc, rafael, ruiu

Subscribers:

Differential Revision: https://reviews.llvm.org/D28692

llvm-svn: 292240
2017-01-17 19:07:42 +00:00
Rui Ueyama
3e6490399e Define sys::path::convert_to_slash
This patch moves convertToUnixPathSeparator from LLD to LLVM.

Differential Revision: https://reviews.llvm.org/D28444

llvm-svn: 291414
2017-01-09 01:47:15 +00:00
Rafael Espindola
1d1f692158 Revert r291221.
Should bring back the windows bots.

llvm-svn: 291263
2017-01-06 19:01:49 +00:00
Rui Ueyama
b4c63caf76 Rename lld::stringize -> lld::toString.
llvm-svn: 291223
2017-01-06 10:04:35 +00:00
Rui Ueyama
dac6169214 Remove lld::convertToUnixPathSeparator.
Now TarWriter takes care of path separator conversion, so we don't
need to handle that in LLD.

llvm-svn: 291221
2017-01-06 09:38:43 +00:00
Rui Ueyama
7f1f912794 Use TarWriter to create tar archives instead of cpio.
This is how we use TarWriter in LLD. Now LLD does not append
a file extension, so you need to pass `--reproduce foo.tar`
instead of `--reproduce foo`.

Differential Revision: https://reviews.llvm.org/D28103

llvm-svn: 291210
2017-01-06 02:33:53 +00:00
Rui Ueyama
9381eb1045 Remove lld/Support/Memory.h.
I thought for a while about how to remove it, but it looks like we
can just copy the file for now. Of course I'm not happy about that,
but it's just less than 50 lines of code, and we already have
duplicate code in Error.h and some other places. I want to solve
them all at once later.

Differential Revision: https://reviews.llvm.org/D27819

llvm-svn: 290062
2016-12-18 14:06:06 +00:00
Rui Ueyama
520d9169e6 Move Memory.{h,cpp} to lld/Support so that we can use them from COFF.
llvm-svn: 289084
2016-12-08 18:31:13 +00:00
Rui Ueyama
f966fe6c38 Do not pass line number to convertToUnixPathSeparator.
Line number can never contain '/' or '\', so the previous code
was pointless at that point.

llvm-svn: 288973
2016-12-07 20:25:45 +00:00
Rui Ueyama
63d064e9d1 Make convertToUnixPathSeparator return a new string instead of mutating argument.
llvm-svn: 288972
2016-12-07 20:22:27 +00:00
George Rimar
92c88a4f76 [ELF] - Print absolute file name in errors when possible.
Currently LLD prints basename of source file name in error messages,
for example:
$ mkdir foo
$ echo 'void _start(void) { foobar(); }' > foo/bar.c
$ gcc -g -c foo/bar.c
$ bin/ld.lld -o out bar.o 
bin/ld.lld: error: bar.c:1: undefined symbol 'foobar'
$
This should say:
bin/ld.lld: error: foo/bar.c:1: undefined symbol 'foobar'

This is PR31299

Differential revision: https://reviews.llvm.org/D27506

llvm-svn: 288966
2016-12-07 19:42:25 +00:00
Rui Ueyama
ca17841fc4 Run the last iteration of parallel_for_loop using a threadpool.
Remainders of tasks were ran in the main thread, so parallel_for_each
could theoretically take 2x time than the ideal.

llvm-svn: 288631
2016-12-05 02:07:29 +00:00
Rui Ueyama
1dd86a664f Add paralell_for and use it where appropriate.
When we iterate over numbers as opposed to iterable elements,
parallel_for fits better than parallel_for_each.

llvm-svn: 288002
2016-11-27 19:28:32 +00:00
Rui Ueyama
87ff6fef0f Reduce number of tasks in parallel_for_each.
TaskGroup has a fairly high overhead, so we don't want to partition
tasks into too small tasks. This patch partition tasks into up to
1024 tasks.

I compared this patch with the original LLD's parallel_for_each.
I reverted r287042 locally for comparison.

With this patch, time to self-link lld with debug info changed from
6.23 seconds to 4.62 seconds (-25.8%), with -threads and without -build-id.
With both -threads and -build-id, it improved from 11.71 seconds
to 4.94 seconds (-57.8%). Full results are below.

BTW, GNU gold takes 11.65 seconds to link the same binary.

NOW

--no-threads --build-id=none
       6789.847776 task-clock (msec)         #    1.000 CPUs utilized            ( +-  1.86% )
               685 context-switches          #    0.101 K/sec                    ( +-  2.82% )
                 4 cpu-migrations            #    0.001 K/sec                    ( +- 31.18% )
         1,424,690 page-faults               #    0.210 M/sec                    ( +-  1.07% )
    21,339,542,522 cycles                    #    3.143 GHz                      ( +-  1.49% )
    13,092,260,230 stalled-cycles-frontend   #   61.35% frontend cycles idle     ( +-  2.23% )
   <not supported> stalled-cycles-backend
    21,462,051,828 instructions              #    1.01  insns per cycle
                                             #    0.61  stalled cycles per insn  ( +-  0.41% )
     3,955,296,378 branches                  #  582.531 M/sec                    ( +-  0.39% )
        75,699,909 branch-misses             #    1.91% of all branches          ( +-  0.08% )

       6.787630744 seconds time elapsed                                          ( +-  1.86% )

--threads --build-id=none
      14767.148697 task-clock (msec)         #    3.196 CPUs utilized            ( +-  2.56% )
            28,891 context-switches          #    0.002 M/sec                    ( +-  1.99% )
               905 cpu-migrations            #    0.061 K/sec                    ( +-  5.49% )
         1,262,122 page-faults               #    0.085 M/sec                    ( +-  1.68% )
    43,116,163,217 cycles                    #    2.920 GHz                      ( +-  3.07% )
    33,690,171,242 stalled-cycles-frontend   #   78.14% frontend cycles idle     ( +-  3.67% )
   <not supported> stalled-cycles-backend
    22,836,731,536 instructions              #    0.53  insns per cycle
                                             #    1.48  stalled cycles per insn  ( +-  1.13% )
     4,382,712,998 branches                  #  296.788 M/sec                    ( +-  1.33% )
        78,622,295 branch-misses             #    1.79% of all branches          ( +-  0.54% )

       4.621228056 seconds time elapsed                                          ( +-  1.90% )

--threads --build-id=sha1
      24594.457135 task-clock (msec)         #    4.974 CPUs utilized            ( +-  1.78% )
            29,902 context-switches          #    0.001 M/sec                    ( +-  2.62% )
             1,097 cpu-migrations            #    0.045 K/sec                    ( +-  6.29% )
         1,313,947 page-faults               #    0.053 M/sec                    ( +-  2.36% )
    70,516,415,741 cycles                    #    2.867 GHz                      ( +-  0.78% )
    47,570,262,296 stalled-cycles-frontend   #   67.46% frontend cycles idle     ( +-  0.86% )
   <not supported> stalled-cycles-backend
    73,124,599,029 instructions              #    1.04  insns per cycle
                                             #    0.65  stalled cycles per insn  ( +-  0.33% )
    10,495,266,104 branches                  #  426.733 M/sec                    ( +-  0.41% )
        91,444,149 branch-misses             #    0.87% of all branches          ( +-  0.83% )

       4.944291711 seconds time elapsed                                          ( +-  1.72% )

PREVIOUS

--threads --build-id=none
       7307.437544 task-clock (msec)         #    1.160 CPUs utilized            ( +-  2.34% )
             3,128 context-switches          #    0.428 K/sec                    ( +-  4.37% )
               352 cpu-migrations            #    0.048 K/sec                    ( +-  5.98% )
         1,354,450 page-faults               #    0.185 M/sec                    ( +-  2.20% )
    22,081,733,098 cycles                    #    3.022 GHz                      ( +-  1.46% )
    13,709,991,267 stalled-cycles-frontend   #   62.09% frontend cycles idle     ( +-  1.77% )
   <not supported> stalled-cycles-backend
    21,634,468,895 instructions              #    0.98  insns per cycle
                                             #    0.63  stalled cycles per insn  ( +-  0.86% )
     3,993,062,361 branches                  #  546.438 M/sec                    ( +-  0.83% )
        76,188,819 branch-misses             #    1.91% of all branches          ( +-  0.19% )

       6.298101157 seconds time elapsed                                          ( +-  2.03% )

--threads --build-id=sha1
      12845.420265 task-clock (msec)         #    1.097 CPUs utilized            ( +-  1.95% )
             4,020 context-switches          #    0.313 K/sec                    ( +-  2.89% )
               369 cpu-migrations            #    0.029 K/sec                    ( +-  6.26% )
         1,464,822 page-faults               #    0.114 M/sec                    ( +-  1.37% )
    40,668,449,813 cycles                    #    3.166 GHz                      ( +-  0.96% )
    18,863,982,388 stalled-cycles-frontend   #   46.38% frontend cycles idle     ( +-  1.82% )
   <not supported> stalled-cycles-backend
    71,560,499,058 instructions              #    1.76  insns per cycle
                                             #    0.26  stalled cycles per insn  ( +-  0.14% )
    10,044,152,441 branches                  #  781.925 M/sec                    ( +-  0.19% )
        87,835,773 branch-misses             #    0.87% of all branches          ( +-  0.09% )

      11.711773314 seconds time elapsed                                          ( +-  1.51% )

llvm-svn: 287140
2016-11-16 19:27:33 +00:00
Rafael Espindola
e5cd5ecd2e Use one task per iteration in parallel_for_loop.
This seems far more natural. A user can create larger chunks if the
overhead is too large.

With this linking xul with "--threads --build-id=sha1 goes from
13.938177535 to 11.035953538 seconds on linux.

llvm-svn: 287042
2016-11-15 22:13:16 +00:00
Mehdi Amini
c1edf566b9 Prevent at compile time converting from Error::success() to Expected<T>
This would trigger an assertion at runtime otherwise.

Differential Revision: https://reviews.llvm.org/D26482

llvm-svn: 286562
2016-11-11 04:29:25 +00:00
Rui Ueyama
8c6a5aaf15 Create a vector containing all input sections.
Previously, we do this piece of code to iterate over all input sections.

  for (elf::ObjectFile<ELFT> *F : Symtab.getObjectFiles())
    for (InputSectionBase<ELFT> *S : F->getSections())

It turned out that this mechanisms doesn't work well with synthetic
input sections because synthetic input sections don't belong to any
input file.

This patch defines a vector that contains all input sections including
synthetic ones.

llvm-svn: 286051
2016-11-05 22:37:59 +00:00
Eugene Zelenko
22886a2853 Fix some Clang-tidy modernize-use-default and Include What You Use warnings; other minor fixes.
Differential revision: https://reviews.llvm.org/D26320

llvm-svn: 286030
2016-11-05 01:00:56 +00:00
Eugene Zelenko
e568b91510 Fix some Clang-tidy modernize-use-default and Include What You Use warnings; other minor fixes.
Differential revision: https://reviews.llvm.org/D26293

llvm-svn: 286000
2016-11-04 17:39:46 +00:00
Rafael Espindola
cb09daab0a Call _exit.
As the state of lld gets more complicated, shutting down gets more
expensive.

In a normal lld run we can just call _exit immediately after renaming
the temporary output file. We still want the ability to run a full
shutdown since that is useful for detecting memory leaks.

This patch adds a --full-shutdown flag and changes lit to use it.

llvm-svn: 285224
2016-10-26 18:59:00 +00:00
Rui Ueyama
80695c1aef Truncate a SVN path part from --version output.
This is in sync with what clang does.

llvm-svn: 285163
2016-10-26 03:52:06 +00:00
Rui Ueyama
837182ec1e Remove std:: from std::size_t.
llvm-svn: 284687
2016-10-20 01:33:09 +00:00
Rui Ueyama
4ff3b5adc3 Move getVersionString to Core and simplify Version.cpp.
llvm-svn: 284641
2016-10-19 20:53:07 +00:00
Duncan P. N. Exon Smith
c9e8bc191c ADT: Remove uses of ilist_*sentinel_traits, NFC
llvm-svn: 279458
2016-08-22 18:59:33 +00:00
Duncan P. N. Exon Smith
4a1e87d9c6 Core: Use ilist_full_embedded_sentinel_traits for SimpleReference
Avoid custom code for sentinel traits in SimpleReference (the ilist node
for references to a SimpleDefinedAtom), since they'll soon/eventually
disappear from ilist entirely.

Rather than using a BumpPtrAllocator, this drops the lazy sentinel
characteristics and stores the sentinel directly in the ilist.  This
unconditionally allocates the sentinel.

At first glance, this looks like it might increase memory usage
slightly, since an unreferenced SimpleDefinedAtom pays for a
6-pointer-sized sentinel even when its list of references is empty.  In
practice, the sentinel was being lazily allocated at the first call to
DefinedAtom::begin/end anyway.  I don't expect any real memory effects
here.

Moreover, this is an intermediate state.  The ilist_*sentinel_traits are
being phased out.

As a preview of the final state: in lieu of a NodeTy sentinel, the ilist
will have a single, untemplated list_node_base that has next/prev
pointers.  This base node will serve both as a sentinel and as a pointer
to the head of the list (the same memory layout as
ilist_half_embedded_sentinel_traits, but without the UB).

llvm-svn: 278521
2016-08-12 16:19:34 +00:00
Davide Italiano
54eb8ffd55 [Core] Retire yet another unused member function.
The code in lib/ could use a lot of love :(

llvm-svn: 278506
2016-08-12 12:37:23 +00:00
Duncan P. N. Exon Smith
1a2cb97f7b Core: Remove custom next-prev traits from SimpleReference, NFC
There's nothing special about the _next/_prev pointers in
SimpleReference, so just use ilist_node to do the work.

llvm-svn: 278468
2016-08-12 03:35:47 +00:00
Pete Cooper
46cd113c87 Change when we choose to add an LC_LOAD_DYLIB to the final image.
Currently we do this when an atom is used, but we need to do it when a
dylib is referenced on the cmdline as this matches ld64.

This fixes much confusion over which maps are indexed with installName
vs path.  There is likely other confusion so i'll be seeing if i can remove
path() completely in a future commit as path() shouldn't really be needed by anyone.

llvm-svn: 278396
2016-08-11 20:10:14 +00:00
Davide Italiano
7ca0645c80 [Core] Retire addReplacement() member function.
llvm-svn: 278327
2016-08-11 10:50:01 +00:00
Davide Italiano
c682303d6f [Core] tentativeDefinition() is now unused.
llvm-svn: 278181
2016-08-10 00:01:13 +00:00
Davide Italiano
6bb5d2ddd3 [MachO/Core] Remove (now) unused static member function. NFCI.
llvm-svn: 277760
2016-08-04 19:49:11 +00:00
Peter Collingbourne
feee2103c6 COFF: Implement /linkrepro flag.
This flag is implemented similarly to --reproduce in the ELF linker.

This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.

One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.

Differential Revision: https://reviews.llvm.org/D22418

llvm-svn: 276719
2016-07-26 02:00:42 +00:00
Lang Hames
622ef17f5d [lld] Update LLD for Archive::child_iterator change in LLVM r275361.
llvm-svn: 275362
2016-07-14 02:35:18 +00:00
Lang Hames
a5c7adcb38 [lld][MachO] Remove SimpleFile::definedAtoms().
This method just duplicates the functionality of SimpleFile::defined().

llvm-svn: 274048
2016-06-28 18:42:33 +00:00
Eugene Zelenko
20a38451f5 Fix some Include What You Use warnings; other minor fixes.
Differential revision: http://reviews.llvm.org/D19735

llvm-svn: 268093
2016-04-29 19:47:09 +00:00
Kevin Enderby
dd98c91e08 Needed change to lld for the change to libObject’s getName() for symbols now returning Expected<...>
With the llvm change in r266919 this is the matching needed change to the lld code
now that libObject’s getName() for symbols now returning Expected<...> .

llvm-svn: 266920
2016-04-20 21:24:51 +00:00
Pete Cooper
eadf9e4ec9 Remove unused variants of make_dynamic_error_code. NFC.
make_dynamic_error_code was used to create a std::error_code with
a std::string message.  Now that we are migrating to llvm::Error,
there are no calls to these make_dynamic_error_code methods.

There is one single call to make_dynamic_error_code remaining, the one
inside GenericError::convertToErrorCode().  That method is only called
from File::doParse() which should be a temporary situation.  We need
to work out how to deal with File::parse() caching the error result from
doParse().  Caching errors isn't supported in the new scheme, and probably
isn't needed here, but we need to work that out.

Once thats done, dynamic error and all utilities around it can be deleted.

llvm-svn: 264982
2016-03-31 01:21:50 +00:00
Pete Cooper
d0a643e7dc Change library search methods to return Optional instead of ErrorOr.
These methods weren't really throwing errors.  The only error used
was that a file could not be found, which isn't really an error at all
as we are searching paths and libraries for a file.  All of the callers
also ignored errors and just used the returned path if one was available.

Changing to return Optional<StringRef> as that actually reflects what
we are trying to do here: optionally find a given path.

llvm-svn: 264979
2016-03-31 01:09:35 +00:00
Pete Cooper
fefbd22814 Convert lld file writing to llvm::Error. NFC.
This converts the writeFile method, as well as some of the ones it calls
in the normalized binary file writer and yaml writer.

llvm-svn: 264961
2016-03-30 23:10:39 +00:00
Pete Cooper
3c40b5b750 Convert file handle* methods to llvm::Error instead of std::error_code. NFC.
This updates most of the file handling methods in the linking context and
resolver to use the new API.

llvm-svn: 264924
2016-03-30 20:56:54 +00:00
Pete Cooper
c73c9d273d Convert lld Pass::runOnFile to llvm::Error from std::error_code. NFC.
Pretty mechanical change here.  Just replacing all the std::error_code() with
llvm::Error() and make_dynamic_error_code with make_error<GenericError>

llvm-svn: 264917
2016-03-30 20:36:31 +00:00
Pete Cooper
1e00911ee0 Change getReferenceInfo/getPairReferenceInfo to use new Error handling. NFC.
Adds a GenericError class to lld/Core which can carry a string.  This is
analygous to the dynamic_error we currently use in lld/Core.

Use this GenericError instead of make_dynamic_error_code.  Also, provide
an implemention of GenericError::convertToErrorCode which for now converts
it in to the dynamic_error_code we used to have.  This will go away once
all the APIs are converted.

llvm-svn: 264910
2016-03-30 20:15:06 +00:00
Rui Ueyama
46a8e29587 Remove dead flags.
searchArchivesToOverrideTentativeDefinitions and
searchSharedLibrariesToOverrideTentativeDefinitions are always false.
For the dead flags, we have a fairly large amount of code which is
never be executed.

http://reviews.llvm.org/D17791

llvm-svn: 264653
2016-03-28 20:59:47 +00:00
Pete Cooper
85698f2568 Fix operator= on OwningAtomPtr to call destructor when needed.
If the LHS of 'a = b' already had an atom in it then we wouldn't
call the destructor.  This happens when we use something like
std::remove_if which is done in the CompactUnwindPass.  Should fix
the leaks on the mach-o/unwind-info-simple-x86_64.yaml test case.

Lang and I are going to take a look at removing OwningAtomPtr in
favour of a std::unique_ptr but just trying to get the bots green
so we have a good baseline first.

llvm-svn: 264097
2016-03-22 21:08:39 +00:00
Pete Cooper
c2d21cb763 Appease MSVC bots by changing visibility of AtomVector.
The AtomVector class is an internal detail of File so I moved it
to be protected in r264067.  However, the MSVC bots don't like the
global declarations of type File::AtomVector in File.cpp so it needs
to go back to being public for now.

llvm-svn: 264070
2016-03-22 17:39:44 +00:00
Pete Cooper
8ad55fb2d0 Use owning pointers instead of raw pointers for Atom's to fix leaks.
This is a re-commit of r264022 with a fix for MSVC.  The issue there was
that the code was running DefinedAtom::~Atom() for some value and instead
needed to cast to Atom before running ~Atom.  Original commit message follows.

Currently each File contains an BumpPtrAllocator in which Atom's are
allocated.  Some Atom's contain data structures like std::vector which
leak as we don't run ~Atom when they are BumpPtrAllocate'd.

Now each File actually owns its Atom's using an OwningAtomPtr.  This
is analygous to std::unique_ptr and may be replaced by it if possible.

An Atom can therefore only be owned by a single File, so the Resolver now
moves them from one File to another.  The MachOLinkingContext owns the File's
and so clears all the Atom's in ~MachOLinkingContext, then delete's all the
File's.  This makes sure all Atom's have been destructed before any of the
BumpPtrAllocator's in which they run have gone away.

Should hopefully fix the remaining leaks.  Will keep an eye on the bots to
make sure.

llvm-svn: 264067
2016-03-22 17:15:50 +00:00
Pete Cooper
3e4d732dd0 Revert "Use owning pointers instead of raw pointers for Atom's to fix leaks."
This reverts commit r264022.

This breaks the Window's bots which don't like that i'm calling ~Atom when
the this pointer is a sublcass of Atom.

Reverting for now until I try find a better fix.  I tried using std::unique_ptr with
a custom deleter as a quick fix, but it didn't work well in the YAML parser.

llvm-svn: 264023
2016-03-22 04:00:41 +00:00
Pete Cooper
572a87e2aa Use owning pointers instead of raw pointers for Atom's to fix leaks.
Currently each File contains an BumpPtrAllocator in which Atom's are
allocated.  Some Atom's contain data structures like std::vector which
leak as we don't run ~Atom when they are BumpPtrAllocate'd.

Now each File actually owns its Atom's using an OwningAtomPtr.  This
is analygous to std::unique_ptr and may be replaced by it if possible.

An Atom can therefore only be owned by a single File, so the Resolver now
moves them from one File to another.  The MachOLinkingContext owns the File's
and so clears all the Atom's in ~MachOLinkingContext, then delete's all the
File's.  This makes sure all Atom's have been destructed before any of the
BumpPtrAllocator's in which they run have gone away.

Should hopefully fix the remaining leaks.  Will keep an eye on the bots to
make sure.

llvm-svn: 264022
2016-03-22 03:44:32 +00:00
Pete Cooper
c0c464cac1 Move ownership of Pass File's to the MachoLinkingContext.
In trying to fix the leaks in the MachO lld codebase, we need to have
a better model for file and atom ownership.  Having the context own
everything seems like the simplest model, so change all the passes to
allocate File's on the context instead of owning files as a member.

llvm-svn: 264004
2016-03-21 23:17:47 +00:00