LLVM is inconsistent about how it converts `errno` to `std::error_code`.
This can cause problems because values outside of `std::errc` compare
differently if one is system and one is generic on POSIX systems.
This is even more of a problem on Windows where use of the system
category is just wrong, as that is for Windows errors, which have a
completely different mapping than POSIX/generic errors. This patch fixes
one instance of this mistake in `JSONTransport.cpp`.
This patch adds `errnoAsErrorCode()` which makes it so people do not
need to think about this issue in the future. It also cleans up a lot of
usage of `errno` in LLVM and Clang.
Detect COFF files by default and allow specifying it with --format
argument.
This is important for ARM64EC, which uses a separated symbol map for EC
symbols. Since K_COFF is mostly compatible with K_GNU, this shouldn't
really make a difference for other targets.
Note that llvm::support::endianness has been renamed to
llvm::endianness while becoming an enum class as opposed to an
enum. This patch replaces support::{big,little,native} with
llvm::endianness::{big,little,native}.
Summary:
if the member file is XCOFF object file and has auxiliary header, the content of the member file need to be aligned at the
MAX(maximum alignment of .text , maximum alignment of .data). The "maximum alignment of .text" and "maximum alignment of .data" are two
field of auxiliary header of XCOFF object file.
Reviewers: James Henderson, Stephen Peckham
Differential Revision: https://reviews.llvm.org/D144872
Summary:
llvm-ar is symlinked as llvm-ranlib and will act as ranlib when invoked in that mode. llvm-ar since [[ 4f2cfbe531 | compiler/llvm-project@4f2cfbe ]] supports the -X options, but doesn't seem to accept them when running as llvm-ranlib.
In AIX OS , according to https://www.ibm.com/docs/en/aix/7.2?topic=r-ranlib-command
-X mode Specifies the type of object file ranlib should examine. The mode must be one of the following:
32
Processes only 32-bit object files
64
Processes only 64-bit object files
32_64, any
Processes both 32-bit and 64-bit object files
The default is to process 32-bit object files (ignore 64-bit objects). The mode can also be set with the OBJECT_MODE environment variable. For example, OBJECT_MODE=64 causes ranlib to process any 64-bit objects and ignore 32-bit objects. The -X flag overrides the OBJECT_MODE variable.
Reviewers: James Henderson, MaskRay, Stephen Peckham
Differential Revision: https://reviews.llvm.org/D142660
Summary:
In big archive , there is 32bit global symbol table and 64 bit global symbol table. llvm-ar only support 32bit global symbol table this moment, we need to support the 64 bit global symbol table.
https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
Global Symbol Tables
Immediately following the member table, the archive file contains two global symbol tables. The first global symbol table locates 32-bit file members that define global symbols; the second global symbol table does the same for 64-bit file members. If the archive has no 32-bit or 64-bit file members, the respective global symbol table is omitted. The strip command can be used to delete one or both global symbol tables from the archive. The fl_gstoff field in the fixed-length header contains the offset to the 32-bit global symbol table, and the fl_gst64off contains the offset to the 64-bit global symbol table.
Reviewers: James Henderson,Stephen Peckham
Differential Revision: https://reviews.llvm.org/D142479
Summary:
1. we use the unsigned type for NextOffset,PrevOffset ,GlobalSymbolOffset , MemberTableSize, it will caused a malform big archive when the archive file size is large than 4G.
2. also fix a NFC comment on https://reviews.llvm.org/D142479#inline-1443927
Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D150462
On AIX, when the input files are LLVM bitcode files, `llvm-ar` should set the archive kind to `K_AIXBIG` as well, instead of leaving it to the default `K_GNU`.
Reviewed By: daltenty
Differential Revision: https://reviews.llvm.org/D149377
ARM64EC allows having both pure ARM64 objects and ARM64EC in the
same archive. This allows using single static library for linking
pure ARM64, pure ARM64EC or mixed modules (what MS calls ARM64X:
a single module that may be used in both modes). To achieve that,
such static libraries need two separated symbol maps. The usual map
contains only pure ARM64 symbols, while a new /<ECSYMBOLS>/ section
contains EC symbols. EC symbols map has very similar format to the
usual map, except it doesn't contain object offsets and uses offsets
from regular map instead. This is true even for pure ARM64EC static
library: it will simply have 0 symbols in the symbol map.
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D143541
We currently just use GNU format for llvm-lib. This mostly works, but
ARM64EC needs an additional section that does not really fit GNU format.
This patch implements writing in COFF format (as in, it's what archive
reader considers as K_COFF). This mostly requires symbol emitting symbol
map. Note that, just like in case of MSVC, symbols are de-duplicated in
both usual symbol table and the new symbol map.
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D143540
Having string table in members vector does not fit later patches in
this series. Symbol map needs to refer to objects' offsets, but string
table should not be referenced. Also for ARM64EC, the new <ECSYMBOLS>
table is inserted after string table.
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D143538
Print out the archive member that failed, to make debugging easier.
Before:
error: failed to build archive: Not an int attribute (Producer: 'LLVM15.0.1git' Reader: 'LLVM 14.0.5-rust-dev')
After:
error: failed to build archive: 'fake_bt_keystore.o': Not an int attribute (Producer: 'LLVM15.0.1git' Reader: 'LLVM 14.0.5-rust-dev')
Differential Revision: https://reviews.llvm.org/D133607
When creating an archive, llvm-ar looks at the host to determine the
archive format to use, on Apple platforms this means it uses the
K_DARWIN format. K_DARWIN is _virtually_ equivalent to K_BSD, expect for
some very slight differences around padding, timestamps in deterministic
mode, and 64 bit formats. When updating an archive using llvm-ar, or
llvm-objcopy, Archive would try to determine the kind, but it was not
possible to get K_DARWIN in the initialization of the archive, because
they're virtually inciting usable from K_BSD, especially since the
slight differences only apply in very specific cases. This leads to
linker failures when the alignment workaround is not applied to an
archive copied with llvm-objcopy. This change teaches Archive to infer
the K_DARWIN type in the cases where it's possible and the first object
in the archive is a macho object. This avoids using the host triple to
determine this to not affect cross compiling.
Ideally we would eliminate the separate K_DARWIN type entirely since
it's not a truly separate archive type, but then we'd have to force the
macho workarounds on the BSD format generally. This might be acceptable
but then it would be unclear how to handle this case without forcing the
K_DARWIN64 format on all BSD users:
```
if (LastOffset >= Sym64Threshold) {
if (Kind == object::Archive::K_DARWIN)
Kind = object::Archive::K_DARWIN64;
else
Kind = object::Archive::K_GNU64;
}
```
The logic used to determine if the object is macho is derived from the
logic llvm-ar uses.
Previous context:
- 111cd669e90e5b2132187d36f8b141b11a671a8b
- 23a76be5adcaa768ba538f8a4514a7afccf61988
Differential Revision: https://reviews.llvm.org/D124895
SUMMARY
1. Enable supporting the write operation of big archive.
2. the first commit come from https://reviews.llvm.org/D104367
3. refactor the first commit and implement writing symbol table.
4. fixed the bugs and add more test cases in the second commit.
Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D123949
Most notably,
llvm/Object/Binary.h no longer includes llvm/Support/MemoryBuffer.h
llvm/Object/MachOUniversal*.h no longer include llvm/Object/Archive.h
llvm/Object/TapiUniversal.h no longer includes llvm/Object/TapiFile.h
llvm-project preprocessed size:
before: 1068185081
after: 1068324320
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119457
Summary:
The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.
the rest of commits of the patch
1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted
3 introduce a new derived class BigArchive for big ar file.
Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889
Summary:
The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.
the rest of commits of the patch
1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted
3 introduce a new derived class BigArchive for big ar file.
Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889
Summary:
The patch is based on the EGuesnet's implement of the "Support of Big archive (read)
the first commit of the patch is come from https://reviews.llvm.org/D100651.
the rest of commits of the patch
1 Addressed the comments on the https://reviews.llvm.org/D100651
2 according to https://www.ibm.com/docs/en/aix/7.2?topic=formats-ar-file-format-big
using the "fl_fstmoff" for the first object file number, using "char ar_nxtmem[20]" to get next object file ,
using the "char fl_lstmoff[20]" for the last of the object file will fix the following problems:
2.1 can not correct reading a archive files which has padding data between too object file
2.2 can not correct reading a archive files from which some object file has be deleted
3 introduce a new derived class BigArchive for big ar file.
Reviewers: James Henderson
Differential Revision: https://reviews.llvm.org/D111889
Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (`mmap` vs `open`),
* assert newly initialized buffer indeed has an implicit null terminator.
This patch adds the paramater to the `SmallVectorMemoryBuffer` constructors, meaning:
* null terminator can now be added to `SmallVector`s that didn't have one before,
* `SmallVectors` that had a null terminator before keep it even after the move.
In line with existing code, the new parameter is defaulted to `true`. This patch makes sure all calls to the `SmallVectorMemoryBuffer` constructor set it to `false` to preserve the current semantics.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D115331
The code to detect the requirement for 64-bit offsets in the archive
symbol table was not correctly accounting for the archive file signature
and the size of all the contents of the symbol table itself, e.g. the
symbol table's header and string table. Also was not considering the
variation in symbol table formats. This could result in the creation of
large archives with a corrupt symbol table.
Change the testing environment variable SYM64_THRESHOLD to be an
absolute value rather than a power of 2 in order to enable precise
testing of this detection code.
Differential Revision: https://reviews.llvm.org/D89891
When adding an archive member with a problem, e.g. a new bitcode with an
old archiver, containing an unsupported attribute, or an ELF file with a
malformed symbol table, the archiver would throw away the error and
simply add the member to the archive without any symbol entries. This
meant that the resultant archive could be silently unusable when not
using --whole-archive, and result in unexpected undefined symbols.
This change fixes this issue by addressing two FIXMEs and only throwing
away not-an-object errors. However, this meant that some LLD tests which
didn't need symbol tables and were using invalid members deliberately to
test the linker's malformed input handling no longer worked, so this
patch also stops the archiver from looking for symbols in an object if
it doesn't require a symbol table, and updates the tests accordingly.
Differential Revision: https://reviews.llvm.org/D88288
Reviewed by: grimar, rupprecht, MaskRay
Refactoring function `writeArchive` in ArchiveWriter. Added a new
function `writeArchiveBuffer` that returns the archive in a memory
buffer instead of writing it out to the disk. This refactor is necessary
so as to allow `llvm-libtool-darwin` to write universal files containing
archives.
Reviewed by jhenderson, MaskRay, smeenai
Differential Revision: https://reviews.llvm.org/D84858
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790
Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson
Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D67499
llvm-svn: 371742
llvm-ar outputs a strange error message when handling archives with
members larger than 4GB due to not checking file size when passing the
value as an unsigned 32 bit integer. This overflow issue caused
malformed archives to be created.:
https://bugs.llvm.org/show_bug.cgi?id=38058
This change allows for members above 4GB and will error in a case that
is over the formats size limit, a 10 digit decimal integer.
Differential Revision: https://reviews.llvm.org/D65093
llvm-svn: 366813
Summary:
On Windows, Posix integer file descriptors are a compatibility layer
over native file handles provided by the C runtime. There is a hard
limit on the maximum number of file descriptors that a process can open,
and the limit is 8192. LLD typically doesn't run into this limit because
it opens input files, maps them into memory, and then immediately closes
the file descriptor. This prevents it from running out of FDs.
For various reasons, I'd like to open handles to every input file and
keep them open during linking. That requires migrating MemoryBuffer over
to taking open native file handles instead of integer FDs.
Reviewers: aganea, Bigcheese
Reviewed By: aganea
Subscribers: smeenai, silvas, mehdi_amini, hiraditya, steven_wu, dexonsmith, dang, llvm-commits, zturner
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D63453
llvm-svn: 365588
Includes a fix for an introduced build failure due to a post c++11 use of std::mismatch.
This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.
Differential Revision: https://reviews.llvm.org/D59491
llvm-svn: 362484
This reverts commit r362407. It broke compilation of
llvm/lib/Object/ArchiveWriter.cpp:
error: type 'llvm::sys::path::const_iterator' does not provide a call
operator
llvm-svn: 362413
This fixes some thin archive relative path issues, paths are shortened where possible and paths are output correctly when using the display table command.
Differential Revision: https://reviews.llvm.org/D59491
llvm-svn: 362407
Summary:
When adding one thin archive to another, we currently chop off the relative path to the flattened members. For instance, when adding `foo/child.a` (which contains `x.txt`) to `parent.a`, when flattening it we should add it as `foo/x.txt` (which exists) instead of `x.txt` (which does not exist).
As a note, this also undoes the `IsNew` parameter of handling relative paths in r288280. The unit test there still passes.
This was reported as part of testing the kernel build with llvm-ar: https://patchwork.kernel.org/patch/10767545/ (see the second point).
Reviewers: mstorsjo, pcc, ruiu, davide, david2050, inglorion
Reviewed By: ruiu
Subscribers: void, jdoerfert, tpimh, mgorny, hans, nickdesaulniers, hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57842
llvm-svn: 353995
This broke the Chromium build on Windows, see https://crbug.com/930058
> Summary:
> When adding one thin archive to another, we currently chop off the relative path to the flattened members. For instance, when adding `foo/child.a` (which contains `x.txt`) to `parent.a`, whe
> lattening it we should add it as `foo/x.txt` (which exists) instead of `x.txt` (which does not exist).
>
> As a note, this also undoes the `IsNew` parameter of handling relative paths in r288280. The unit test there still passes.
>
> This was reported as part of testing the kernel build with llvm-ar: https://patchwork.kernel.org/patch/10767545/ (see the second point).
>
> Reviewers: mstorsjo, pcc, ruiu, davide, david2050
>
> Subscribers: hiraditya, llvm-commits
>
> Tags: #llvm
>
> Differential Revision: https://reviews.llvm.org/D57842
This reverts commit bf990ab5aab03aa0aac53c9ef47ef264307804ed.
llvm-svn: 353507
Summary:
When adding one thin archive to another, we currently chop off the relative path to the flattened members. For instance, when adding `foo/child.a` (which contains `x.txt`) to `parent.a`, when flattening it we should add it as `foo/x.txt` (which exists) instead of `x.txt` (which does not exist).
As a note, this also undoes the `IsNew` parameter of handling relative paths in r288280. The unit test there still passes.
This was reported as part of testing the kernel build with llvm-ar: https://patchwork.kernel.org/patch/10767545/ (see the second point).
Reviewers: mstorsjo, pcc, ruiu, davide, david2050
Subscribers: hiraditya, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57842
llvm-svn: 353424
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
