Instead of the GNU extension `SHF_GNU_RETAIN`, Solaris provides equivalent
functionality with `SHF_SUNW_NODISCARD`. This patch implements the necessary
support.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D107955
Users should generally observe no difference as long as they don't use
unintended option forms. Behavior changes:
* `-t=d` is removed. Use `-t d` instead.
* `--demangle=false` and `--demangle=0` cannot be used. Omit the option or use `--no-demangle`. Other flag-style options don't have `--no-` forms.
* `--help-list` is removed. This is a `cl::` specific option.
* llvm-readobj now supports grouped short options as well.
* `--color` is removed. This is generally not useful (only apply to errors/warnings) but was inherited from Support.
Some adjustment to the canonical forms
(usually from GNU readelf; currently llvm-readobj has too many redundant aliases):
* --dyn-syms is canonical. --dyn-symbols is a hidden alias
* --file-header is canonical. --file-headers is a hidden alias
* --histogram is canonical. --elf-hash-histogram is a hidden alias
* --relocs is canonical. --relocations is a hidden alias
* --section-groups is canonical. --elf-section-groups is a hidden alias
OptTable avoids global option collision if we decide to support multiplexing for binary utilities.
* Most one-dash long options are still supported. `-dt, -sd, -st, -sr` are dropped due to their conflict with grouped short options.
* `--section-mapping=false` (D57365) is strange but is kept for now.
* Many `cl::opt` variables were unnecessarily external. I added `static` whenever appropriate.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D105532
llvm-readobj is an internal testing tool for binary formats. Its output and
command line options do not need to be stable. It isn't supposed to be part of a
build process.
llvm-readelf was created as a user-facing utility and its interface intends to
be compatible with GNU readelf (unless there are good reasons not to).
The two tools have mostly compatible options. -s and -t are noticeable
exceptions due to history. I think the cost of keeping the inconsistency
overweighs the little history-compatible benefit and hinders transition from
cl::opt to OptTable, so let's change it.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D105055
Currently when .llvm.call-graph-profile is created by llvm it explicitly encodes the symbol indices. This section is basically a black box for post processing tools. For example, if we run strip -s on the object files the symbol table changes, but indices in that section do not. In non-visible behavior indices point to wrong symbols. The visible behavior indices point outside of Symbol table: "invalid symbol index".
This patch changes the format by using R_*_NONE relocations to indicate the from/to symbols. The Frequency (Weight) will still be in the .llvm.call-graph-profile, but symbol information will be in relocation section. In LLD information from both sections is used to reconstruct call graph profile. Relocations themselves will never be applied.
With this approach post processing tools that handle relocations correctly work for this section also. Tools can add/remove symbols and as long as they handle relocation sections with this approach information stays correct.
Doing a quick experiment with clang-13.
The size went up from 107KB to 322KB, aggregate of all the input sections. Size of clang-13 binary is ~118MB. For users of -fprofile-use/-fprofile-sample-use the size of object files will go up slightly, it will not impact final binary size.
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D104080
This patch adds support for a new field in the FileHeader, which states
the name to use for the section header string table. This also allows
combining the string table with another string table in the object, e.g.
the symbol name string table. The field is optional. By default,
.shstrtab will continue to be used.
This partially fixes https://bugs.llvm.org/show_bug.cgi?id=50506.
Reviewed by: Higuoxing
Differential Revision: https://reviews.llvm.org/D104035
There was an off-by-one error caused by an index (which included an
index for the null section header) being used to check against the size
of a list of sections (which didn't include the null section header).
This is a partial fix for https://bugs.llvm.org/show_bug.cgi?id=50506.
Reviewed by: MaskRay
Differential Revision: https://reviews.llvm.org/D104098
This patch uses the errno python library to print out the correct error messages instead of hardcoding the error message per platform.
Reviewed By: jhenderson, ASDenysPetrov
Differential Revision: https://reviews.llvm.org/D97472
As discussed in D95511, this allows us to encode invalid BBAddrMap
sections to be used in more rigorous testing.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D96831
On z/OS, other error messages are not matched correctly in lit tests.
```
EDC5121I Invalid argument.
EDC5111I Permission denied.
```
This patch adds a lit substitution to fix it.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D95808
Currently we don't allow the following definition:
```
Sections:
- Type: SectionHeaderTable
- Name: .foo
Type: SHT_PROGBITS
```
We report an error: "SectionHeaderTable can't be empty. Use 'NoHeaders' key to drop the section header table".
It was implemented in this way earlier, when `SectionHeaderTable`
was a dedicated key outside of the `Sections` list. And we did not
allow to select where the table is written.
Currently it makes sense to allow it, because a user might
want to place the default section header table at an arbitrary position,
e.g. before other sections. In this case it is not convenient and error prone
to require specifying all sections:
```
Sections:
- Type: SectionHeaderTable
Sections:
- Name: .foo
- Name: .strtab
- Name: .shstrtab
- Name: .foo
Type: SHT_PROGBITS
```
This patch allows empty SectionHeaderTable definitions.
Differential revision: https://reviews.llvm.org/D95341
This was discussed in D93678 thread.
Currently we have one special chunk - Fill.
This patch re implements the "SectionHeaderTable" key to become a special chunk too.
With that we are able to place the section header table at any location,
just like we place sections.
Differential revision: https://reviews.llvm.org/D95140
This makes the following improvements.
For `SHT_GNU_versym`:
* yaml2obj: set `sh_link` to index of `.dynsym` section automatically.
For `SHT_GNU_verdef`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version definitions.
For `SHT_GNU_verneed`:
* yaml2obj: set `sh_link` to index of `.dynstr` section automatically.
* yaml2obj: set `sh_info` field automatically.
* obj2yaml: don't dump the `Info` field when its value matches the number of version dependencies.
Also, simplifies few test cases.
Differential revision: https://reviews.llvm.org/D94956
This reuses the code from yaml2obj (moves it to ELFYAML.h).
With it we can set the `sh_entsize` in a single place in `obj2yaml`.
Note that it also fixes a bug of `yaml2obj`: we do not
set the `sh_entsize` field for the `SHT_ARM_EXIDX` section properly.
Differential revision: https://reviews.llvm.org/D93858
Currently we don't support multiple SHT_SYMTAB_SHNDX sections
and the DT_SYMTAB_SHNDX tag currently.
This patch implements it and fixes the
https://bugs.llvm.org/show_bug.cgi?id=43991.
I had to introduce the `struct DataRegion` to ELF.h,
it is used to represent a region that might have no known size.
It is needed, because we don't know the size of the extended
section indices table when it is located via DT_SYMTAB_SHNDX.
In this case we still want to validate that we don't read
past the end of the file.
Differential revision: https://reviews.llvm.org/D92923
We have the following issues related to group sections:
1) yaml2obj is unable to set the custom `sh_entsize` value, because the `EntSize`
key is currently ignored.
2) obj2yaml is unable to dump the group section which `sh_entsize != 4`.
3) obj2yaml always dumps the "EntSize" for group sections, though
usually we are trying to omit dumping default values when dumping keys.
I.e. we should not print the "EntSize" key when `sh_entsize` == 4.
This patch fixes (1),(3) and adds the test case to document the behavior of (2).
Differential revision: https://reviews.llvm.org/D93854
Currently, `ELFFile<ELFT>::getEntry` does not check an index of
an entry. Because of that the code might read past the end of the symbol
table silently. I've added a test to `llvm-readobj\ELF\relocations.test`
to demonstrate the possible issue. Also, I've added a unit test for
this method.
After this change, `getEntry` stops reporting the section index and
reuses the `getSectionContentsAsArray` method, which already has
all the validation needed. Our related warnings now provide
more and better context sometimes.
Differential revision: https://reviews.llvm.org/D93209
When a field is optional we can use the `=<none>` syntax in macros.
This patch makes `Value`/`Size` fields of `Symbol` optional
and adds test cases for them.
Differential revision: https://reviews.llvm.org/D93010
This is similar to what we did earlier for fields of the Section class.
When a field is optional we can use the =<none> syntax in macros.
This was splitted from D92478.
Differential revision: https://reviews.llvm.org/D92565
When we produce an YAML output, we also print leading zeroes currently.
An output might look like this:
```
- Name: .dynsym
Type: SHT_DYNSYM
Address: 0x0000000000001000
EntSize: 0x0000000000000018
```
There are probably no reason to print leading zeroes.
It just makes harder to read values. This patch stops printing them.
The output becomes like:
```
- Name: .dynsym
Type: SHT_DYNSYM
Address: 0x1000
EntSize: 0x18
```
This affects obj2yaml mostly, but also dsymutil and llvm-xray tools output.
Differential revision: https://reviews.llvm.org/D90930
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.
To put them into segment we can do (1*):
```
Sections:
- Section: foo1
- Section: foo4
```
or we can use (2*):
```
Sections:
- Section: foo1
- Section: foo3
- Section: foo4
```
or (3*) :
```
Sections:
- Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
- Section: (index 2)
- Section: foo3
- Section: foo4
```
It looks really confusing that we don't have to list all of sections.
At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).
This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.
Differential revision: https://reviews.llvm.org/D90458
This is recommit for D90903 with fixes for BB:
1) Used std::move<> when returning Expected<> (http://lab.llvm.org:8011/#/builders/112/builds/913)
2) Fixed the name of temporarily file in the file-headers.test (http://lab.llvm.org:8011/#/builders/36/builds/1269)
(a local old temporarily file was used before)
For creating `ELFObjectFile` instances we have the factory method
`ELFObjectFile<ELFT>::create(MemoryBufferRef Object)`.
The problem of this method is that it scans the section header to locate some sections.
When a file is truncated or has broken fields in the ELF header, this approach does
not allow us to create the `ELFObjectFile` and dump the ELF header.
This is https://bugs.llvm.org/show_bug.cgi?id=40804
This patch suggests a solution - it allows to delay scaning sections in the
`ELFObjectFile<ELFT>::create`. It now allows user code to call an object
initialization (`initContent()`) later. With that it is possible,
for example, for dumpers just to dump the file header and exit.
By default initialization is still performed as before, what helps to keep
the logic of existent callers untouched.
I've experimented with different approaches when worked on this patch.
I think this approach is better than doing initialization of sections (i.e. scan of them)
on demand, because normally users of `ELFObjectFile` API expect to work with a valid object.
In most cases when a section header table can't be read (because of an error), we don't
have to continue to work with object. So we probably don't need to implement a more complex API.
Differential revision: https://reviews.llvm.org/D90903
For creating `ELFObjectFile` instances we have the factory method
`ELFObjectFile<ELFT>::create(MemoryBufferRef Object)`.
The problem of this method is that it scans the section header to locate some sections.
When a file is truncated or has broken fields in the ELF header, this approach does
not allow us to create the `ELFObjectFile` and dump the ELF header.
This is https://bugs.llvm.org/show_bug.cgi?id=40804
This patch suggests a solution - it allows to delay scaning sections in the
`ELFObjectFile<ELFT>::create`. It now allows user code to call an object
initialization (`initContent()`) later. With that it is possible,
for example, for dumpers just to dump the file header and exit.
By default initialization is still performed as before, what helps to keep
the logic of existent callers untouched.
I've experimented with different approaches when worked on this patch.
I think this approach is better than doing initialization of sections (i.e. scan of them)
on demand, because normally users of `ELFObjectFile` API expect to work with a valid object.
In most cases when a section header table can't be read (because of an error), we don't
have to continue to work with object. So we probably don't need to implement a more complex API.
Differential revision: https://reviews.llvm.org/D90903
YAML support allows us to better test the feature in the subsequent patches. The implementation is quite similar to the .stack_sizes section.
Reviewed By: jhenderson, grimar
Differential Revision: https://reviews.llvm.org/D88717
Currently it is impossible to create an instance of ELFObjectFile when the
SHT_SYMTAB_SHNDX can't be read. We error out when fail to parse the
SHT_SYMTAB_SHNDX section in the factory method.
This change delays reading of the SHT_SYMTAB_SHNDX section entries,
with it llvm-readobj is now able to work with such inputs.
Differential revision: https://reviews.llvm.org/D89379
These sections are implicit and handled a bit differently.
Currently the "Offset" is ignored for them.
This patch fixes an issue.
Differential revision: https://reviews.llvm.org/D90446
`Link` is not an optional field currently.
Because of this it is not convenient to write macros.
This makes it optional and fixes corresponding test cases.
Differential revision: https://reviews.llvm.org/D90390
This creates `strtab-implicit-sections.yaml` and merges 2 `strtab-implicit-sections*` tests into it.
I've also added a few tests for `.shstrtab` section related to section flags.
With that we have a single place where we can test implicit string table sections and
the `.shstrtab` section in particular.
Differential revision: https://reviews.llvm.org/D90372
This simplifies the dynsymtab-shlink.yaml test (with use of macros)
and merges it into the dynsym-section.yaml test.
Differential revision: https://reviews.llvm.org/D90301
When `NoHeaders` is set, we still have following issues:
1) We emit the `.shstrtab` implicit section of size 1 (empty string table).
2) We still align the start of the section header table, what affects the output size.
3) We still write section header table bytes.
This patch fixes all of these issues.
Differential revision: https://reviews.llvm.org/D90295
Our "implicit" sections are handled separately from regular ones.
It turns out that the "Offset" key is not handled properly for them.
Perhaps we can generalize handling in one place, but before doing that I'd like
to add support and test cases for each implicit section.
(I need this particular single change to unblock another patch that is already on review,
and I guess doing it independently for each section will be cleaner, see below).
In this patch I've removed `explicit-dynsym-no-dynstr.yaml` to `dynsym-section.yaml`
and added the new test into. In a follow-up we probably might want
to merge 2 another existent `dynsymtab-*.yaml` tests into it too.
Differential revision: https://reviews.llvm.org/D90224
Imagine the following declaration of a section:
```
Sections:
- Name: .dynsym
Type: SHT_DYNSYM
AddressAlign: 0x1111111111111111
```
The aligment is large and yaml2obj reports an error currently:
"the desired output size is greater than permitted. Use the --max-size option to change the limit"
This patch implements the "ShAddrAlign" key, which is similar to other "Sh*" keys we have.
With it it is possible to override the `sh_addralign` field, ignoring the writing of alignment bytes.
Differential revision: https://reviews.llvm.org/D90019
This reverts commit 1b589f4d4db27e3fcd81fdc5abeb9407753ab790 and relands the D89463
with the fix: update `MappingTraits<FileFilter>::validate()` in ClangTidyOptions.cpp to
match the new signature (change the return type to "std::string" from "StringRef").
Original commit message:
This:
Changes the return type of MappingTraits<T>>::validate to std::string
instead of StringRef. It allows to create more complex error messages.
It introduces std::vector<std::pair<StringRef, bool>> getEntries():
a new virtual method of Section, which is the base class for all sections.
It returns names of special section specific keys (e.g. "Entries") and flags that says if them exist in a YAML.
The code in validate() uses this list of entries descriptions to generalize validation.
This approach was discussed in the D89039 thread.
Differential revision: https://reviews.llvm.org/D89463
This:
1) Changes the return type of `MappingTraits<T>>::validate` to `std::string`
instead of `StringRef`. It allows to create more complex error messages.
2) It introduces std::vector<std::pair<StringRef, bool>> getEntries():
a new virtual method of Section, which is the base class for all sections.
It returns names of special section specific keys (e.g. "Entries") and flags that
says if them exist in a YAML. The code in validate() uses this list of entries
descriptions to generalize validation.
This approach was discussed in the D89039 thread.
Differential revision: https://reviews.llvm.org/D89463
Currently we have a few sections that
does not support specifying no keys for them. E.g. it is required that one
of "Content", "Size" or "Entries" key is present. There is no reason to
have this restriction. We can allow this and emit an empty section instead.
This opens road for a simplification and generalization of the code in `validate()`
that is discussed in the D89039 thread.
Depends on D89039.
Differential revision: https://reviews.llvm.org/D89391
Many sections either do not have a support of `Size`/`Content` or support just a
one of them, e.g only `Content`.
`Section` is the base class for sections. This patch adds `Content` and `Size` members
to it and removes similar members from derived classes. This allows to cleanup and
generalize the code and adds a support of these keys for all sections (`SHT_MIPS_ABIFLAGS`
is a only exception, it requires unrelated specific changes to be done).
I had to update/add many tests to test the new functionality properly.
Differential revision: https://reviews.llvm.org/D89039
This patch makes the opcode_base and the standard_opcode_lengths fields
of the line table optional. When both of them are not specified,
yaml2obj emits them according to the line table's version.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D88355
Specification for SHT_HASH table says (https://refspecs.linuxbase.org/elf/gabi4+/ch5.dynamic.html#hash)
that it contains Elf32_Word entries for both 32/64 bit objects.
Currently both GNU linkers and LLD sets the `sh_entsize` field to `4`.
At the same time, `yaml2obj` ignores the `EntSize` field for SHT_HASH sections.
This patch fixes this and also adds a support for obj2yaml: it will not
dump this field when the `sh_entsize` contains the default value (`4`).
Differential revision: https://reviews.llvm.org/D88652
Currently we are always recognizing the `SHT_MIPS_ABIFLAGS` section,
even on non-MIPS targets.
The problem of doing this is briefly discussed in D88228 which does the same for `SHT_ARM_EXIDX`:
"The problem is that `SHT_ARM_EXIDX` shares the value with `SHT_X86_64_UNWIND (0x70000001U)`.
We might have other machine specific conflicts, e.g.
`SHT_ARM_ATTRIBUTES` vs `SHT_MSP430_ATTRIBUTES` vs `SHT_RISCV_ATTRIBUTES (0x70000003U)`."
I think we should only recognize target specific sections when the machine type
matches. I.e. `SHT_MIPS_*` should be recognized only on `MIPS`, `SHT_ARM_*`
only on `ARM` etc.
This patch stops recognizing `SHT_MIPS_ABIFLAGS` on `non-MIPS` targets.
Note: I had to update `ScalarEnumerationTraits<ELFYAML::MIPS_ISA>::enumeration`, because
otherwise test crashes, calling `llvm_unreachable`.
Differential revision: https://reviews.llvm.org/D88294
We have the `Object/Mips/abi-flags.yaml` which tests how yaml2obj/obj2yaml
handle `SHT_MIPS_ABIFLAGS` sections.
This patch splits it into two tests: one for obj2yaml and one for yaml2obj
and moves the result to right places.
Differential revision: https://reviews.llvm.org/D88231
This patch makes the 'ExtLen' field of extended opcodes optional. We
don't need to manually calculate it in the future.
Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D88136
