This rename was made as part of
https://github.com/llvm/llvm-project/pull/147835 in order to ease
rebasing the PR, and give a nice window for other patches to get rebased
as well.
It has been a while already, so lets go ahead and rename it back.
Switch to the `.Cases({S0, S1, ...}, Value)` overload instead, and the
manually-enumerated overloads with 6+ arguments are getting deprecated
in https://github.com/llvm/llvm-project/pull/163405.
This pre-commits API updates ahead of the deprecation to make potential
reverts cleaner. This was already reviewed in #163405.
This is a major change on how we represent nested name qualifications in
the AST.
* The nested name specifier itself and how it's stored is changed. The
prefixes for types are handled within the type hierarchy, which makes
canonicalization for them super cheap, no memory allocation required.
Also translating a type into nested name specifier form becomes a no-op.
An identifier is stored as a DependentNameType. The nested name
specifier gains a lightweight handle class, to be used instead of
passing around pointers, which is similar to what is implemented for
TemplateName. There is still one free bit available, and this handle can
be used within a PointerUnion and PointerIntPair, which should keep
bit-packing aficionados happy.
* The ElaboratedType node is removed, all type nodes in which it could
previously apply to can now store the elaborated keyword and name
qualifier, tail allocating when present.
* TagTypes can now point to the exact declaration found when producing
these, as opposed to the previous situation of there only existing one
TagType per entity. This increases the amount of type sugar retained,
and can have several applications, for example in tracking module
ownership, and other tools which care about source file origins, such as
IWYU. These TagTypes are lazily allocated, in order to limit the
increase in AST size.
This patch offers a great performance benefit.
It greatly improves compilation time for
[stdexec](https://github.com/NVIDIA/stdexec). For one datapoint, for
`test_on2.cpp` in that project, which is the slowest compiling test,
this patch improves `-c` compilation time by about 7.2%, with the
`-fsyntax-only` improvement being at ~12%.
This has great results on compile-time-tracker as well:
This patch also further enables other optimziations in the future, and
will reduce the performance impact of template specialization resugaring
when that lands.
It has some other miscelaneous drive-by fixes.
About the review: Yes the patch is huge, sorry about that. Part of the
reason is that I started by the nested name specifier part, before the
ElaboratedType part, but that had a huge performance downside, as
ElaboratedType is a big performance hog. I didn't have the steam to go
back and change the patch after the fact.
There is also a lot of internal API changes, and it made sense to remove
ElaboratedType in one go, versus removing it from one type at a time, as
that would present much more churn to the users. Also, the nested name
specifier having a different API avoids missing changes related to how
prefixes work now, which could make existing code compile but not work.
How to review: The important changes are all in
`clang/include/clang/AST` and `clang/lib/AST`, with also important
changes in `clang/lib/Sema/TreeTransform.h`.
The rest and bulk of the changes are mostly consequences of the changes
in API.
PS: TagType::getDecl is renamed to `getOriginalDecl` in this patch, just
for easier to rebasing. I plan to rename it back after this lands.
Fixes#136624
Fixes https://github.com/llvm/llvm-project/issues/43179
Fixes https://github.com/llvm/llvm-project/issues/68670
Fixes https://github.com/llvm/llvm-project/issues/92757
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
With the introduction of tbd-v5 holding rpaths, the order in which those
attributes are passed to `clang-installapi` must be represented in tbd
files. Previously, all dylib attributes were stored in a
non-deterministic `StringMap`. Instead, hold them in a custom collection
with an underlying vector to continue supporting searching by attribute.
This makes the order of all diagnostics related to load command
comparisons stable.
This approach resolves errors when building with reverse-iteration.
* Have clang always append & pass System/Library/SubFrameworks when determining default sdk search paths.
* Teach clang-installapi to traverse there for framework input.
* Teach llvm-readtapi that the library files (TBD or binary) in there should be considered private.
resolves: rdar://137457006
Some `FileManager` APIs still return `{File,Directory}Entry` instead of
the preferred `{File,Directory}EntryRef`. These are documented to be
deprecated, but don't have the attribute that warns on their usage. This
PR marks them as such with `LLVM_DEPRECATED()` and replaces their usage
with the recommended counterparts. NFCI.
`$ld$previous` symbols need to be exported for them to be seen by
clients. TAPI cannot omit them in tbd files, so account for this in
installapi verification when handling reexport verification.
Reviewed internally by Zixu Wang
resolves: rdar://131317591
Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.
Work towards TODO item to remove `raw_string_ostream::str()`.
Virtual function pointer entries in v-tables are signed with address
discrimination in addition to declaration-based discrimination, where an
integer discriminator the string hash (see
`ptrauth_string_discriminator`) of the mangled name of the overridden
method. This notably provides diversity based on the full signature of
the overridden method, including the method name and parameter types.
This patch introduces ItaniumVTableContext logic to find the original
declaration of the overridden method.
On AArch64, these pointers are signed using the `IA` key (the
process-independent code key.)
V-table pointers can be signed with either no discrimination, or a
similar scheme using address and decl-based discrimination. In this
case, the integer discriminator is the string hash of the mangled
v-table identifier of the class that originally introduced the vtable
pointer.
On AArch64, these pointers are signed using the `DA` key (the
process-independent data key.)
Not using discrimination allows attackers to simply copy valid v-table
pointers from one object to another. However, using a uniform
discriminator of 0 does have positive performance and code-size
implications on AArch64, and diversity for the most important v-table
access pattern (virtual dispatch) is already better assured by the
signing schemas used on the virtual functions. It is also known that
some code in practice copies objects containing v-tables with `memcpy`,
and while this is not permitted formally, it is something that may be
invasive to eliminate.
This is controlled by:
```
-fptrauth-vtable-pointer-type-discrimination
-fptrauth-vtable-pointer-address-discrimination
```
In addition, this provides fine-grained controls in the
ptrauth_vtable_pointer attribute, which allows overriding the default
ptrauth schema for vtable pointers on a given class hierarchy, e.g.:
```
[[clang::ptrauth_vtable_pointer(no_authentication, no_address_discrimination,
no_extra_discrimination)]]
[[clang::ptrauth_vtable_pointer(default_key, default_address_discrimination,
custom_discrimination, 0xf00d)]]
```
The override is then mangled as a parametrized vendor extension:
```
"__vtptrauth" I
<key>
<addressDiscriminated>
<extraDiscriminator>
E
```
To support this attribute, this patch adds a small extension to the
attribute-emitter tablegen backend.
Note that there are known areas where signing is either missing
altogether or can be strengthened. Some will be addressed in later
changes (e.g., member function pointers, some RTTI).
`dynamic_cast` in particular is handled by emitting an artificial
v-table pointer load (in a way that always authenticates it) before the
runtime call itself, as the runtime doesn't have enough information
today to properly authenticate it. Instead, the runtime is currently
expected to strip the v-table pointer.
---------
Co-authored-by: John McCall <rjmccall@apple.com>
Co-authored-by: Ahmed Bougacha <ahmed@bougacha.org>
Apple's ld supports alias_lists, described as
```
-alias_list filename
The specified filename contains a list of aliases. The symbol name and its alias are on one
line, separated by whitespace. Lines starting with # are ignored.
```
To handle this for installapi-produced TBD files, pass along the same
input and account for it in verification.
A zippered framework is a single framework that can be loaded in both
macOS and macatalyst processes. Broadly to InstallAPI, it means the same
interface can represent two separate platforms.
A dylib's symbol table does not distinguish between macOS/macCatalyst.
`InstallAPI` provides the ability for the tbd file to distinct
symbols between them.
The verifier handles this special logic by tracking all unavailable and
obsoleted APIs in this context and checking against those when
determining dylib symbols with no matching declaration.
* If there exists an available decl for either platform, do not warn.
* If there is no available decl, emit a diagnostic and print the source
location for both decls.
A few verification checks need to happen until all AST's have been
traversed, specifically for zippered framework checking. To keep source
location until that time valid, hold onto to references of
FrontendRecords + SourceManager.
* Capture reexported libraries, allowable clients, rpaths, shared cache
eligibility.
* Add support for select Xarch options.
* Add diagnostics related to capturing these options.
* Add support for verifying these attributes against what is encoded in
the dylib.
InstallAPI does not directly look at object files in the dylib for
verification. To help diagnose violations where a declaration is
undiscovered in headers, parse the dSYM and look up the source location
for symbols. Emitting out the source location with a diagnostic is
enough for some IDE's (e.g. Xcode) to have them map back to editable
source files.
Reported by Static Analyzer Tool:
In clang::installapi::InstallAPIVisitor::VisitFunctionDecl(clang::FunctionDecl
const *): Using the auto keyword without an & causes the copy of an
object of type DynTypedNode.
This reverts commit b7d8c6188986f62573b9516fe27fdd0c7df1aaf9. And
This reverts commit 2d40f179124f874aca4cf1145fdbc42fb8fb17f3.
It caused a build failure i'll need to reproduce.
` error: could not convert ‘Rule’ from ‘llvm::Regex’ to ‘llvm::Expected<llvm::Regex>’`
InstallAPI takes a json list of headers that is typically generated from
a build system like Xcode based on a project's attributes. Sometimes,
maintainers may want to alter this for tapi input. Using e.g.
`--extra-public-headers`, users can manipulate what headers will be used
for TBD file generation.
* This completes support for verifying every declaration found in a
header is discovered in the dylib. Diagnostics are reported for each
class for differences that are representable in TBD files.
* This patch also now captures unavailable attributes that depend on
target triples. This is needed for proper tbd file generation.
This adds basic support for calling the verifier on global declarations
that are expected to represent symbol exports. The driver now
exclusively uses this for knowing what symbols make up a TBD file.
Future patches will check against the dylib's symbol table.
Before it gets too unwieldy, add a common header for all MachO types
that are used across InstallAPI. Also, break up the types in
`InstallAPI/Frontend`. This both avoids circular dependencies and is
logically easier to maintain as more functionality gets added.
This includes capturing symbols for global variables, functions,
classes, and templated defintions. As pre-determing what symbols are
generated from C++ declarations can be non-trivial, InstallAPI only
parses select declarations for symbol generation when parsing c++.
For example, installapi only looks at explicit template instantiations
or full template specializations, instead of general function or class
templates, for symbol emittion.
* Include whether functions are inlinable as they impact whether to add
them into the tbd file and for future verification.
* Fix how clang arguments got passed along, previously spacing was
passed along to CC1 causing search path inputs to look non-existent.
* This patch introduces a container class, for holding records and
attributes only collectible from the clang frontend, which is a subclass
of `llvm::MachO::RecordsSlice`
* This also prunes out collecting declarations from headers that aren't
considered input to installapi.
* Uses these constructs for collecting global objective-c interfaces.
This patch takes in json files as input to determine that header files
to process, and in which order, to pass along for CC1 invocations. This
patch also includes an ASTVisitor to collect simple global variables.
Installapi has important distinctions when compared to the clang driver,
so much that, it doesn't make much sense to try to integrate into it.
This patch partially reverts the CC1 action & driver support to replace
with its own driver as a clang tool.
For distribution, we could use `LLVM_TOOL_LLVM_DRIVER_BUILD` mechanism
for integrating the functionality into clang such that the toolchain
size is less impacted.
This patch adds support for expected InstallAPI inputs. InstallAPI
accepts a well defined filelist of headers and how those headers
represent a single library.
InstallAPI captures header files to determine linkable symbols to then
compare against what was compiled in a binary dylib and generate TBD
files.
This introduces a basic outline of installapi as a clang driver option.
It captures relevant information as cc1 args, which are common arguments
already passed to the linker to encode into TBD file outputs. This is
effectively an upstream for what already exists as `tapi installapi` in
Xcode toolchains, but directly in Clang. This patch does not handle any
AST traversing on input yet.
InstallAPI is broadly an operation that takes a series of header files
that represent a single dynamic library and generates a TBD file out of
it which represents all the linkable symbols and necessary attributes
for statically linking in clients. It is the linkable object in all
Apple SDKs and when building dylibs in Xcode. `clang -installapi` also
will support verification where it compares all the information recorded
for the TBD files against the already built binary, to catch possible
mismatches like when a declaration is missing a definition for an
exported symbol.
