This commit handles the following types:
- clang::ExternalASTSource
- clang::TargetInfo
- clang::ASTContext
- clang::SourceManager
- clang::FileManager
Part of cleanup #151026
Handles clang::DiagnosticsEngine and clang::DiagnosticIDs.
For DiagnosticIDs, this mostly migrates from `new DiagnosticIDs` to
convenience method `DiagnosticIDs::create()`.
Part of cleanup https://github.com/llvm/llvm-project/issues/151026
This switches to `makeIntrusiveRefCnt<FileSystem>` where creating a new
object, and to passing/returning by `IntrusiveRefCntPtr<FileSystem>`
instead of `FileSystem*` or `FileSystem&`, when dealing with existing
objects.
Part of cleanup #151026.
This reverts commit e2a885537f11f8d9ced1c80c2c90069ab5adeb1d. Build failures were fixed right away and reverting the original commit without the fixes breaks the build again.
The `DiagnosticOptions` class is currently intrusively
reference-counted, which makes reasoning about its lifetime very
difficult in some cases. For example, `CompilerInvocation` owns the
`DiagnosticOptions` instance (wrapped in `llvm::IntrusiveRefCntPtr`) and
only exposes an accessor returning `DiagnosticOptions &`. One would
think this gives `CompilerInvocation` exclusive ownership of the object,
but that's not the case:
```c++
void shareOwnership(CompilerInvocation &CI) {
llvm::IntrusiveRefCntPtr<DiagnosticOptions> CoOwner = &CI.getDiagnosticOptions();
// ...
}
```
This is a perfectly valid pattern that is being actually used in the
codebase.
I would like to ensure the ownership of `DiagnosticOptions` by
`CompilerInvocation` is guaranteed to be exclusive. This can be
leveraged for a copy-on-write optimization later on. This PR changes
usages of `DiagnosticOptions` across `clang`, `clang-tools-extra` and
`lldb` to not be intrusively reference-counted.
With the introduction of tbd-v5 holding rpaths, the order in which those
attributes are passed to `clang-installapi` must be represented in tbd
files. Previously, all dylib attributes were stored in a
non-deterministic `StringMap`. Instead, hold them in a custom collection
with an underlying vector to continue supporting searching by attribute.
This makes the order of all diagnostics related to load command
comparisons stable.
This approach resolves errors when building with reverse-iteration.
This patch replaces:
llvm::copy(Src, std::back_inserter(Dst));
with:
llvm::append_range(Dst, Src);
for breavity.
One side benefit is that llvm::append_range eventually calls
llvm::SmallVector::reserve if Dst is of llvm::SmallVector.
Apologies for the large change, I looked for ways to break this up and
all of the ones I saw added real complexity. This change focuses on the
option's prefixed names and the array of prefixes. These are present in
every option and the dominant source of dynamic relocations for PIE or
PIC users of LLVM and Clang tooling. In some cases, 100s or 1000s of
them for the Clang driver which has a huge number of options.
This PR addresses this by building a string table and a prefixes table
that can be referenced with indices rather than pointers that require
dynamic relocations. This removes almost 7k dynmaic relocations from the
`clang` binary, roughly 8% of the remaining dynmaic relocations outside
of vtables. For busy-boxing use cases where many different option tables
are linked into the same binary, the savings add up a bit more.
The string table is a straightforward mechanism, but the prefixes
required some subtlety. They are encoded in a Pascal-string fashion with
a size followed by a sequence of offsets. This works relatively well for
the small realistic prefixes arrays in use.
Lots of code has to change in order to land this though: both all the
option library code has to be updated to use the string table and
prefixes table, and all the users of the options library have to be
updated to correctly instantiate the objects.
Some follow-up patches in the works to provide an abstraction for this
style of code, and to start using the same technique for some of the
other strings here now that the infrastructure is in place.
Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.
It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.
Some `FileManager` APIs still return `{File,Directory}Entry` instead of
the preferred `{File,Directory}EntryRef`. These are documented to be
deprecated, but don't have the attribute that warns on their usage. This
PR marks them as such with `LLVM_DEPRECATED()` and replaces their usage
with the recommended counterparts. NFCI.
Projects like libc use mutually exclusive macros to compile files
multiple times and then merge the result into the final library. For
installapi to accept these, we'd need to parse the same declarations in
different ways. This patch adds the basic pipelining for installapi to
create the correct TBD file.
* -Xproject allows: -fmodules, -fobjc-arc, fvisibility=hidden, prefix
headers
* -Xlabel allows: -D and -U settings
* Error on 'private' and 'public' labels -X<label>
* Xplatform allows: -iframework <path> This is to support the case where
zippered frameworks want to pass in iOSSupport search path.
Apple's ld supports alias_lists, described as
```
-alias_list filename
The specified filename contains a list of aliases. The symbol name and its alias are on one
line, separated by whitespace. Lines starting with # are ignored.
```
To handle this for installapi-produced TBD files, pass along the same
input and account for it in verification.
A zippered framework is a single framework that can be loaded in both
macOS and macatalyst processes. Broadly to InstallAPI, it means the same
interface can represent two separate platforms.
A dylib's symbol table does not distinguish between macOS/macCatalyst.
`InstallAPI` provides the ability for the tbd file to distinct
symbols between them.
The verifier handles this special logic by tracking all unavailable and
obsoleted APIs in this context and checking against those when
determining dylib symbols with no matching declaration.
* If there exists an available decl for either platform, do not warn.
* If there is no available decl, emit a diagnostic and print the source
location for both decls.
A few verification checks need to happen until all AST's have been
traversed, specifically for zippered framework checking. To keep source
location until that time valid, hold onto to references of
FrontendRecords + SourceManager.
* Capture reexported libraries, allowable clients, rpaths, shared cache
eligibility.
* Add support for select Xarch options.
* Add diagnostics related to capturing these options.
* Add support for verifying these attributes against what is encoded in
the dylib.
InstallAPI does not directly look at object files in the dylib for
verification. To help diagnose violations where a declaration is
undiscovered in headers, parse the dSYM and look up the source location
for symbols. Emitting out the source location with a diagnostic is
enough for some IDE's (e.g. Xcode) to have them map back to editable
source files.
Umbrella headers are a concept for Darwin-based libraries. They allow
framework authors to control the order in which their headers should be
parsed and allow clients to access available headers by including a
single header.
InstallAPI will attempt to find the umbrella based on the name of the
framework. Users can also specify this explicitly by using command line
options specifying the umbrella header by file path. There can be an
umbrella header per access level.
This reverts commit b7d8c6188986f62573b9516fe27fdd0c7df1aaf9. And
This reverts commit 2d40f179124f874aca4cf1145fdbc42fb8fb17f3.
It caused a build failure i'll need to reproduce.
` error: could not convert ‘Rule’ from ‘llvm::Regex’ to ‘llvm::Expected<llvm::Regex>’`
InstallAPI takes a json list of headers that is typically generated from
a build system like Xcode based on a project's attributes. Sometimes,
maintainers may want to alter this for tapi input. Using e.g.
`--extra-public-headers`, users can manipulate what headers will be used
for TBD file generation.
* This completes support for verifying every declaration found in a
header is discovered in the dylib. Diagnostics are reported for each
class for differences that are representable in TBD files.
* This patch also now captures unavailable attributes that depend on
target triples. This is needed for proper tbd file generation.
This adds basic support for calling the verifier on global declarations
that are expected to represent symbol exports. The driver now
exclusively uses this for knowing what symbols make up a TBD file.
Future patches will check against the dylib's symbol table.
* A lot of `tapi installapi` options are already shared with clang, but
not all. This patch handles installapi-specific options by filtering for
them in the initial argv input, then passing the rest to the clang
driver.
* Installapi not only generates a text file but also reports to library
developers when there are inconsistencies between an interface and its
implementation. To allow this, add support for reporting installapi
diagnostics. This will be leveraged in the verifier service.
Before it gets too unwieldy, add a common header for all MachO types
that are used across InstallAPI. Also, break up the types in
`InstallAPI/Frontend`. This both avoids circular dependencies and is
logically easier to maintain as more functionality gets added.
This includes capturing symbols for global variables, functions,
classes, and templated defintions. As pre-determing what symbols are
generated from C++ declarations can be non-trivial, InstallAPI only
parses select declarations for symbol generation when parsing c++.
For example, installapi only looks at explicit template instantiations
or full template specializations, instead of general function or class
templates, for symbol emittion.
* Include whether functions are inlinable as they impact whether to add
them into the tbd file and for future verification.
* Fix how clang arguments got passed along, previously spacing was
passed along to CC1 causing search path inputs to look non-existent.
* This patch introduces a container class, for holding records and
attributes only collectible from the clang frontend, which is a subclass
of `llvm::MachO::RecordsSlice`
* This also prunes out collecting declarations from headers that aren't
considered input to installapi.
* Uses these constructs for collecting global objective-c interfaces.
This patch takes in json files as input to determine that header files
to process, and in which order, to pass along for CC1 invocations. This
patch also includes an ASTVisitor to collect simple global variables.
