This reverts commit e2a885537f11f8d9ced1c80c2c90069ab5adeb1d. Build failures were fixed right away and reverting the original commit without the fixes breaks the build again.
The `DiagnosticOptions` class is currently intrusively
reference-counted, which makes reasoning about its lifetime very
difficult in some cases. For example, `CompilerInvocation` owns the
`DiagnosticOptions` instance (wrapped in `llvm::IntrusiveRefCntPtr`) and
only exposes an accessor returning `DiagnosticOptions &`. One would
think this gives `CompilerInvocation` exclusive ownership of the object,
but that's not the case:
```c++
void shareOwnership(CompilerInvocation &CI) {
llvm::IntrusiveRefCntPtr<DiagnosticOptions> CoOwner = &CI.getDiagnosticOptions();
// ...
}
```
This is a perfectly valid pattern that is being actually used in the
codebase.
I would like to ensure the ownership of `DiagnosticOptions` by
`CompilerInvocation` is guaranteed to be exclusive. This can be
leveraged for a copy-on-write optimization later on. This PR changes
usages of `DiagnosticOptions` across `clang`, `clang-tools-extra` and
`lldb` to not be intrusively reference-counted.
This reverts commit 7a242387c950c7060143da6da0e6fb91f36bb458. Even after 175f8a44, the Modules/fmodules-validate-once-per-build-session.c test is not fixed on the clang-armv8-quick build bot. (Failure occurs on line 114.)
This reverts commit 18b885f66babff3a10451bc811ffc077d61ed8ee, effectively reapplying #139987. This commit fixes unit tests (for example ASTUnitTest.SaveLoadPreservesLangOptionsInPrintingPolicy) where the `ASTUnit::ModCache` pointer dereferenced within `ASTUnit::serialize()` was null. This commit makes sure each factory function does initialize `ASTUnit::ModCache`.
Since getLastArgValue returns StringRef, and the constructor of
SmallString accepts StringRef, we do not need to go through a
temporary instance of std::string.
Fixes#139375
Clang expects command line source locations to be provided using 1-based
indexing.
Currently, Clang does not reject zero as invalid argument for column or
line number, which can cause Clang to crash.
This commit extends validation in `ParsedSourceLocation::FromString` to
only accept (unsinged) non-zero integers.
Timestamps are an implementation detail of the cross-process module
cache implementation. This PR hides it from the `ModuleCache` API, which
simplifies the in-process implementation.
There are checks in clang codebase that determine the type of source
file, associated with a given location - specifically, if it is an
ordonary file or comes from sources like command-line options or a
built-in definitions. These checks often rely on calls to
`getPresumedLoc`, which is relatively expensive. In certain cases, these
checks are combined, leading to repeated calculations of the costly
function negatively affecting compile time.
This change tries to optimize such checks. It must fix compile time
regression introduced in
https://github.com/llvm/llvm-project/pull/137306/.
---------
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
Force-validation of user headers was implemented in acb803e8 to deal
with files changing during build. The dependency scanner guarantees an
immutable file system during single build session, so the validation is
unnecessary. (We don't hit the disk too often due to the caching VFS,
but even avoiding going to the cache and deserializing the input files
makes sense.)
Move the Darwin framework search path logic from
InitHeaderSearch::AddDefaultIncludePaths to
DarwinClang::AddClangSystemIncludeArgs. Add a new -internal-iframework
cc1 argument to support the tool chain adding these paths.
Now that the tool chain is adding search paths via cc1 flag, they're
only added if they exist, so the Preprocessor/cuda-macos-includes.cu
test is no longer relevant.
Change Driver/driverkit-path.c and Driver/darwin-subframeworks.c to do
-### style testing similar to the darwin-header-search and
darwin-embedded-search-paths tests. Rename darwin-subframeworks.c to
darwin-framework-search-paths.c and have it test all framework search
paths, not just SubFrameworks.
Add a unit test to validate that the myriad of search path flags result
in the expected search path list.
Fixes https://github.com/llvm/llvm-project/issues/75638
@fmayer introduced '-mllvm -array-bounds-pseudofn'
(https://github.com/llvm/llvm-project/pull/128977/) to make it easier to
see why crashes occurred, and to estimate with a profiler the cycles
spent on these array-bounds checks. This functionality could be usefully
generalized to other checks in future work.
This patch adds the plumbing for -fsanitize-annotate-debug-info, and
connects it to the existing array-bounds-pseudo-fn functionality i.e.,
-fsanitize-annotate-debug-info=array-bounds can be used as a replacement
for '-mllvm -array-bounds-pseudofn', though we do not yet delete the
latter.
Note: we replaced '-mllvm -array-bounds-pseudofn' in
clang/test/CodeGen/bounds-checking-debuginfo.c, because adding test
cases would modify the line numbers in the test assertions, and
therefore obscure that the test output is the same between '-mllvm
-array-bounds-pseudofn' and -fsanitize-annotate-debug-info=array-bounds.
The allocation interposition mode had a number of issues, which are
primarily addressed in the library component via
<https://github.com/ROCm/rocThrust/pull/543>. However, it is necessary
to interpose some additional symbols, which this patch does.
Furthermore, to implement this in a compatible way, we guard the new
implementation under a V1 macro, which is defined in addition to the
existing `__HIPSTDPAR_INTERPOSE_ALLOC__` one.
Instead of eagerly populating the `clang::ModuleMap` when looking up a
module by name, this patch changes `HeaderSearch` to only load the
modules that are actually used.
This introduces `ModuleMap::findOrLoadModule` which will load modules
from parsed but not loaded module maps. This cannot be used anywhere
that the module loading code calls into as it can create infinite
recursion.
This currently just reparses module maps when looking up a module by
header. This is fine as redeclarations are allowed from the same file,
but future patches will also make looking up a module by header lazy.
This patch changes the shadow.m test to use explicitly built modules and
`#import`. This test and the shadow feature are very brittle and do not
work in general. The test relied on pcm files being left behind by prior
failing clang invocations that were then reused by the last invocation.
If you clean the cache then the last invocation will always fail. This
is because the input module map and the `-fmodule-map-file=` module map
are parsed in the same module scope, and `-fmodule-map-file=` is
forwarded to implicit module builds. That means you are guaranteed to
hit a module redeclaration error if the TU actually imports the module
it is trying to shadow.
This patch changes when we load A2's module map to after the `A` module
has been loaded, which sets the `IsFromModuleFile` bit on `A`. This
means that A2's `A` is skipped entirely instead of creating a shadow
module, and we get textual inclusion. It is possible to construct a case
where this would happen before this patch too.
An upcoming patch in this series will rework shadowing to work in the
general case, but that's only possible once header -> module lookup is
lazy too.
This adds
- The parsing of `trivially_relocatable_if_eligible`,
`replaceable_if_eligible` keywords
- `__builtin_trivially_relocate`, implemented in terms of memmove. In
the future this should
- Add the appropriate start/end lifetime markers that llvm does not have
(`start_lifetime_as`)
- Add support for ptrauth when that's upstreamed
- the `__builtin_is_cpp_trivially_relocatable` and
`__builtin_is_replaceable` traits
Fixes#127609
This PR makes it so that `CompilerInvocation` needs to be provided to
`CompilerInstance` on construction. There are a couple of benefits in my
view:
* Making it impossible to mis-use some `CompilerInstance` APIs. For
example there are cases, where `createDiagnostics()` was called before
`setInvocation()`, causing the `DiagnosticEngine` to use the
default-constructed `DiagnosticOptions` instead of the intended ones.
* This shrinks `CompilerInstance`'s state space.
* This makes it possible to access **the** invocation in
`CompilerInstance`'s constructor (to be used in a follow-up).
Reland https://github.com/llvm/llvm-project/pull/133173
Clang spawns a new thread to avoid running out of stack space. This can
make debugging and performance analysis more difficult as how the
threads are connected is difficult to recover.
This patch introduces `runOnNewStack` and applies it in Clang. On
platforms that have good support for it this allocates a new stack and
moves to it using assembly. Doing split stacks like this actually runs
on most platforms, but many debuggers and unwinders reject the large or
backwards stack offsets that occur. Apple platforms and tools are known
to support this, so this only enables it there for now.
This patch adds a new flag, -ftime-report-json, which outputs the same
information as -ftime-report but as JSON instead of -ftime-report's
pretty printed format.
This adds a new filtering option to be used along with the
-header-include-format=json option. The existing "only-direct-system"
filtering option is missing some things:
- It does not show module imports.
- It does not show includes of non-system headers. This new
"direct-per-file" filtering has a separate entry in the JSON output for
each non-system source file, showing the direct includes and imports
from that file. You can use this to see uses of non-system headers, and
also find the paths through non-system headers that lead up to module
imports and system headers. Modules are identified here by their
modulemap files.
Fixes#132692.
`clang -cc1` crashes when generating a module interface with
`emit-module-interface` or `emit-reduced-module-interface` using an
input file which is already a precompiled module (`.pcm`) file.
This commit adds validation for the input file format and Clang will now
emit an error message instead of crashing.
This PR makes `CompilerInvocation` the sole owner of the
`AnalyzerOptions` instance. Clients can no longer become co-owners by
doing something like this:
```c++
void shareOwnership(CompilerInvocation &CI) {
IntrusiveRefCntPtr Shared = &CI.getAnalyzerOpts();
}
```
The motivation for this is given here:
https://github.com/llvm/llvm-project/pull/133467#issuecomment-2762065443
This PR hides the reference-counted pointer that holds `TargetOptions`
from the public API of `CompilerInvocation`. This gives
`CompilerInvocation` an exclusive control over the lifetime of this
member, which will eventually be leveraged to implement a copy-on-write
behavior.
There are two clients that currently share ownership of that pointer:
* `TargetInfo` - This was refactored to hold a non-owning reference to
`TargetOptions`. The options object is typically owned by the
`CompilerInvocation` or by the new `CompilerInstance::AuxTargetOpts` for
the auxiliary target. This needed a bit of care in `ASTUnit::Parse()` to
keep the `CompilerInvocation` alive.
* `clangd::PreambleData` - This was refactored to exclusively own the
`TargetOptions` that get moved out of the `CompilerInvocation`.
This PR makes another piece of the
`CompilerInstance::cloneForModuleCompile()` result thread-safe: the
module build stack. This data structure is used to detect cyclic
dependencies between modules. The problem is that it uses
`FullSourceLoc` which refers to the `SourceManager` of the parent
`CompilerInstance`: if two threads happen to execute `CompilerInstance`s
cloned from the same parent concurrently, and both discover a dependency
cycle, they may concurrently access the parent `SourceManager` when
emitting the diagnostic, creating a data race.
In this PR, we prevent this by keeping the stack empty and moving the
responsibility of cycle detection to the client. The client can recreate
the same module build stack externally and ensure thread-safety by
enforcing mutual exclusion.
If you specify -header-include-format=json, the only filtering option
currently supported is -header-include-filtering=only-direct-system. If
you specify some other filtering option, Clang gives an error message.
But, if you do not specify the filtering option at all, Clang crashes
when producing the error message, since it tries to get the value of the
unused option.
This PR fixes two issues in one go:
1. The dependency directives getter (a `std::function`) was being stored
in `PreprocessorOptions`. This goes against the principle where the
options classes are supposed to be value-objects representing the `-cc1`
command line arguments. This is fixed by moving the getter directly to
`CompilerInstance` and propagating it explicitly.
2. The getter was capturing the `ScanInstance` VFS. That's fine in
synchronous implicit module builds where the same VFS instance is used
throughout, but breaks down once you try to build modules asynchronously
(which forces the use of separate VFS instances). This is fixed by
explicitly passing a `FileManager` into the getter and extracting the
right instance of the scanning VFS out of it.
The `DiagnosticConsumer` interface is not thread-safe. To enable
thread-safety of `CompilerInstance` objects cloned from the same parent,
this PR allows passing an explicit `DiagnosticConsumer` to
`cloneForModuleCompile()`. This will be used from the dependency
scanner.
The `llvm::vfs::FileSystem` interface makes no promises around
thread-safety. To enable making `CompilerInstance` thread-safe, this PR
allows passing an explicit VFS to `cloneForModuleCompile()`. This will
be used from the dependency scanner.
This PR reland https://github.com/llvm/llvm-project/pull/135808, fixed
some missed changes in LLDB.
I found this issue when I working on
https://github.com/llvm/llvm-project/pull/107168.
Currently we have many similiar data structures like:
- std::pair<IdentifierInfo *, SourceLocation>.
- Element type of ModuleIdPath.
- IdentifierLocPair.
- IdentifierLoc.
This PR unify these data structures to IdentifierLoc, moved
IdentifierLoc definition to SourceLocation.h, and deleted other similer
data structures.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
This PR will fix the following lit failure seeing on z/OS and most likely on Windows:
`FAIL: Clang :: Frontend/dump-minimization-hints.cpp`
Without `OF_TextWithCRLF` flag, a file is treated as binary and is read improperly, at least on z/OS.
Reverts llvm/llvm-project#135808
Example from the LLDB macOS CI:
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/24084/execution/node/54/log/?consoleFull
```
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangModulesDeclVendor.cpp:360:49: error: no viable conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'clang::ModuleIdPath' (aka 'ArrayRef<IdentifierLoc>')
clang::Module *top_level_module = DoGetModule(clang_path.front(), false);
^~~~~~~~~~~~~~~~~~
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:41:40: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'const llvm::ArrayRef<clang::IdentifierLoc> &' for 1st argument
class LLVM_GSL_POINTER [[nodiscard]] ArrayRef {
^
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:41:40: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'llvm::ArrayRef<clang::IdentifierLoc> &&' for 1st argument
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:70:18: note: candidate constructor not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'std::nullopt_t' for 1st argument
/*implicit*/ ArrayRef(std::nullopt_t) {}
```
I found this issue when I working on
https://github.com/llvm/llvm-project/pull/107168.
Currently we have many similiar data structures like:
- `std::pair<IdentifierInfo *, SourceLocation>`.
- Element type of `ModuleIdPath`.
- `IdentifierLocPair`.
- `IdentifierLoc`.
This PR unify these data structures to `IdentifierLoc`, moved
`IdentifierLoc` definition to SourceLocation.h, and deleted other
similer data structures.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
…thread to get more stack space (#133173)"
This change breaks the Clang build on Mac AArch64.
This reverts commit d0c973a7a0149db3b71767d4c5a20a31e6a8ed5b. This
reverts commit 429a84f8a4bf559f43f50072747ef49d3e3b2cf1. This reverts
commit 4f64c80d5a23c244f942193e58ecac666c173308.
Clang spawns a new thread to avoid running out of stack space. This can
make debugging and performance analysis more difficult as how the
threads are connected is difficult to recover.
This patch introduces `runOnNewStack` and applies it in Clang. On
platforms that have good support for it this allocates a new stack and
moves to it using assembly. Doing split stacks like this actually runs
on most platforms, but many debuggers and unwinders reject the large or
backwards stack offsets that occur. Apple platforms and tools are known
to support this, so this only enables it there for now.
This PR makes some progress towards making it possible to create clones
of `CompilerInstance` that are independent of each other and can be used
in a multi-threaded setting. This PR tackles
`CompilerInstance::FailedModules`, makes it a value-type instead of a
mutable shared pointer, and adds explicit copies & moves where
appropriate.
Besides that change, this PR also turns two previously free functions
with internal linkage into member functions of `CompilerInstance`, which
makes it possible to reduce the public API of that class that relates to
`FailedModules`. This reduces some API churn that was necessary for each
new member of `CompilerInstance` that needs to be cloned.
This PR exposes `cloneForModuleCompile()` as a public `CompilerInstance`
member function. This will be eventually used in the dependency scanner
to customize implicit module builds.
This PR extracts the creation of `CompilerInstance` for compiling an
implicitly-discovered module out of `compileModuleImpl()` into its own
separate function and passes it into `compileModuleImpl()` from the
outside. This makes the instance creation logic reusable (useful for my
experiments) and also simplifies the API, removing the `PreBuildStep`
and `PostBuildStep` hooks from `compileModuleImpl()`.
This PR implements a CC1 flag `-dump-minimization-hints`.
The flag allows to specify a file path to dump ranges of deserialized
declarations in `ASTReader`. Example usage:
```
clang -Xclang=-dump-minimization-hints=/tmp/decls -c file.cc -o file.o
```
Example output:
```
// /tmp/decls
{
"required_ranges": [
{
"file": "foo.h",
"range": [
{
"from": {
"line": 26,
"column": 1
},
"to": {
"line": 27,
"column": 77
}
}
]
},
{
"file": "bar.h",
"range": [
{
"from": {
"line": 30,
"column": 1
},
"to": {
"line": 35,
"column": 1
}
},
{
"from": {
"line": 92,
"column": 1
},
"to": {
"line": 95,
"column": 1
}
}
]
}
]
}
```
Specifying the flag creates an instance of
`DeserializedDeclsSourceRangePrinter`, which dumps ranges of deserialized
declarations to aid debugging and bug minimization (we use is as input to [C-Vise](https://github.com/emaxx-google/cvise/tree/multifile-hints).
Required ranges are computed from source ranges of Decls.
`TranslationUnitDecl`, `LinkageSpecDecl` and `NamespaceDecl` are ignored
for the sake of this PR.
Technical details:
* `DeserializedDeclsSourceRangePrinter` implements `ASTConsumer` and
`ASTDeserializationListener`, so that an object of
`DeserializedDeclsSourceRangePrinter` registers as its own listener.
* `ASTDeserializationListener` interface provides the `DeclRead`
callback that we use to collect the deserialized Decls.
Printing or otherwise processing them as this point is dangerous, since
that could trigger additional deserialization and crash compilation.
* The collected Decls are processed in `HandleTranslationUnit` method of
`ASTConsumer`. This is a safe point, since we know that by this point
all the Decls needed by the compiler frontend have been deserialized.
* In case our processing causes further deserialization, `DeclRead` from
the listener might be called again. However, at that point we don't
accept any more Decls for processing.
