This patch adds logic to canonicalize `-include-pch`'s input in the
frontend. This way, the `ASTWriter` always serializes the canonicalized
path to the included pch file whether the input is an absolute path or a
relative path.
Fixes rdar://168596546.
Previously, `-fmodules-single-module-parse-mode` only prevented module
compilation/loading when initiated from an `#include` or `#import`
directive. This PR does the same for `@import`, `#pragma clang module
import` and `#pragma clang module load`. This is done by sinking the
logic down into `CompilerInstance::loadModule()`.
Close https://github.com/llvm/llvm-project/issues/177385
The root cause of the problem is, when we decide to mangle a lamdba in a
module interface while the ManglingContextDecl is nullptr, we didn't
update ManglingContextDecl. So that the following use of
ManglingContextDecl is an invalid value.
Every time DarwinSDKInfo reads a new key out of SDKSettings, a boatload
of test SDKSettings files need to be updated across several repositories
and forks and branches. It’s tedious to be careful to update those with
real values so that the tests are properly regression testing older
SDKs. It’s important to be careful so that the tests are accurate, e.g.
to prevent the scenario where DarwinSDKInfo starts reading a new key out
of SDKSettings and assumes that it’s always available everywhere, when
in reality it was only added a few releases ago and will break with
older SDKs. If the test SDKSettings files continue to be updated ad hoc,
it’s going to be really easy to copy/paste a default value everywhere,
and then clients will see incorrect behaviors with the real SDKs, or
even compiler crashes if the key is unconditionally read. Preemptively
add all of the maybe-possibly-compiler relevant keys to the test
SDKSettings files from the real SDKs so that the test files are an
accurate representation and shouldn't need to be touched in the future.
Where the test SDKSettings have intentionally doctored data, add a
Comments key explaining what is changed from the real SDK, and alter the
SDK name with a tag indicating the change.
The patch reapply https://github.com/llvm/llvm-project/pull/173130.
This patch implement the following papers:
[P1857R3 Modules Dependency Discovery](https://wg21.link/p1857r3).
[P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1).
[CWG2947](https://cplusplus.github.io/CWG/issues/2947.html).
At the start of phase 4 an import or module token is treated as starting
a directive and are converted to their respective keywords iff:
- After skipping horizontal whitespace are
- at the start of a logical line, or
- preceded by an export at the start of the logical line.
- Are followed by an identifier pp token (before macro expansion), or
- <, ", or : (but not ::) pp tokens for import, or
- ; for module
Otherwise the token is treated as an identifier.
Additionally:
- The entire import or module directive (including the closing ;) must
be on a single logical line and for module must not come from an
#include.
- The expansion of macros must not result in an import or module
directive introducer that was not there prior to macro expansion.
- A module directive may only appear as the first preprocessing tokens
in a file (excluding the global module fragment.)
- Preprocessor conditionals shall not span a module declaration.
After this patch, we handle C++ module-import and module-declaration as
a real pp-directive in preprocessor. Additionally, we refactor module
name lexing, remove the complex state machine and read full module name
during module/import directive handling. Possibly we can introduce a
tok::annot_module_name token in the future, avoid duplicatly parsing
module name in both preprocessor and parser, but it's makes error
recovery much diffcult(eg. import a; import b; in same line).
This patch also introduce 2 new keyword `__preprocessed_module` and
`__preprocessed_import`. These 2 keyword was generated during `-E` mode.
This is useful to avoid confusion with `module` and `import` keyword in
preprocessed output:
```cpp
export module m;
struct import {};
#define EMPTY
EMPTY import foo;
```
Fixes https://github.com/llvm/llvm-project/issues/54047
The previous patch has an use-after-free issue in
Lexer::LexTokenInternal function. Since C++20, the `export`, `import`
and `module` identifiers may be an introducer of a C++ module
declaration/importing directive, and the directive will handled in
`LexIdentifierContinue`. Unfortunately, the EOF may be encountered in
`LexIdentifierContinue` and `CurLexer` might be destructed in
`HandleEndOfFile`, If the code after `LexIdentifierContinue` try to
access `LangOps` or other class members in this Lexer, it will hit
undefined behavior.
This patch also fix the use-after-free issue in Lexer by introduce a
mechanism to delay the destruction of `CurLexer` in `Preprocessor`
class.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
…nd a module with config macros (#174034)"
This reverts commit 71925cbdf80d4fc88ef844e726f89aa71c64bceb.
The test case is causing test failures on downstream Swift forks. This
PR reverts the commit to avoid failing the test case on Linux.
rdar://168174041
Reverts #172430 (c560f1cf03aa06c0bdd00c5a9b558c16d882af6f).
Causes some failures like `error: static assertion expression is not an
integral constant expression` and `error: substitution into constraint
expression resulted in a non-constant expression` in modules builds.
Repro TBD.
This PR unifies the terminology for:
* "context hash" - previously ambiguously referred to as "module hash"
or as overly specific "module context hash"
* "specific module cache path" - previously referred to as just "module
cache path" - hard to distinguish from the command-line-provided module
cache path without the context hash
NFCI
The search path prefix is really a property of the SDK, and not the
target triple. The target is just being used as a proxy for the SDK.
That's problematic when the SDK being used doesn't match the target
assumption, and the prefix should be taken from the SDK rather than hard
coded.
Parse the SupportedTargets, which is what holds the platform prefix, in
DarwinSDKInfo. SupportedTargets contains an entry for the SDK's
canonical name, and that entry holds a valid OS value so use that
instead of the hard coded map. Include the environment which is also
relevant in some situations, and the vendor and object format in case
they're useful later. Skip architectures because they typically aren't
used for doing platform matching.
When a PCH is compiled with macro definitions on the command line, such
as `-DCONFIG1`, an unexpected warning can occur if the macro definitions
happen to belong to an imported module's config macros. The warning may
look like the following:
```
definition of configuration macro 'CONFIG1' has no effect on the import of 'Mod1'; pass '-DCONFIG1=...' on the command line to configure the module
```
while `-DCONFIG1` is clearly on the command line when `clang` compiles
the source that uses the PCH and the module.
The reason this can happen is a combination of two things:
1. The logic that checks for config macros is not aware of any command
line macros passed through the PCH
([here](7976ac9900/clang/lib/Frontend/CompilerInstance.cpp (L1562))).
2. `clang` _replaces_ the predefined macros on the command line with the
predefined macros from the PCH, which does not include any builtins
([here](7976ac9900/clang/lib/Frontend/CompilerInstance.cpp (L679))).
This PR teaches the preprocessor to recognize the command line macro
definitions passed transitively through the PCH, so that the error check
does not miss these definitions by mistake. The config macro itself
works fine, and it is only the error check that needs fixing.
rdar://95261458
PCHs (but also modules generated from several implicit invocations like
swiftc) previously reported a confusing diagnostic about module caches
being mismatched by subdir. This is an implementation detail of the
module machinery, and not very useful to the end user. Instead, report
this case as a configuration mismatch when the compiler can confirm the
module cache was passed the same between the current TU & previously
compiled products.
Ideally, each argument that could result in this error would be uniquely
reported (e.g., O3), but as a starting point, providing something more
general is strictly better than pointing the user to the module cache.
This patch also includes NFCs for renaming variable names from Module to
AST and formatting cleanup in related areas.
resolves: rdar://167453135
Now that the internal shell is enabled everywhere by default, the shell
feature makes little sense and essentially only implies windows is
unsupported. These tests all already require specific POSIX systems, so
the feature is no-op. Remove them to eventually remove the shell feature
from lit.
This PR implement the following papers:
[P1857R3 Modules Dependency Discovery](https://wg21.link/p1857r3).
[P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1).
[CWG2947](https://cplusplus.github.io/CWG/issues/2947.html).
At the start of phase 4 an import or module token is treated as starting
a directive and are converted to their respective keywords iff:
- After skipping horizontal whitespace are
- at the start of a logical line, or
- preceded by an export at the start of the logical line.
- Are followed by an identifier pp token (before macro expansion), or
- <, ", or : (but not ::) pp tokens for import, or
- ; for module
Otherwise the token is treated as an identifier.
Additionally:
- The entire import or module directive (including the closing ;) must
be on a single logical line and for module must not come from an
#include.
- The expansion of macros must not result in an import or module
directive introducer that was not there prior to macro expansion.
- A module directive may only appear as the first preprocessing tokens
in a file (excluding the global module fragment.)
- Preprocessor conditionals shall not span a module declaration.
After this patch, we handle C++ module-import and module-declaration as
a real pp-directive in preprocessor. Additionally, we refactor module
name lexing, remove the complex state machine and read full module name
during module/import directive handling. Possibly we can introduce a
tok::annot_module_name token in the future, avoid duplicatly parsing
module name in both preprocessor and parser, but it's makes error
recovery much diffcult(eg. import a; import b; in same line).
This patch also introduce 2 new keyword `__preprocessed_module` and
`__preprocessed_import`. These 2 keyword was generated during `-E` mode.
This is useful to avoid confusion with `module` and `import` keyword in
preprocessed output:
```cpp
export module m;
struct import {};
#define EMPTY
EMPTY import foo;
```
Fixes https://github.com/llvm/llvm-project/issues/54047
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
Signed-off-by: Wang, Yihan <yronglin777@gmail.com>
This PR introduces new single-module preprocessing mode. It is very
similar to single-file-parse mode, but has the following differences:
* Single-file mode skips over all inclusion directives, while the
single-module mode skips only over import directives and does resolve
textual inclusion directives.
* Single-file mode enters all branches of a conditional directive with
an undefined identifier, while the single-module enters none of them.
This will be used from the dependency scanner to quickly discover a
subset of modular dependencies of a TU/module. The dependencies aren't
being imported in this mode, but the file-inclusion preprocessor
callback does get invoked.
This reverts commit
54a4da9df6.
MSVC supports an extension allowing to delete an array of objects via
pointer whose static type doesn't match its dynamic type. This is done
via generation of special destructors - vector deleting destructors.
MSVC's virtual tables always contain a pointer to the vector deleting
destructor for classes with virtual destructors, so not having this
extension implemented causes clang to generate code that is not
compatible with the code generated by MSVC, because clang always puts a
pointer to a scalar deleting destructor to the vtable. As a bonus the
deletion of an array of polymorphic object will work just like it does
with MSVC - no memory leaks and correct destructors are called.
This patch will cause clang to emit code that is compatible with code
produced by MSVC but not compatible with code produced with clang of
older versions, so the new behavior can be disabled via passing
-fclang-abi-compat=21 (or lower).
Fixes https://github.com/llvm/llvm-project/issues/19772
Fix the propagation added in commit 0d490ae55f to include all redecls,
not only previous ones. This fixes another instance of the assertion
"Cannot get layout of forward declarations" in getASTRecordLayout().
Kudos to Alexander Kornienko for providing an initial version of the
reproducer that I further simplified.
Fixes#170084
Close https://github.com/llvm/llvm-project/issues/170235
The cause of the issue is it didn't check friendness for decls
in ordinary namespace if it isn't visible.
It is fine for code before modules, since everything is visible.
But it is not true after modules came in. This patch adjusts this.
Note that this doesn't change the control flow for non-modules codes,
as the decls in ordinary namespace is always visible then it won't never
fall in following friendness check.
In debug-info we soon have the need to print names using the full scope
of the entity (see discussion in
https://github.com/llvm/llvm-project/pull/159592). Particularly, when a
structure is scoped inside a function, we'd like to emit the name as
`func()::foo`. `CGDebugInfo` uses the `TypePrinter` to print type names
into debug-info. However, `TypePrinter` stops (and ignores)
`DeclContext`s that are functions. I.e., it would just print `foo`.
Ideally it would behave the same way `printNestedNameSpecifier` does.
The FIXME in
47c1aa4cef/clang/lib/AST/TypePrinter.cpp (L1520-L1521)
motivated this patch.
See https://github.com/llvm/llvm-project/pull/168533 for how this will
be used by `CGDebugInfo`. The plan is to introduce a new
`PrintingPolicy` that prints anonymous entities using their full scope
(including function/anonymous scopes) and the mangling number.
MSVC supports an extension allowing to delete an array of objects via
pointer whose static type doesn't match its dynamic type. This is done
via generation of special destructors - vector deleting destructors.
MSVC's virtual tables always contain a pointer to the vector deleting
destructor for classes with virtual destructors, so not having this
extension implemented causes clang to generate code that is not
compatible with the code generated by MSVC, because clang always puts a
pointer to a scalar deleting destructor to the vtable. As a bonus the
deletion of an array of polymorphic object will work just like it does
with MSVC - no memory leaks and correct destructors are called.
This patch will cause clang to emit code that is compatible with code
produced by MSVC but not compatible with code produced with clang of
older versions, so the new behavior can be disabled via passing
-fclang-abi-compat=21 (or lower).
This is yet another attempt to land vector deleting destructors support
originally implemented by
https://github.com/llvm/llvm-project/pull/133451.
This PR contains fixes for issues reported in the original PR as well as
fixes for issues related to operator delete[] search reported in several
issues like
https://github.com/llvm/llvm-project/pull/133950#issuecomment-2787510484https://github.com/llvm/llvm-project/issues/134265
Fixes https://github.com/llvm/llvm-project/issues/19772
Similar to previous no transitive changes to decls, types, identifiers
and source locations (
https://github.com/llvm/llvm-project/pull/92083https://github.com/llvm/llvm-project/pull/92085https://github.com/llvm/llvm-project/pull/92511https://github.com/llvm/llvm-project/pull/86912
)
This patch does the same thing for MacroID and PreprocessedEntityID.
---
### Some background
Previously we record different IDs linearly. That is, when writing a
module, if we have 17 decls in imported modules, the ID of decls in the
module will start from 18. This makes the contents of the BMI changes if
the we add/remove any decls, types, identifiers and source locations in
the imported modules.
This makes it hard for us to reduce recompilations with modules. We want
to skip recompilations as we think the modules can help us to remove
fake dependencies. This can be done by split the ID into <ModuleIndex,
LocalIndex> pairs.
This is ALREADY done for several different ID above. We call it
non-casacading changes
(https://clang.llvm.org/docs/StandardCPlusPlusModules.html#experimental-non-cascading-changes).
Our internal users have already used this feature and it works well for
years.
Now we want to extend this to MacroID and PreprocessedEntityID. This is
helpful for us in the downstream as we allowed named modules to export
macros. But I believe this is also helpful for header-like modules if
you'd like to explore the area.
And also I think this is a nice cleanup too.
---
Given the use of MacroID and PreprocessedEntityID are not as complicated
as other IDs in the above series, I feel the patch itself should be
good. I hope the vendors can test the patch to make sure it won't affect
existing users.
Close https://github.com/llvm/llvm-project/issues/166068
The cause of the problem is that we would import initializers and
pending implicit instantiations from other named module. This is very
bad and it may waste a lot of time.
And we didn't observe it as the weak symbols can live together and the
strong symbols would be removed by other mechanism. So we didn't observe
the bad behavior for a long time. But it indeeds waste compilation time.
This PR implements `CompilerInstanceWithContext` to improve by-name
dependency queries.
Cases exist where we query the dependency of different names, with
otherwise identical working directory and compile command line inputs.
In these cases, we can create one `CompilerInstance`, whose lifetime is
the same as the dependency scanning worker, and reuse the same compiler
instance to complete the queries. This way we reduce the amount of
header search we need to perform per query, since the already completed
header search results are cached in the compiler instance.
Using a microbenchmark on a prototype of this implementation, we are
seeing a scanning performance improvement of about 20%. The
microbenchmark scans a Swift file that imports everything importable.
When measuring against a set of internal project builds, the geo mean of
total build time improvement is around 1.02x to 1.04x depending on
whether the module caches are populated or not.
Part of work for rdar://136303612.
Fixes#165445.
Fixes a crash when `ASTWriter::GenerateNameLookupTable` processes enum
constants from C++20 header units.
The special handling for enum constants, introduced in fccc6ee, doesn't
account for declarations whose owning module is a C++20 header unit. It
calls `isNamedModule()` on the result of
`getTopLevelOwningNamedModule()`, which returns null for header units,
causing a null pointer dereference.
These were disabled when adjusting tests to work with the internal shell
because the implementation on these systems of env did not support the
-u option. Now that we have switched to the internal shell and env -u is
implemented internally, these tests should work again.
In C23 mode, both float.h and math.h are required to define INIFINITY
and NAN. However, with clang modules, there needs to be a single owner
for the declarations. Let that be float.h since that's the suggested
true owner in Annex F.2.2 of ISO/IEC 9899:2024, and introduce
`__need_infinity_nan` so that math.h can pick up the compiler
definitions.
PR https://github.com/llvm/llvm-project/pull/150123 changed how we
normalize the modules cache path. Unfortunately, empty path would get
normalized to the current working directory. This means that even
explicitly-built PCMs that don't rely on the CWD now embed it, leading
to surprising behavior. This PR fixes that by normalizing an empty
modules cache path to an empty string.
Otherwise these tests run into an exec format error when attempting to run
them with the internal shell. This is similar to other patches that have
done the same for similar tests.
#137363 was supposed to be NFC for the `CrossProcessModuleCache` (a.k.a
normal implicit module builds), but accidentally passed the wrong path
to `sys::fs::status`. Then, #141358 removed the correct path that
should've been passed instead. (The variable was flagged as unused.)
None of our existing tests caught this regression, we only found out due
to a SourceKit-LSP benchmark getting slower.
This PR re-implements the original behavior, adds new remark to Clang
for PCM input file validation, and uses it to create more reliable tests
of the `-fmodules-validate-once-per-build-session` flag.
With lazy template loading, it is possible to find non-canonical
FunctionDecls, depending on when redecl chains are completed. This
is a problem for templated conversion operators that would allow to
call either the copy assignment or the move assignment operator.
This ambiguity is resolved by isBetterReferenceBindingKind (called
from CompareStandardConversionSequences) ranking rvalue refs over
lvalue refs.
Unfortunately, this fix is hard to test in isolation without the
changes in https://github.com/llvm/llvm-project/pull/133057 that
make lazy template loading more likely to complete redecl chains
at "inconvenient" times. The added reproducer passes before and
after this commit, but would have failed with the proposed changes
of the linked PR.
Kudos to Maksim Ivanov for providing an initial version of the
reproducer that I further simplified.
