After this patch, we will generate the function body from implicitly
instantiated class. This is important for consumers with same
template arguments. Otherwise the consumers won't see the function body.
Since the consumers won't instantiate the templates again if they find an
instantiation.
Also we will generate the variable definition if the variable is
non-inline but known as constant. Such variables may not affect the
ABI, but they may get involved into the compile time constant computation
in the consumer's code. So we have to generate such definitions.
Previously we disabled to compute ODR hash for declarations from the
global module fragment. However, we missed the case that the functions
lives in the concept requiments (see the attached the test files for
example). And the mismatch causes the potential crashment.
Due to we will set the function body as lazy after we deserialize it and
we will only take its body when needed. However, we don't allow to take
the body during deserializing. So it is actually potentially problematic
if we set the body as lazy first and computing the hash value of the
function, which requires to deserialize its body. So we will meet a
crash here.
This patch tries to solve the issue by not taking the body of the
function from GMF. Note that we can't skip comparing the constraint
expression from the GMF directly since it is an key part of the
function selecting and it may be the reason why we can't return 0
directly for `FunctionDecl::getODRHash()` from the GMF.
Close https://github.com/llvm/llvm-project/issues/71034
See
https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755
This patch introduces reduced BMI, which doesn't contain the definitions
of functions and variables if its definitions won't contribute to the
ABI.
Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.
The user interfaces part it left to following patches to ease the
reviewing.
Close https://github.com/llvm/llvm-project/issues/80570.
In
a0b6747804,
we skipped ODR checks for decls in GMF. Then it should be natural to
skip storing the ODR values in BMI.
Generally it should be fine as long as the writer and the reader keep
consistent.
However, the use of preamble in clangd shows the tricky part.
For,
```
// test.cpp
module;
// any one off these is enough to crash clangd
// #include <iostream>
// #include <string_view>
// #include <cmath>
// #include <system_error>
// #include <new>
// #include <bit>
// probably many more
// only ok with libc++, not the system provided libstdc++ 13.2.1
// these are ok
export module test;
```
clangd will store the headers as preamble to speedup the parsing and the
preamble reuses the serialization techniques. (Generally we'd call the
preamble as PCH. However it is not true strictly. I've tested the PCH
wouldn't be problematic.) However, the tricky part is that the preamble
is not modules. It literally serialiaze and deserialize things. So
before clangd parsing the above test module, clangd will serialize the
headers into the preamble. Note that there is no concept like GMF now.
So the ODR bits are stored. However, when clangd parse the file
actually, the decls from preamble are thought as in GMF literally, then
hte ODR bits are skipped. Then mismatch happens.
To solve the problem, this patch adds another bit for decls to record
whether or not the ODR bits are skipped.
Close https://github.com/llvm/llvm-project/issues/79240
Cite the comment from @mizvekov in
//github.com/llvm/llvm-project/issues/79240:
> There are two kinds of bugs / issues relevant here:
>
> Clang bugs that this change hides
> Here we can add a Frontend flag that disables the GMF ODR check, just
> so
> we can keep tracking, testing and fixing these issues.
> The Driver would just always pass that flag.
> We could add that flag in this current issue.
> Bugs in user code:
> I don't think it's worth adding a corresponding Driver flag for
> controlling the above Frontend flag, since we intend it's behavior to
> become default as we fix the problems, and users interested in testing
> the more strict behavior can just use the Frontend flag directly.
This patch follows the suggestion:
- Introduce the CC1 flag `-fskip-odr-check-in-gmf` which is by default
off, so that the every existing test will still be tested with checking
ODR violations.
- Passing `-fskip-odr-check-in-gmf` in the driver to keep the behavior
we intended.
- Edit the document to tell the users who are still interested in more
strict checks can use `-Xclang -fno-skip-odr-check-in-gmf` to get the
existing behavior.
Close https://github.com/llvm/llvm-project/issues/79240.
See the linked issue for details. Given the frequency of issue reporting
about false positive ODR checks (I received private issue reports too),
I'd like to backport this to 18.x too.
Close https://github.com/llvm/llvm-project/issues/76443
Previously we assume the bits of the function scope index of ParmVarDecl
won't exceed 8. But this is a misreading. See the implementation of
`ParmVarDecl::getParameterIndex()`, which may
exceed the size of the normal bitfield. So it may be better to not
pack these bits.
bits and extract big bits from packed bits
Previously I tried to improve the size of .pcm files by introducing
packed bits. And I find we can improve it further by reordering the
bits.
The secret comes from the VBR format. We can find the formal definition
of VBR format in the doc of LLVM. The VBR format will be pretty
efficicent for small numbers.
For example, if we need to pack 8 bits into a value and the stored value
is 0xf0, the actual stored value will be 0b000111'110000, which takes 12
bits actually. However, if we changed the order to be 0x0f, then we
can store it as 0b001111, which takes 6 bits only now.
So we can improve the size by placing bits with lower probability to be
1 in the higher bits and extract bit bigs from the packed bits to make
it possible to be optimized by VBR.
After this patch, the size of std module becomes to 27.7MB from 28.1MB.
This patch tries to pack more bits into a value to reduce the size of
.pcm files. Also, after we introduced BitsPackers, it may slightly
better to adjust the way we use Abbrev.
After this patch, the size of the BMI for std module reduce from 28.94MB
to 28.1 MB.
This was reverted due to it broke the build of lldb. The reason that we
skip the serialization of a source location incorrectly. And this patch
now fixes that.
This patch tries to pack more bits into a value to reduce the size of
.pcm files. Also, after we introduced BitsPackers, it may slightly
better to adjust the way we use Abbrev.
After this patch, the size of the BMI for std module reduce from 28.94MB
to 28.1 MB.
This patch tries to reduce the size of the BMIs by packing more bits
into an unsigned integer.
This patch was reverted due to buildbot failure report. But it should be
irrevelent after I took a double look. So I tried to recommit this NFC
change again.
This reverts commit 10951050b5f371eb3e7cacce1691c4eb2fe2eab5.
This should be part of 8c334627818437180176b16b1932 to revert
9406ea3fe32e59a7d2 completely.
Close https://github.com/llvm/llvm-project/issues/71347
Previously I misread the concept of module purview. I thought if a
declaration attached to a unnamed module, it can't be part of the module
purview. But after the issue report, I recognized that module purview is
more of a concept about locations instead of semantics.
Concretely, the things in the language linkage after module declarations
can be exported.
This patch refactors `Module::isModulePurview()` and introduces some
possible code cleanups.
This patch converts `ImplicitParamDecl::ImplicitParamKind` into a scoped enum at namespace scope, making it eligible for forward declaring. This is useful for `preferred_type` annotations on bit-fields.
Previously, the boolean values will occupy spaces that can contain
integers. It wastes the spaces especially if the boolean values are
serialized consecutively. The patch tries to pack such consecutive
boolean values (and enum values) so that we can save more spaces and so
the times.
Before the patch, we need 4.478s (in my machine) to build the std module
(https://libcxx.llvm.org/Modules.html) with 28712 bytes for size of the
BMI. After the patch, the time becomes to 4.374s and the size becomes to
27388 bytes for the size of the BMI.
This is intended to be a NFC patch.
This patch doesn't optimize all such cases. We can do it later after we
have consensus on this.
This patch introduces a new enumerator `Invalid = 0`, shifting other enumerators by +1. Contrary to how it might sound, this actually affirms status quo of how this enum is stored in `clang::Decl`:
```
/// If 0, we have not computed the linkage of this declaration.
/// Otherwise, it is the linkage + 1.
mutable unsigned CacheValidAndLinkage : 3;
```
This patch makes debuggers to not be mistaken about enumerator stored in this bit-field. It also converts `clang::Linkage` to a scoped enum.
This patch converts `LinkageSpecDecl::LanguageIDs` into scoped enum, and moves it to namespace scope, so that it can be forward-declared where required.
This patch moves `ObjCMethodDecl::ImplementationControl` to a DeclBase.h so that it's complete at the point where corresponsing bit-field is declared. This patch also converts it to a scoped enum `clang::ObjCImplementationControl`.
This patch moves `OMPDeclareReductionDecl::InitKind` to DeclBase.h, so that it's complete at the point where corresponding bit-field is declared. This patch also converts it to scoped enum named `OMPDeclareReductionInitKind`
This patch refactor said enums to hold total size of a bit-field, and not just non-inherited bits. This brings `Type` and `DeclContext` in line with `Comment` and `Stmt`. It also makes it unnecessary to list all transitive bases of a bit-field as unnamed bit-fields, which makes it more friendly towards debuggers.
This removes the `ClassScopeFunctionSpecializationDecl` `Decl` node, and
instead uses `DependentFunctionTemplateSpecializationInfo` to handle
such declarations. `DependentFunctionTemplateSpecializationInfo` is also
changed to store a `const ASTTemplateArgumentListInfo*` to be more in
line with `FunctionTemplateSpecializationInfo`.
This also changes `FunctionDecl::isFunctionTemplateSpecialization` to
return `true` for dependent specializations, and
`FunctionDecl::getTemplateSpecializationKind`/`FunctionDecl::getTemplateSpecializationKindForInstantiation`
to return `TSK_ExplicitSpecialization` for non-friend dependent
specializations (the same behavior as dependent class scope
`ClassTemplateSepcializationDecl` & `VarTemplateSepcializationDecl`).
This patch adds a concept AST node (`ConceptLoc`) and uses it at the corresponding places.
There are three objects that might have constraints via concepts:
`TypeConstraint`, `ConceptSpecializationExpr` and `AutoTypeLoc`.
The first two inherit from `ConceptReference` while the latter has
the information about a possible constraint directly stored in `AutoTypeLocInfo`. It would be nice if the concept information would be stored the same way in all three cases.
Moreover the current structure makes it difficult to deal with these concepts. For example in Clangd accessing the locations of constraints of a `AutoTypeLoc` can only be done with quite ugly hacks.
So we think that it makes sense to create a new AST node for such concepts.
In details we propose the following:
- Rename `ConceptReference` to `ConceptLoc` (or something else what is approriate)
and make it the new AST node.
- `TypeConstraint` and `ConceptSpecializationExpr` do not longer inherit from `ConceptReference` but store a pointer to a `ConceptLoc`.
- `AutoTypeLoc` stores a pointer to `ConceptLoc` instead of storing the concept info in `AutoTypeLocInfo`.
This patch implements a first version of this idea which compiles and where the existing tests pass.
To make this patch as small as possible we keep the existing member functions to access concept data. Later these can be replaced by directly calling the corresponding functions of the `ConceptLoc`s.
Differential Revision: https://reviews.llvm.org/D155858
Close https://github.com/llvm/llvm-project/issues/61064.
The root cause of the issue is that we will deserilize some declarations
eagerly when reading the BMI. However, many declarations in the BMI are
not necessary for the importer. So it wastes a lot of time.
The new commit handles the MSVC's extension #pragma comment and #pragma
detect_mismatch to follow MSVC's behavior. See pr61783 for details.
This is important to break deserialization cycles, where a lambda in a
default member initializer can refer to the field as its context
declaration, and the initializer of the field can refer back to the
lambda.
This is a follow-up to bc73ef0031b5, which applied the same fix to
variable declarations for the same reason.
Previously, distinct lambdas would get merged, and multiple definitions
of the same lambda would not get merged, because we attempted to
identify lambdas by their ordinal position within their lexical
DeclContext. This failed for lambdas within namespace-scope variables
and within variable templates, where the lexical position in the context
containing the variable didn't uniquely identify the lambda.
In this patch, we instead identify lambda closure types by index within
their context declaration, which does uniquely identify them in a way
that's consistent across modules.
This change causes a deserialization cycle between the type of a
variable with deduced type and a lambda appearing as the initializer of
the variable -- reading the variable's type requires reading and merging
the lambda, and reading the lambda requires reading and merging the
variable. This is addressed by deferring loading the deduced type of a
variable until after we finish recursive deserialization.
This also exposes a pre-existing subtle issue where loading a
variable declaration would trigger immediate loading of its initializer,
which could recursively refer back to properties of the variable. This
particularly causes problems if the initializer contains a
lambda-expression, but can be problematic in general. That is addressed
by switching to lazily loading the initializers of variables rather than
always loading them with the variable declaration. As well as fixing a
deserialization cycle, that should improve laziness of deserialization
in general.
LambdaDefinitionData had 63 spare bits in it, presumably caused by an
off-by-one-error in some previous change. This change claims 32 of those bits
as a counter for the lambda within its context. We could probably move the
numbering to separate storage, like we do for the device-side mangling number,
to optimize the likely-common case where all three numbers (host-side mangling
number, device-side mangling number, and index within the context declaration)
are zero, but that's not done in this change.
Fixes#60985.
Reviewed By: #clang-language-wg, aaron.ballman
Differential Revision: https://reviews.llvm.org/D145737
This reverts commit af86957cbbffd3dfff3c6750ebddf118aebd0069.
Close https://github.com/llvm/llvm-project/issues/61733.
Previously I banned the eagerly loading for declarations from named
modules to speedup the process of reading modules. But I didn't think
about special decls like PragmaCommentDecl and PragmaDetectMismatchDecl.
So here is the issue https://github.com/llvm/llvm-project/issues/61733.
Note that the current behavior is still incorrect. Given:
```
// mod.cppm
module;
export module mod;
```
and
```
// user.cpp
import mod;
```
Now the IR of `user.cpp` will contain the metadata '!0 =
!{!"msvcprt.lib"}' incorrectly. The root cause of the problem is that
`EagerlyDeserializedDecls` is designed for headers and it didn't take
care for named modules. We need to redesign a new mechanism for named
modules.
Close https://github.com/llvm/llvm-project/issues/61064.
The root cause of the issue is that we will deserilize some declarations
eagerly when reading the BMI. However, many declarations in the BMI are
not necessary for the importer. So it wastes a lot of time.
Close https://github.com/llvm/llvm-project/issues/60693.
In this issue, we can find that the importer will try to generate the
template specialization again in the importer, which is not good and
wastes time. This patch tries to address the problem.
This re-commits part of c79635cce845. It is reverted since it contains
platform-inconsistent constant. Now the patch only contains constant
defined in DeclBase.h so it should be platform-independent. And this
should be still helpful.
Reviewed By: erichkeane
Differential Revision: https://reviews.llvm.org/D141992
When two modules contain interfaces with the same name, check the
definitions are equivalent and diagnose if they are not.
Differential Revision: https://reviews.llvm.org/D140073
When two modules contain struct/union with the same name, check the
definitions are equivalent and diagnose if they are not. This is similar
to `CXXRecordDecl` where we already discover and diagnose mismatches.
rdar://problem/56764293
Differential Revision: https://reviews.llvm.org/D71734
mention developers to remember to touch the serializer after them
modified the field of decls
It is easy for the developers to forget to touch the serializer after
they add new field to decls. Then if the existing tests fail to catch
such cases, it may be a bug report from users some day. And it is
time-consuming to solve such bugs.
To mitigate the problem, I add the static_asserts in the serializer. So
that the developers can understand they need to modify the serializer
after they saw the static assertion failure. Although this can't solve
all the problems, I feel the current status can be much better.
Reviewed By: erichkeane
Differential Revision: https://reviews.llvm.org/D141992
Close https://github.com/llvm/llvm-project/issues/59719.
The root cause of the problem is that we forgot to serialize a new
introduced bit to FunctionDeclBits. Maybe we need to find some methods
to work for detecting this.
