This is a major change on how we represent nested name qualifications in
the AST.
* The nested name specifier itself and how it's stored is changed. The
prefixes for types are handled within the type hierarchy, which makes
canonicalization for them super cheap, no memory allocation required.
Also translating a type into nested name specifier form becomes a no-op.
An identifier is stored as a DependentNameType. The nested name
specifier gains a lightweight handle class, to be used instead of
passing around pointers, which is similar to what is implemented for
TemplateName. There is still one free bit available, and this handle can
be used within a PointerUnion and PointerIntPair, which should keep
bit-packing aficionados happy.
* The ElaboratedType node is removed, all type nodes in which it could
previously apply to can now store the elaborated keyword and name
qualifier, tail allocating when present.
* TagTypes can now point to the exact declaration found when producing
these, as opposed to the previous situation of there only existing one
TagType per entity. This increases the amount of type sugar retained,
and can have several applications, for example in tracking module
ownership, and other tools which care about source file origins, such as
IWYU. These TagTypes are lazily allocated, in order to limit the
increase in AST size.
This patch offers a great performance benefit.
It greatly improves compilation time for
[stdexec](https://github.com/NVIDIA/stdexec). For one datapoint, for
`test_on2.cpp` in that project, which is the slowest compiling test,
this patch improves `-c` compilation time by about 7.2%, with the
`-fsyntax-only` improvement being at ~12%.
This has great results on compile-time-tracker as well:
This patch also further enables other optimziations in the future, and
will reduce the performance impact of template specialization resugaring
when that lands.
It has some other miscelaneous drive-by fixes.
About the review: Yes the patch is huge, sorry about that. Part of the
reason is that I started by the nested name specifier part, before the
ElaboratedType part, but that had a huge performance downside, as
ElaboratedType is a big performance hog. I didn't have the steam to go
back and change the patch after the fact.
There is also a lot of internal API changes, and it made sense to remove
ElaboratedType in one go, versus removing it from one type at a time, as
that would present much more churn to the users. Also, the nested name
specifier having a different API avoids missing changes related to how
prefixes work now, which could make existing code compile but not work.
How to review: The important changes are all in
`clang/include/clang/AST` and `clang/lib/AST`, with also important
changes in `clang/lib/Sema/TreeTransform.h`.
The rest and bulk of the changes are mostly consequences of the changes
in API.
PS: TagType::getDecl is renamed to `getOriginalDecl` in this patch, just
for easier to rebasing. I plan to rename it back after this lands.
Fixes#136624
Fixes https://github.com/llvm/llvm-project/issues/43179
Fixes https://github.com/llvm/llvm-project/issues/68670
Fixes https://github.com/llvm/llvm-project/issues/92757
It doesn't make sense that we only build a RecoveryExpr for expressions
with invalid trailing commas. This patch extends it so that we now
always build up a RecoveryExpr whenever the call contains anything
invalid. As a result, we can back out HasTrailingComma.
There is only one diagnostic change as to concepts, where a RecoveryExpr
than an ExprError is now used to model an invalid requires clause, for
which we suggest adding parentheses around it. (This looks like what GCC
diagnoses)
This PR reapply https://github.com/llvm/llvm-project/pull/117437.
The issue has been fixed by the 2nd commit, we need to ignore parens in
CXXDefaultArgExpr when build CFG, because CXXDefaultArgExpr::getExpr
stripped off the top level FullExpr and ConstantExpr, ParenExpr may
occurres in the top level.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
This caused assertion failures:
clang/lib/Analysis/CFG.cpp:822:
void (anonymous namespace)::CFGBuilder::appendStmt(CFGBlock *, const Stmt *):
Assertion `!isa<Expr>(S) || cast<Expr>(S)->IgnoreParens() == S' failed.
See comment on the PR.
This reverts commit 44aa618ef67d302f5ab77cc591fb3434fe967a2e.
With the commit d5308949cf884d8e4b971d51a8b4f73584c4adec, we now
preserve the initializer for invalid decls with the recovery-expr.
However there is a chance that the original init expr is a typo-expr, we
should not preserve it in the final AST, as typo-expr is an internal AST
node. We should use the one after the typo correction.
This is spotted by a clangd hover crash on the testcase.
These are an artifact of how types are structured but serve little
purpose, merely showing that the type is sugared in some way. For
example, ElaboratedType's existence means struct S gets printed as
'struct S':'struct S' in the AST, which is unnecessary visual clutter.
Note that skipping the second print when the types have the same string
matches what we do for diagnostics, where the aka will be skipped.
Prior to the patch, we didn't build a DeclRefExpr if the Decl being
referred to is invalid, because many clang downstream AST consumers
assume it, violating it will cause many diagnostic regressions.
With this patch, we build a DeclRefExpr enven for an invalid decl (when the
AcceptInvalidDecl is true), and wrap it with a dependent-type
RecoveryExpr (to prevent follow-up semantic analysis, and diagnostic
regressions).
This is a revised version of https://reviews.llvm.org/D76831
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D121599
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
---
Troubleshooting list to deal with any breakage seen with this patch:
1) The most likely effect one would see by this patch is a change in how
a type is printed. The type printer will, by design and default,
print types as written. There are customization options there, but
not that many, and they mainly apply to how to print a type that we
somehow failed to track how it was written. This patch fixes a
problem where we failed to distinguish between a type
that was written without any elaborated-type qualifiers,
such as a 'struct'/'class' tags and name spacifiers such as 'std::',
and one that has been stripped of any 'metadata' that identifies such,
the so called canonical types.
Example:
```
namespace foo {
struct A {};
A a;
};
```
If one were to print the type of `foo::a`, prior to this patch, this
would result in `foo::A`. This is how the type printer would have,
by default, printed the canonical type of A as well.
As soon as you add any name qualifiers to A, the type printer would
suddenly start accurately printing the type as written. This patch
will make it print it accurately even when written without
qualifiers, so we will just print `A` for the initial example, as
the user did not really write that `foo::` namespace qualifier.
2) This patch could expose a bug in some AST matcher. Matching types
is harder to get right when there is sugar involved. For example,
if you want to match a type against being a pointer to some type A,
then you have to account for getting a type that is sugar for a
pointer to A, or being a pointer to sugar to A, or both! Usually
you would get the second part wrong, and this would work for a
very simple test where you don't use any name qualifiers, but
you would discover is broken when you do. The usual fix is to
either use the matcher which strips sugar, which is annoying
to use as for example if you match an N level pointer, you have
to put N+1 such matchers in there, beginning to end and between
all those levels. But in a lot of cases, if the property you want
to match is present in the canonical type, it's easier and faster
to just match on that... This goes with what is said in 1), if
you want to match against the name of a type, and you want
the name string to be something stable, perhaps matching on
the name of the canonical type is the better choice.
3) This patch could expose a bug in how you get the source range of some
TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
which only looks at the given TypeLoc node. This patch introduces a new,
and more common TypeLoc node which contains no source locations on itself.
This is not an inovation here, and some other, more rare TypeLoc nodes could
also have this property, but if you use getLocalSourceRange on them, it's not
going to return any valid locations, because it doesn't have any. The right fix
here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
into the inner TypeLoc to get the source range if it doesn't find it on the
top level one. You can use getLocalSourceRange if you are really into
micro-optimizations and you have some outside knowledge that the TypeLocs you are
dealing with will always include some source location.
4) Exposed a bug somewhere in the use of the normal clang type class API, where you
have some type, you want to see if that type is some particular kind, you try a
`dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
Again, like 2), this would usually have been tested poorly with some simple tests with
no qualifications, and would have been broken had there been any other kind of type sugar,
be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.
5) It could be a bug in this patch perhaps.
Let me know if you need any help!
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
This reverts commit 7c51f02effdbd0d5e12bfd26f9c3b2ab5687c93f because it
stills breaks the LLDB tests. This was re-landed without addressing the
issue or even agreement on how to address the issue. More details and
discussion in https://reviews.llvm.org/D112374.
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
---
Troubleshooting list to deal with any breakage seen with this patch:
1) The most likely effect one would see by this patch is a change in how
a type is printed. The type printer will, by design and default,
print types as written. There are customization options there, but
not that many, and they mainly apply to how to print a type that we
somehow failed to track how it was written. This patch fixes a
problem where we failed to distinguish between a type
that was written without any elaborated-type qualifiers,
such as a 'struct'/'class' tags and name spacifiers such as 'std::',
and one that has been stripped of any 'metadata' that identifies such,
the so called canonical types.
Example:
```
namespace foo {
struct A {};
A a;
};
```
If one were to print the type of `foo::a`, prior to this patch, this
would result in `foo::A`. This is how the type printer would have,
by default, printed the canonical type of A as well.
As soon as you add any name qualifiers to A, the type printer would
suddenly start accurately printing the type as written. This patch
will make it print it accurately even when written without
qualifiers, so we will just print `A` for the initial example, as
the user did not really write that `foo::` namespace qualifier.
2) This patch could expose a bug in some AST matcher. Matching types
is harder to get right when there is sugar involved. For example,
if you want to match a type against being a pointer to some type A,
then you have to account for getting a type that is sugar for a
pointer to A, or being a pointer to sugar to A, or both! Usually
you would get the second part wrong, and this would work for a
very simple test where you don't use any name qualifiers, but
you would discover is broken when you do. The usual fix is to
either use the matcher which strips sugar, which is annoying
to use as for example if you match an N level pointer, you have
to put N+1 such matchers in there, beginning to end and between
all those levels. But in a lot of cases, if the property you want
to match is present in the canonical type, it's easier and faster
to just match on that... This goes with what is said in 1), if
you want to match against the name of a type, and you want
the name string to be something stable, perhaps matching on
the name of the canonical type is the better choice.
3) This patch could exposed a bug in how you get the source range of some
TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
which only looks at the given TypeLoc node. This patch introduces a new,
and more common TypeLoc node which contains no source locations on itself.
This is not an inovation here, and some other, more rare TypeLoc nodes could
also have this property, but if you use getLocalSourceRange on them, it's not
going to return any valid locations, because it doesn't have any. The right fix
here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
into the inner TypeLoc to get the source range if it doesn't find it on the
top level one. You can use getLocalSourceRange if you are really into
micro-optimizations and you have some outside knowledge that the TypeLocs you are
dealing with will always include some source location.
4) Exposed a bug somewhere in the use of the normal clang type class API, where you
have some type, you want to see if that type is some particular kind, you try a
`dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
Again, like 2), this would usually have been tested poorly with some simple tests with
no qualifications, and would have been broken had there been any other kind of type sugar,
be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.
5) It could be a bug in this patch perhaps.
Let me know if you need any help!
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
This reverts commit bdc6974f92304f4ed542241b9b89ba58ba6b20aa because it
breaks all the LLDB tests that import the std module.
import-std-module/array.TestArrayFromStdModule.py
import-std-module/deque-basic.TestDequeFromStdModule.py
import-std-module/deque-dbg-info-content.TestDbgInfoContentDequeFromStdModule.py
import-std-module/forward_list.TestForwardListFromStdModule.py
import-std-module/forward_list-dbg-info-content.TestDbgInfoContentForwardListFromStdModule.py
import-std-module/list.TestListFromStdModule.py
import-std-module/list-dbg-info-content.TestDbgInfoContentListFromStdModule.py
import-std-module/queue.TestQueueFromStdModule.py
import-std-module/stack.TestStackFromStdModule.py
import-std-module/vector.TestVectorFromStdModule.py
import-std-module/vector-bool.TestVectorBoolFromStdModule.py
import-std-module/vector-dbg-info-content.TestDbgInfoContentVectorFromStdModule.py
import-std-module/vector-of-vectors.TestVectorOfVectorsFromStdModule.py
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45301/
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
Previously, we didin't build a DeclRefExpr which refers to an invalid declaration.
In this patch, we handle this case by building an empty RecoveryExpr,
which will preserve more broken code (AST parent nodes that contain the
RecoveryExpr is preserved in the AST).
Differential Revision: https://reviews.llvm.org/D120812
This improves diagnostic (& important to me, DWARF) accuracy - otherwise
there could be ambiguities between "std::nullptr_t" and some user-defined
type that's /actually/ "nullptr_t" defined in the global namespace.
Differential Revision: https://reviews.llvm.org/D110044
Before this patch, CXXCtorInitializers that don't typecheck get discarded in
most cases. In particular:
- typos that can't be corrected don't turn into RecoveryExpr. The full expr
disappears instead, and without an init expr we discard the node.
- initializers that fail initialization (e.g. constructor overload resolution)
are discarded too.
This patch addresses both these issues (a bit clunkily and repetitively, for
member/base/delegating initializers)
It does not preserve any AST nodes when the member/base can't be resolved or
other problems of that nature. That breaks invariants of CXXCtorInitializer
itself, and we don't have a "weak" RecoveryCtorInitializer like we do for Expr.
I believe the changes to diagnostics in existing tests are improvements.
(We're able to do some analysis on the non-broken parts of the initializer)
Differential Revision: https://reviews.llvm.org/D101641
This reverts commit efa9aaad703e6b150980ed1a74b4e7c9da7d85a2 and adds a
crash test.
The commit caused a crash in CodeGen with -fms-compatibility, see
https://bugs.llvm.org/show_bug.cgi?id=48690.
Because of typo-correction, the AST can be transformed, and the transformed
AST is marginally useful for diagnostics purpose, the following
diagnostics usually do harm than good (easily cause confusions).
Given the following code:
```
void abcc();
void test() {
if (abc());
// diagnostic 1 (for the typo-correction): the typo is correct to `abcc()`, so the code is treate as `if (abcc())` in AST perspective;
// diagnostic 2 (for mismatch type): we perform an type-analysis on `if`, discover the type is not match
}
```
The secondary diagnostic "convertable to bool" is likely bogus to users.
The idea is to use RecoveryExpr (clang's dependent mechanism) to preserve the
recovery behavior but suppress all follow-up diagnostics.
Differential Revision: https://reviews.llvm.org/D89946
RecoveryExpr was always lvalue, but it is wrong if we use it to model
broken function calls, function call expression has more compliated rules:
- a call to a function whose return type is an lvalue reference yields an lvalue;
- a call to a function whose return type is an rvalue reference yields an xvalue;
- a call to a function whose return type is nonreference type yields a prvalue;
This patch makes the recovery-expr align with the function call if it is
modeled a broken call.
Differential revision: https://reviews.llvm.org/D83201
Summary:
For a none-function-like unresolved expression, clang builds a TypoExpr
for it, and tries to correct it afterwards. If the typo-correction
fails, clang just drops the whole expr.
This patch improves the recovery strategy -- if the typo-correction
fails, we preserve the AST by degrading the typo exprs to recovery
exprs.
This would improve toolings for "undef_var" broken cases:
```
void foo();
void test() {
fo^o(undef_var); // go-to-def, hover still works.
}
```
TESTED=ran tests with this patch + turn-on-recovery-ast patch, it breaks
one declare_variant_messages testcase (the diagnostics are slightly
changed), I think it is acceptable.
```
Error: 'error' diagnostics seen but not expected:
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 16: expected 'match' clause on 'omp declare variant' directive
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 57: expected 'match' clause on 'omp declare variant' directive
error: 'warning' diagnostics expected but not seen:
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 47: the context selector 'kind' in the context set 'device' cannot have a score ('<invalid>'); score ignored
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 87: the context selector 'kind' in the context set 'device' cannot have a score ('<invalid>'); score ignored
error: 'warning' diagnostics seen but not expected:
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 47: the context selector 'kind' in the context set 'device' cannot have a score ('<recovery-expr>()'); score ignored
File llvm-project/clang/test/OpenMP/declare_variant_messages.cpp Line 87: the context selector 'kind' in the context set 'device' cannot have a score ('<recovery-expr>()'); score ignored
6 errors generated.
```
Reviewers: sammccall, jdoerfert
Subscribers: sstefan1, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80733
Summary:
The Initializer of a InitListExpr can be reset to null, which leads to
nullptr-acces crashes.
Reviewers: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80980
Summary:
With recovery-ast, we will get an undeduced `auto` return type for
"auto foo()->undef()" function declaration, the function decl still keeps
valid, it is dangerous, and breaks assumptions in clang, and leads crashes.
This patch invalidates these functions, if we deduce autos from the
return rexpression, which is similar to auto VarDecl.
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80221
RecoveryExprs are modeled as dependent type to prevent bogus diagnostics
and crashes in clang.
This patch allows to preseve the type for broken calls when the
RecoveryEprs have a known type, e.g. a broken non-overloaded call, a
overloaded call when the all candidates have the same return type, so
that more features (code completion still work on "take2args(x).^") still
work.
However, adding the type is risky, which may result in more clang code being
affected leading to new crashes and hurt diagnostic, and it requires large
effort to minimize the affect (update all sites in clang to handle errorDepend
case), so we add a new flag (off by default) to allow us to develop/test
them incrementally.
This patch also has some trivial fixes to suppress diagnostics (to prevent regressions).
Tested:
all existing tests are passed (when both "-frecovery-ast", "-frecovery-ast-type" flags are flipped on);
Reviewed By: sammccall
Subscribers: rsmith, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D79160
Summary:
This patch contains 2 separate changes:
1) the initializer of a variable should play no part in decl "invalid" bit;
2) preserve the invalid initializer via recovery exprs;
With 1), we will regress the diagnostics (one big regression is that we loose
the "selected 'begin' function with iterator type" diagnostic in for-range stmt;
but with 2) together, we don't have regressions (the new diagnostics seems to be
improved).
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D78116
Summary: Previously, we dropped the AST node for nonexistent member exprs.
Reviewers: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76764
Normally clang avoids creating expressions when it encounters semantic
errors, even if the parser knows which expression to produce.
This works well for the compiler. However, this is not ideal for
source-level tools that have to deal with broken code, e.g. clangd is
not able to provide navigation features even for names that compiler
knows how to resolve.
The new RecoveryExpr aims to capture the minimal set of information
useful for the tools that need to deal with incorrect code:
source range of the expression being dropped,
subexpressions of the expression.
We aim to make constructing RecoveryExprs as simple as possible to
ensure writing code to avoid dropping expressions is easy.
Producing RecoveryExprs can result in new code paths being taken in the
frontend. In particular, clang can produce some new diagnostics now and
we aim to suppress bogus ones based on Expr::containsErrors.
We deliberately produce RecoveryExprs only in the parser for now to
minimize the code affected by this patch. Producing RecoveryExprs in
Sema potentially allows to preserve more information (e.g. type of an
expression), but also results in more code being affected. E.g.
SFINAE checks will have to take presence of RecoveryExprs into account.
Initial implementation only works in C++ mode, as it relies on compiler
postponing diagnostics on dependent expressions. C and ObjC often do not
do this, so they require more work to make sure we do not produce too
many bogus diagnostics on the new expressions.
See documentation of RecoveryExpr for more details.
original patch from Ilya
This change is based on https://reviews.llvm.org/D61722
Reviewers: sammccall, rsmith
Reviewed By: sammccall, rsmith
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69330
