For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-IdExpression
| `-UnqualifiedId
| `-<
`-IdExpression
`-UnqualifiedId
`-y
```
But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-<
`-IdExpression
`-UnqualifiedId
`-y
```
Differential Revision: https://reviews.llvm.org/D85750
Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.
To achieve this we:
* Use the `RecursiveASTVisitor`'s API to traverse
`NestedNameSpecifierLoc`s and automatically create its syntax nodes
* Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.
In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.
This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
/*empty*/
simple-template-specifier:
template_opt simple-template-id
name-specifier:
global-specifier
decltype-specifier
identifier
simple-template-specifier
nested-name-specifier:
list(name-specifier, ::, non-empty, terminated)
It is a relaxed version of C++ [expr.prim.id] and quite simpler to map to our API.
TODO: refine name specifiers, `simple-template-name-specifier` and
decltype-name-specifier` are token soup for now.
* as we are using them only for integer and floating literals they have
the same behavior
* FileRange::text is simpler to call and is within the context of
syntax trees
Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the operator and interpret it as the underlying literal.
e.g.: `1.2_w` token generates syntax node IntegerLiteral(1.2_w)
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82157
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?
* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
callback from the default implementation of Traverse callbacks.
* In post-order traversal mode when we don't have a DataRecursionQueue,
RecursiveASTVisitor also calls the WalkUp callback from the default
implementation of Traverse callbacks.
* However, in post-order traversal mode when we have a DataRecursionQueue,
RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.
As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.
I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.
In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.
Reviewers: eduucaldas, ymandel
Reviewed By: eduucaldas, ymandel
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82486
This reverts commit 8bf4c40af813e73de77739b33b8808f6bd13497b.
This reverts commit 7b0be962d681c408c8ecf7180c6ad8f9fbcdaf2d.
This reverts commit 94454442c3c15a67ae70ef3a73616632968973fc.
Some compilers on some buildbots didn't accept the specialization of
is_same_method_impl in a non-namespace scope.
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?
* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
callback from the default implementation of Traverse callbacks.
* In post-order traversal mode when we don't have a DataRecursionQueue,
RecursiveASTVisitor also calls the WalkUp callback from the default
implementation of Traverse callbacks.
* However, in post-order traversal mode when we have a DataRecursionQueue,
RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.
As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.
I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.
In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.
Reviewers: eduucaldas, ymandel
Reviewed By: eduucaldas, ymandel
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82486
Summary:
I changed `markStmtChild` to ignore implicit expressions the same way as
`markExprChild` does it already. The test that I modified crashes
without this change.
Reviewers: hlopko, eduucaldas
Reviewed By: hlopko, eduucaldas
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D81019
Summary:
This patch removes delayed folding and replaces it with forward peeking.
Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:
```
int a,b;
```
is expressed in the AST as:
```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```
And in the syntax tree we need:
```
*: TranslationUnit
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| `-a
|-,
|-SimpleDeclarator
| `-b
|-;
```
So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.
This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).
This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
method in all places.
* Adds a bunch of tests
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76922
Summary:
Copy of https://reviews.llvm.org/D72446, submitting with Ilya's permission.
Only used to assign roles to child nodes for now. This is more efficient
than doing range-based queries.
In the future, will be exposed in the public API of syntax trees.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76355
Summary:
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.
Original summary:
> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> * specializations of functions and variables,
> * template parameters.
Reviewers: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76418
Summary:
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
Handles template declaration of all kinds.
Also builds template declaration nodes for specializations and explicit
instantiations of classes.
Some missing things will be addressed in the follow-up patches:
specializations of functions and variables,
template parameters.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76346
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Add checks for some structural invariants when building and mutating
the syntax trees.
Fix a bug failing the invariants after mutations: the parent of nodes
added into the tree was null.
When they are free-standing, e.g. `struct X;` or `struct X {};`.
Although this complicates the common case (of free-standing class
declarations), this ensures the less common case (e.g. `struct X {} a;`)
are handled uniformly and produce similar syntax trees.
This is both more efficient and avoids corner cases in
`SourceManager::isBeforeInTranslationUnit`.
The change is trivial and clearly a performance improvement on the hot
path of building the syntax tree, so sending without review.
Summary:
This patch adds facilities to mutate the syntax trees and produce
corresponding text replacements.
The public interface of the syntax library now includes facilities to:
1. perform type-safe modifications of syntax trees,
2. compute textual replacements to apply the modifications,
3. create syntax trees not backed by the source code.
For each of the three, we only add a few example transformations in this
patch to illustrate the idea, support for more kinds of nodes and
transformations will be done in follow-up patches.
The high-level mutation operations are implemented on top of operations
that allow to arbitrarily change the trees. They are considered to be
implementation details and are not available to the users of the
library.
Reviewers: sammccall, gribozavr2
Reviewed By: gribozavr2
Subscribers: merge_guards_bot, mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64573
Summary:
Also remove the temporary TopLevelDeclaration node and add
UnknownDeclaration to represent other unknown nodes.
See the follow-up change for building more top-level declarations.
Adding declarators is also pretty involved and will be done in another
follow-up patch.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: merge_guards_bot, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70787
Summary:
Most of the statements mirror the ones provided by clang AST.
Major differences are:
- expressions are wrapped into 'ExpressionStatement' instead of being
a subclass of statement,
- semicolons are always consumed by the leaf expressions (return,
expression satement, etc),
- some clang statements are not handled yet, we wrap those into an
UnknownStatement class, which is not present in clang.
We also define an 'Expression' and 'UnknownExpression' classes in order
to produce 'ExpressionStatement' where needed. The actual implementation
of expressions is not yet ready, it will follow later.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63835
Summary:
While useful as a sentinel value when iterating over tokens, having
'eof' in the tree, seems to do more harm than good.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: javed.absar, kristof.beyls, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64576
llvm-svn: 368062
