10 Commits

Author SHA1 Message Date
Haojian Wu
6a9f79e102 [pseudo] Eliminate the type-name identifier ambiguities in the grammar.
See https://reviews.llvm.org/D130626 for motivation.

Identifier in the grammar has different categories (type-name, template-name,
namespace-name), they requires semantic information to resolve. This patch is
to eliminate the "local" ambiguities in type-name, and namespace-name, which
gives us a performance boost of the parser:

  - eliminate all different type rules (class-name, enum-name, typedef-name), and
    fold them into a unified type-name, this removes the #1 type-name ambiguity, and
    gives us a big performance boost;
  - remove the namespace-alis rules, as they're hard and uninteresting;

Note that we could eliminate more and gain more performance (like fold template-name,
type-name, namespace together), but at current stage, we'd like keep all existing
categories of the identifier (as they might assist in correlated disambiguation &
keep the representation of important concepts uniform).

| file               |ambiguous nodes |  forest size     | glrParse performance |
|SemaCodeComplete.cpp|  11k -> 5.7K   | 10.4MB -> 7.9MB  | 7.1MB/s -> 9.98MB/s  |
|       AST.cpp      |  1.3k -> 0.73K | 0.99MB -> 0.77MB | 6.7MB/s -> 8.4MB/s   |

Differential Revision: https://reviews.llvm.org/D130747
2022-08-17 14:30:53 +02:00
Sam McCall
d9d554a3f4 [pseudo] Add ambiguity & unparseability metrics to -print-statistics
These can be used to quantify parsing improvements from a change.

Differential Revision: https://reviews.llvm.org/D130199
2022-07-22 10:35:06 +02:00
Haojian Wu
d489b3807f [pseudo] Implement a guard to determine function declarator.
This eliminates some simple-declaration/function-definition false
parses.

- implement a function to determine whether a declarator ForestNode is a
  function declarator;
- extend the standard declarator to two guarded function-declarator and
  non-function-declarator nonterminals;

Differential Revision: https://reviews.llvm.org/D129222
2022-07-19 09:44:45 +02:00
Sam McCall
9b6bb12b85 [pseudo] Add ForestNode descendants iterator, print ambiguous/opaque node stats.
Differential Revision: https://reviews.llvm.org/D128930
2022-06-30 21:20:55 +02:00
Sam McCall
bc5e7ced1c [pseudo] Fix bugs/inconsistencies in forest dump.
- when printing a shared node for the second time, don't print its children
  (This keeps output proportional to the size of the structure)
- when printing a shared node for the second time, print its type only, not rule
  (for consistency with above: don't dump details of nodes twice)
- don't abbreviate shared nodes, to ensure we can prune the tree there

Differential Revision: https://reviews.llvm.org/D128805
2022-06-29 22:56:26 +02:00
Haojian Wu
28eeea1e27 [pseudo]Pull out the operator< test, NFC
Fix the review comment in https://reviews.llvm.org/D125479.
2022-06-07 11:00:08 +02:00
Haojian Wu
c4546091ed [pseudo] Use a real language option in the parser.
Differential Revision: https://reviews.llvm.org/D124831
2022-05-03 22:24:56 +02:00
Haojian Wu
9f38da258e [pseudo] Implement the GLR parsing algorithm.
This patch implements a standard GLR parsing algorithm, the
core piece of the pseudoparser.

- it parses preprocessed C++ code, currently it supports correct code
  only and parse them as a translation-unit;
- it produces a forest which stores all possible trees in an efficient
  manner (only a single node being build for per (SymbolID, Token Range));
  no disambiguation yet;

Reland with a fix for g++'s -fpermissive error on previous declaration `GSS& GSS;`.

Differential Revision: https://reviews.llvm.org/D121150
2022-05-03 20:25:23 +02:00
Haojian Wu
860eabb395 Revert "[pseudo] Implement the GLR parsing algorithm."
This breaks some buildbots (on the declaration GSS& GSS), will fix it
later.

This reverts commit eac22d0754f70df10ea0eb6f59cbd1ef012ab5a4.
2022-05-03 15:54:10 +02:00
Sam McCall
eac22d0754 [pseudo] Implement the GLR parsing algorithm.
This patch implements a standard GLR parsing algorithm, the
core piece of the pseudoparser.

- it parses preprocessed C++ code, currently it supports correct code
  only and parse them as a translation-unit;
- it produces a forest which stores all possible trees in an efficient
  manner (only a single node being build for per (SymbolID, Token Range));
  no disambiguation yet;

Differential Revision: https://reviews.llvm.org/D121150
2022-05-03 15:42:07 +02:00