65 Commits

Author SHA1 Message Date
Paul Kirth
2600533a66
[clang-doc] Switch to string internment (#190044)
This is the first step in migrating all the Info types to be POD. We
introduced a shared string saver that can be used safely across threads,
and updated the internal represntation of various data types to use
these over owned strings, like SmallString or std::string.

This also required changes to YAMLGenerator to keep the single quoted
string formatting and to update the YAML traits.

This change gives an almost 50% reduction in peak memory when building
documentation for clang, at about a 10% performance loss. Future patches
can mitigate the performance penalties, and further reduce memory use.

| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 920.5s | 1011.0s | +9.8% | +9.8% |
| Memory | 86.0G | 86.0G | 44.9G | -47.8% | -47.8% |

| Benchmark | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| BM_BitcodeReader_Scale/10 | 67.9us | 67.9us | 70.0us | +3.0% | +3.0% |
| BM_BitcodeReader_Scale/10000 | 70.5ms | 70.5ms | 21.3ms | -69.8% |
-69.8% |
| BM_BitcodeReader_Scale/4096 | 23.2ms | 23.2ms | 4.5ms | -80.7% |
-80.7% |
| BM_BitcodeReader_Scale/512 | 509.4us | 509.4us | 538.8us | +5.8% |
+5.8% |
| BM_BitcodeReader_Scale/64 | 114.8us | 114.8us | 118.0us | +2.8% |
+2.8% |
| BM_Index_Insertion/10 | 2.3us | 2.3us | 4.0us | +71.6% | +71.6% |
| BM_Index_Insertion/10000 | 3.1ms | 3.1ms | 5.0ms | +60.6% | +60.6% |
| BM_Index_Insertion/4096 | 1.3ms | 1.3ms | 2.0ms | +57.1% | +57.1% |
| BM_Index_Insertion/512 | 153.6us | 153.6us | 245.0us | +59.6% | +59.6%
|
| BM_Index_Insertion/64 | 18.1us | 18.1us | 28.9us | +60.0% | +60.0% |
| BM_JSONGenerator_Scale/10 | 36.8us | 36.8us | 36.4us | -1.3% | -1.3% |
| BM_Mapper_Scale/10000 | 104.3ms | 104.3ms | 105.4ms | +1.0% | +1.0% |
| BM_Mapper_Scale/512 | 7.6ms | 7.6ms | 7.7ms | +1.9% | +1.9% |
| BM_MergeInfos_Scale/10000 | 12.2ms | 12.2ms | 1.4ms | -88.2% | -88.2%
|
| BM_MergeInfos_Scale/2 | 1.9us | 1.9us | 1.7us | -10.3% | -10.3% |
| BM_MergeInfos_Scale/4096 | 2.8ms | 2.8ms | 495.6us | -82.2% | -82.2% |
| BM_MergeInfos_Scale/512 | 68.9us | 68.9us | 34.6us | -49.7% | -49.7% |
| BM_MergeInfos_Scale/64 | 10.3us | 10.3us | 6.0us | -41.6% | -41.6% |
| BM_MergeInfos_Scale/8 | 2.8us | 2.8us | 2.1us | -24.4% | -24.4% |
| BM_SerializeFunctionInfo | 25.5us | 25.5us | 26.8us | +4.9% | +4.9% |

note: I used an LLM to help generate the test code adjustments and the
YAML traits.
2026-04-02 23:25:21 +00:00
Paul Kirth
69902769c7
[clang-doc] Introduce abstractions for pointer operations (#184872)
Since we're migrating from std::unique_ptr to raw pointers via
arena allocation, we want to have some interfaces that abstract
these operations away, and can be changed to keep the system
working without introducing a lot of unnecessary churn in the code.
2026-03-07 02:04:13 +00:00
Paul Kirth
263e3a3837
[clang-doc] Introduce type alias for OwningPtrVec/Array (#184871)
We commonly have vectors/arrays of owned pointers. This should simplify
future refactoring when switching to arena allocation.
2026-03-07 01:45:33 +00:00
Paul Kirth
110d3baa82
[clang-doc][NFC] Introduce OwnedPtr abstraction (#184869)
Eventually, we want clang-doc to support arena allocation, but the
widespread use of owning pointers in the data types prevents this.
Rather than have wide scale refactoring, we can introduce a type alias
that can be swapped out atomically to switch from smart pointers to raw
pointers. This is the first of several refactorings that are intended to
make the transition simpler.
2026-03-07 01:16:54 +00:00
Erick Velez
abee8a8e13
[clang-doc] Add a breadcrumb navigation bar (#173297)
This patch adds a breadcrumb navigation bar to the `<navbar>` element. Now, you can navigate between the different scopes of a record or namespace. This is done by keeping track of a Decl's parent Decl through its USR. That allows us to traverse the set of `Info`s through a directed graph during JSON generation to create `Context`s. A context is just a `Reference` that holds a relative path to a scope's file from a particular `Info`.
2026-01-07 13:35:35 -08:00
Erick Velez
7b73ba5162
[clang-doc] Add friends to class template (#173960)
This patch also allows comments to be associated with friend
declarations. Currently, it seems like the comments for friend `RecordDecl`
are taken from the actual class declaration, while a friend
function's comments are taken from the actual `friend` declaration.
2026-01-02 12:43:31 -08:00
Erick Velez
5a9f34f16b
[clang-doc] Add typedefs and aliases to templates (#173957)
This patch also adds optional template information to the typedef info
struct.
2026-01-02 10:15:45 -08:00
Erick Velez
1746af5b2e
[clang-doc] Add concepts to namespace template (#173956)
This patch serializes concepts in HTML. This patch also includes changes
to bitcode reading/writing and JSON to serialize the concept's location,
which was missing.
2026-01-02 09:45:10 -08:00
Paul Kirth
8126baa6d7
[clang-doc] Remove uses of consumeError (#168759)
In BitcodeReader, we were using consumeError(), which drops the error
and hides it from normal usage. To avoid that, we can just slightly
tweak the API to return an Expected<T>, and propagate the error
accordingly.
2025-12-04 17:00:48 -08:00
Paul Kirth
9349cb1523
[clang-doc] Use DiagnosticsEngine to handle diagnostic output (#170219)
[clang-doc] Use DiagnosticsEngine to handle diagnostic output

Right now we use a combination of outs() and errs() to handle tool
output. Instead, we can use existing diagnostic support in clang and
LLVM to ensure our tool has a consistent behavior with other tools.
2025-12-04 15:35:01 -08:00
Erick Velez
88055b3a56
[clang-doc][NFC] Remove unused headers (#168806)
Removes unused headers or replaces them with headers that directly
provide the symbol instead. For example, `Serialize.h` included `AST.h`,
but it was actually `Serialize.cpp` that needed concept expressions, so
now it includes just `ExprConcepts.h`.
2025-11-20 11:05:37 -08:00
Erick Velez
4db2f3ac89
[clang-doc] refactor JSON for better Mustache compatibility (#149588)
This patch contains changes for the JSON generator that will enable compatibility with Mustache templates, like booleans to check for the existence and bounds of arrays to avoid duplication.
2025-07-23 12:53:07 -07:00
Erick Velez
94bb9e12ec
[clang-doc] Serialize record files with mangled name (#148021)
This patch changes JSON file serialization. Now, files are serialized
to a single directory instead of nesting them based on namespaces. The
global namespace retains the "index.json" name.

This solves the problem of class template specializations being serialized to the
same file as its base template. This is also planned as part of
future integration with the Mustache generator which will consume the JSON files.
2025-07-11 13:39:41 -07:00
Erick Velez
0a02c28e29
[clang-doc] add namespace references to VarInfo (#146964)
VarInfo was missing its addReference specialization. This causes errors
when a VarInfo is inside a namespace that isn't the global namespace.
2025-07-03 15:50:23 -07:00
Erick Velez
a68e4470c1
[clang-doc] serialize friends (#146165)
Parse friends into a new FriendInfo and serialize them in JSON. We keep track of the friend declaration's template and function information if applicable.
2025-06-30 12:43:52 -07:00
Erick Velez
6d817810da
[clang-doc] serialize IsBuiltIn and IsTemplate for types (#146149)
IsBuiltIn and IsTemplate were being emitted as their default values.
2025-06-27 16:17:39 -07:00
Erick Velez
ab1e4d55d8
[clang-doc] refactor BitcodeReader::readSubBlock (#145835)
Reduce boilerplate code in readSubBlock by creating a callable from a higher-order lambda based on the block's add need.
2025-06-26 20:44:14 -07:00
Erick Velez
b8ea65025d
[clang-doc] document global variables (#145070)
Visit and map VarDecls to document variables declared in namespace scope.
2025-06-24 11:36:31 -07:00
Erick Velez
8050a6e073
[clang-doc] add support for concepts (#144430)
Add support for documenting concepts. This handles concepts and constraints on function and class templates.

Atomic constraints are not considered yet. We don't order constraints based on their conjunctive or disjunctive properties.
2025-06-20 17:39:31 -07:00
Erick Velez
7f69cd578d
[clang-doc] remove default label on some switches (#143919)
LLVM style prefers no default label on fully covered switches to warn if
new enums are added. This patch removes the default label for that
purpose or uses IT_default instead of default if that was the only enum
not covered.
2025-06-13 16:35:30 -07:00
Samarth Narang
d5704097fc
Refactor clang doc comment structure (#142273)
This patch refactors CommentKind handling in clang-doc by introducing a
strongly typed enum class for better type safety and clarity. It updates
all relevant places, including YAML traits and serialization, to work
with the new enum. Additionally, it enhances the Mustache-based HTML
generation by fully supporting all comment kinds, ensuring accurate
structured rendering of comment blocks. The changes simplify future
maintenance, improve robustness by eliminating unchecked defaults, and
ensure consistency between generators.

Fixes https://github.com/llvm/llvm-project/issues/142083
2025-06-07 08:04:56 -07:00
Paul Kirth
a56861777a
[clang-doc] Reenable time trace support (#141139)
This patch re-enables -ftime-trace support in clang-doc. Initial support
in #97644 was reverted, and never relanded. This patch adds back the
command line option, and leverages the RAII tracing infrastructure more
thoroughly.
2025-06-03 09:56:25 -07:00
Paul Kirth
97ec3ef4ab
[clang-doc] Add start and end line numbers (#137732)
This patch adds start and end line numbers to clang-doc. Currently
clang-doc only encodes the start line numbers of records, struct, etc.
This patch adds start and end line number to clang-doc bitcode which is
passed to the generator. This will be used by the mustache backend to
generate line ranges. Based on the orginal patch in #135081.

Co-author: Paul Kirth <paulkirth@google.com>

Co-authored-by: PeterChou1 <peter.chou@mail.utoronto.ca>
2025-04-29 14:57:35 -07:00
Kazu Hirata
90de46c51b [clang-doc] Fix warnings
This patch fixes:

  clang-tools-extra/clang-doc/BitcodeReader.cpp:47:20: error: unused
  function 'decodeRecord' [-Werror,-Wunused-function]

  clang-tools-extra/clang-doc/BitcodeReader.cpp:437:31: error: unused
  function 'getCommentInfo<std::unique_ptr<clang::doc::CommentInfo>
  &>' [-Werror,-Wunused-function]
2025-04-19 00:39:40 -07:00
Paul Kirth
c274bbed6a
[clang-doc][NFC] Prefer static functions for internal APIs (#136391)
Additionally moving the declarations from Generator.cpp
prevents misc-use-internal-linkage warnings from clang-tidy.
We fix those here too, since the static functions are part
of the same diagnostic
2025-04-18 23:34:57 -07:00
Paul Kirth
c5c285178c
[clang-doc][NFC] Avoid else after return (#136389) 2025-04-18 23:34:44 -07:00
Paul Kirth
8376edb143
[clang-doc][NFC] Avoid C-style casts (#136390) 2025-04-18 23:34:32 -07:00
Paul Kirth
5c3789811f
[clang-doc] Handle static members and functions (#135457)
clang-doc didn't visit VarDecl, and hence never collected info
from class statics members and functions.

Fixes #59813.
2025-04-18 18:08:06 -07:00
PeterChou1
5ef2456a43
[clang-doc] add support for enums comments in html generation (#101282)
Part of https://github.com/llvm/llvm-project/issues/101129

This patch adds support for attaching comments to enums for HTML in
clang-doc. It changes the enum generation to table tags where as
perviously we're using lists which is more in line with what other doc
generators are doing. It also gives clang-doc the ability to show user
specified enum values
2024-08-16 18:37:17 -04:00
PeterChou1
80d1c6acc0
Revert "[clang-doc] add ftime profiling" (#100251)
This broke the build due to failing some clang-tidy checks:
https://lab.llvm.org/buildbot/#/builders/145/builds/789
2024-07-23 18:31:11 -04:00
PeterChou1
7868c04d97
[clang-doc] add ftime profiling (#97644)
this patches adds ftime-tracing option for clang-doc which was what I
used to profile the performance problems we had with clang-doc
generating the llvm docs.
- ftime-trace produces a time profiler json file called
clang-doc-tracing.json
2024-07-23 17:45:42 -04:00
Kazu Hirata
2b76bdc33b [clang-doc] Fix a warning
This patch fixes:

  clang-tools-extra/clang-doc/BitcodeReader.cpp:78:3: error: default
  label in switch which covers all enumeration values
  [-Werror,-Wcovered-switch-default]
2023-11-03 11:03:15 -07:00
Vlad Serebrennikov
edd690b02e
[clang][NFC] Refactor TagTypeKind (#71160)
This patch converts TagTypeKind into scoped enum. Among other benefits,
this allows us to forward-declare it where necessary.
2023-11-03 21:45:39 +04:00
Brett Wilson
4a68babd99 [clang-doc] Add template support.
Reads template information from the AST and adds template parameters and
specialization information to the corresponding clang-doc structures.

Add a "QualName" to the Reference struct which includes the full
qualified type name. The Reference object represents a link in the
HTML/MD generators so is based on the unqualified name. But this does
not encode C-V qualifiers or template information that decorate the
name. The new QualName member encodes all of this information and also
makes it easier for the generators or downsteam YAML consumers to
generate the full name (before they had to process the "Path").

In test code that was changed, remove made-up paths to built-in types
like "int". In addition to slightnly cleaning up the code, these types
do not have paths in real execution, and generating incorrect references
to nonexistant data may complicate future changes in the generators.

Convert llvm::Optional to std::optional (YAML library requires this for
the new usage, and this makes everything consistent according to the
llvm::Optional -> std::optional transition).

Differential Revision: https://reviews.llvm.org/D139154
2022-12-08 08:02:02 -08:00
Krzysztof Parzyszek
49e75ebd85 [Bitcode(Reader|Writer)] Convert Optional to std::optional 2022-12-07 15:27:38 -08:00
Brett Wilson
91b38c6aad Revert "[clang-doc] Add template support."
Causes a build failure in YAML specializations.

This reverts commit 0f6dbb5f164662c3e6a167a89e7a89f07c60e32b.
2022-12-07 10:22:51 -08:00
Brett Wilson
0f6dbb5f16 [clang-doc] Add template support.
Reads template information from the AST and adds template parameters and
specialization information to the corresponding clang-doc structures.

Add a "QualName" to the Reference struct which includes the full
qualified type name. The Reference object represents a link in the
HTML/MD generators so is based on the unqualified name. But this does
not encode C-V qualifiers or template information that decorate the
name. The new QualName member encodes all of this information and also
makes it easier for the generators or downsteam YAML consumers to
generate the full name (before they had to process the "Path").

In test code that was changed, remove made-up paths to built-in types
like "int". In addition to slightnly cleaning up the code, these types
do not have paths in real execution, and generating incorrect references
to nonexistant data may complicate future changes in the generators.

Differential Revision: https://reviews.llvm.org/D139154
2022-12-07 09:48:13 -08:00
Krzysztof Parzyszek
c589730ad5 [YAML] Convert Optional to std::optional 2022-12-06 12:49:32 -08:00
Brett Wilson
7231c9966e [clang-doc] Fix typedef/using output.
Provides an initializer for the TypedefInfo.IsUsing member. Previously
this member was uninitialized and would produce random output.

Adds the Description (code comments) to the bitcode reader/writer.
Previously the typedef/using descriptions were lost during the bitcode
round-trip. Adds a test for this.

Differential Revision: https://reviews.llvm.org/D136638
2022-10-25 13:47:24 -07:00
Brett Wilson
21fb70c6ab [clang-doc] Add typedef/using information.
Read typedef and "using" type alias declarations and serialize into the
internal structures. Emit this information in the YAML output. The HTML
and MD generators are unchanged.

Separate out the logic to create the parent namespace or record object
and insert the newly created child into it. This logic was previously
duplicated for every "info" type and is now shared.

To help this, a struct containing the child vectors was separated out so
children can be added generically and without having too many templates.

A small change was made to populateParentNamespaces() to allow using
types that aren't themselves DeclContexts (typedefs are the first
example of this).

Differential Revision: https://reviews.llvm.org/D134371
2022-10-14 14:59:29 -07:00
Haowei Wu
a759477222 Revert "[clang-doc] Add typedef/using information."
This reverts commit eed22583fd78d4d657fb70b99e62fbdc1f83b8f9, which
breaks clang build with -Werror.
2022-09-27 18:35:34 -07:00
Brett Wilson
eed22583fd [clang-doc] Add typedef/using information.
Read typedef and "using" type alias declarations and serialize into the internal structures. Emit this information in the YAML output. The HTML and MD generators are unchanged.

Separate out the logic to create the parent namespace or record object and insert the newly created child into it. This logic was previously duplicated for every "info" type and is now shared.

To help this, a struct containing the child vectors was separated out so children can be added generically and without having too many templates.

A small change was made to populateParentNamespaces() to allow using types that aren't themselves DeclContexts (typedefs are the first example of this).

Reviewed By: paulkirth, haowei

Differential Revision: https://reviews.llvm.org/D134371
2022-09-27 23:35:16 +00:00
Brett Wilson
0afc60858e [clang-doc] Clean up *Info constructors.
The *Info object (for the copy of the AST") constructors had many duplicated variants. Many of the variants seemed to be in an attempt to avoid default arguments. But default arguments are not prohibited and using them allows most of the variants to be removed which improves readability.

Remove the IsInGlobalNamespace flag on a Reference. This is set when the path is empty, and only read once in the HTML generator with the identical condition. The constructor cleanup exposed a problem where this was set to false when the constructor with no path was used, but true when the path was set to empty.

There should be no observable change with the exception that IsInGlobalNamespace is no longer emitted in YAML.

Reviewed By: paulkirth, haowei

Differential Revision: https://reviews.llvm.org/D134235
2022-09-27 23:31:41 +00:00
Brett Wilson
eaa7b324d5 [clang-doc] Add support for explicitly typed enums
Add support for explicitly typed enums:
  enum Foo : unsigned { ... };
to the internal representation and to the YAML output.

Add support for getting the value of an enum constant, as well as accessing the original expression that produced it. This changes the YAML output of enums from an array of strings for the enum members to an array of dictionaries. These dictionaries now report the name, value, and original expression.

The markdown and HTML outputs are unchanged, they still output the name from the new enhanced internal schema.

Reviewed By: paulkirth

Differential Revision: https://reviews.llvm.org/D134055
2022-09-19 21:54:41 +00:00
Brett Wilson
e191086bfc [clang-doc] Support default args for functions.
Adds support for default arguments in the internal representation and reads these values from the source. Implements writing these values to YAML but does not implement this for the HTML or markdown outputs.

Reviewed By: paulkirth

Differential Revision: https://reviews.llvm.org/D133732
2022-09-16 17:26:07 +00:00
Brett Wilson
99baa10f8f [clang-doc] Read docstrings for record members
Struct/class data members did not have the comments associated with
them. This adds that information to the MemberTypeInfo class and emits
it in the YAML. This does not update the frontends yet.

Reviewed By: paulkirth

Differential Revision: https://reviews.llvm.org/D131298
2022-08-11 17:14:15 +00:00
Simon Pilgrim
45617e1dd8 [clang-doc] Pass Record argument by const-ref. NFCI.
Record is a SmallVector<uint64_t, 1024> - we really need to avoid passing this by value.

Avoid unnecessary big copies, reported by coverity.
2021-09-24 10:06:13 +01:00
Fangrui Song
180380651c [clang-doc] Use llvm::createStringError and canonicalize error messages
"Bad block found.\n" -> "bad block found"

The lower cased form with no full stop or newline is more common in LLVM
tools.

Reviewed By: juliehockett

Differential Revision: https://reviews.llvm.org/D66783

llvm-svn: 370155
2019-08-28 02:56:03 +00:00
Diego Astiazaran
ba3d595f93 [clang-doc] Serialize inherited attributes and methods
clang-doc now serializes the inherited attributes and methods, not only the name of the base class.
All inherited are tracked, if B:A and C:B, info of A is included in C.
This data is stored in attribute Bases in a RecordInfo.
Previously tracked inheritance data, stored in Parents and VParents, hasn't been removed to reduce review load.

Differential revision: https://reviews.llvm.org/D66238

llvm-svn: 369075
2019-08-16 00:10:49 +00:00
Jonas Devlieghere
1c705d9c53 [clang-tools-extra] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368944
2019-08-14 23:52:23 +00:00