A CMake change included in CMake 4.0 makes `AIX` into a variable
(similar to `APPLE`, etc.)
ff03db6657
However, `${CMAKE_SYSTEM_NAME}` unfortunately also expands exactly to
`AIX` and `if` auto-expands variable names in CMake. That means you get
a double expansion if you write:
`if (${CMAKE_SYSTEM_NAME} MATCHES "AIX")`
which becomes:
`if (AIX MATCHES "AIX")`
which is as if you wrote:
`if (ON MATCHES "AIX")`
You can prevent this by quoting the expansion of "${CMAKE_SYSTEM_NAME}",
due to policy
[CMP0054](https://cmake.org/cmake/help/latest/policy/CMP0054.html#policy:CMP0054)
which is on by default in 4.0+. Most of the LLVM CMake already does
this, but this PR fixes the remaining cases where we do not.
Followup work of #140498 to continue the work on clangd/clangd#529
Introduce the use of the Clang doxygen parser to parse the documentation
of hovered code.
- ASTContext independent doxygen parsing
- Parsing doxygen commands to markdown for hover information
Note: after this PR I have planned another patch to rearrange the
information shown in the hover info.
This PR is just for the basic introduction of doxygen parsing for hover
information.
---------
Co-authored-by: Maksim Ivanov <emaxx@google.com>
If a add_clang_library call doesn't specify building as static or shared
library they are implicitly added to the list static libraries that is
linked in to clang-cpp shared library here.
315ba77406/clang/cmake/modules/AddClang.cmake (L107)
Because the clang-tools-extra libraries targets were declared after
clang-cpp they by luck never got linked to clang-cpp.
This change is required for clang symbol visibility macros on windows to
work correctly for clang tools since we need to distinguish if a target
being built will be importing or exporting clang symbols from the
clang-cpp DLL.
Alternatives to https://reviews.llvm.org/D153114.
Try to address https://github.com/clangd/clangd/issues/1293.
See the links for design ideas and the consensus so far. We want to have
some initial support in clang18.
This is the initial support for C++20 Modules in clangd.
As suggested by sammccall in https://reviews.llvm.org/D153114,
we should minimize the scope of the initial patch to make it easier
to review and understand so that every one are in the same page:
> Don't attempt any cross-file or cross-version coordination: i.e. don't
> try to reuse BMIs between different files, don't try to reuse BMIs
> between (preamble) reparses of the same file, don't try to persist the
> module graph. Instead, when building a preamble, synchronously scan
> for the module graph, build the required PCMs on the single preamble
> thread with filenames private to that preamble, and then proceed to
> build the preamble.
This patch reflects the above opinions.
# Testing in real-world project
I tested this with a modularized library:
https://github.com/alibaba/async_simple/tree/CXX20Modules. This library
has 3 modules (async_simple, std and asio) and 65 module units. (Note
that a module consists of multiple module units). Both `std` module and
`asio` module have 100k+ lines of code (maybe more, I didn't count). And
async_simple itself has 8k lines of code. This is the scale of the
project.
The result shows that it works pretty well, ..., well, except I need to
wait roughly 10s after opening/editing any file. And this falls in our
expectations. We know it is hard to make it perfect in the first move.
# What this patch does in detail
- Introduced an option `--experimental-modules-support` for the support
for C++20 Modules. So that no matter how bad this is, it wouldn't affect
current users. Following off the page, we'll assume the option is
enabled.
- Introduced two classes `ModuleFilesInfo` and
`ModuleDependencyScanner`. Now `ModuleDependencyScanner` is only used by
`ModuleFilesInfo`.
- The class `ModuleFilesInfo` records the built module files for
specific single source file. The module files can only be built by the
static member function `ModuleFilesInfo::buildModuleFilesInfoFor(PathRef
File, ...)`.
- The class `PreambleData` adds a new member variable with type
`ModuleFilesInfo`. This refers to the needed module files for the
current file. It means the module files info is part of the preamble,
which is suggested in the first patch too.
- In `isPreambleCompatible()`, we add a call to
`ModuleFilesInfo::CanReuse()` to check if the built module files are
still up to date.
- When we build the AST for a source file, we will load the built module
files from ModuleFilesInfo.
# What we need to do next
Let's split the TODOs into clang part and clangd part to make things
more clear.
The TODOs in the clangd part include:
1. Enable reusing module files across source files. The may require us
to bring a ModulesManager like thing which need to handle `scheduling`,
`the possibility of BMI version conflicts` and `various events that can
invalidate the module graph`.
2. Get a more efficient method to get the `<module-name> ->
<module-unit-source>` map. Currently we always scan the whole project
during `ModuleFilesInfo::buildModuleFilesInfoFor(PathRef File, ...)`.
This is clearly inefficient even if the scanning process is pretty fast.
I think the potential solutions include:
- Make a global scanner to monitor the state of every source file like I
did in the first patch. The pain point is that we need to take care of
the data races.
- Ask the build systems to provide the map just like we ask them to
provide the compilation database.
3. Persist the module files. So that we can reuse module files across
clangd invocations or even across clangd instances.
TODOs in the clang part include:
1. Clang should offer an option/mode to skip writing/reading the bodies
of the functions. Or even if we can requrie the parser to skip parsing
the function bodies.
And it looks like we can say the support for C++20 Modules is initially
workable after we made (1) and (2) (or even without (2)).
A prototype of using include-cleaner library in clangd:
- (re)implement clangd's "unused include" warnings with the library
- the new implementation is hidden under a flag `Config::UnusedIncludesPolicy::Experiment`
Differential Revision: https://reviews.llvm.org/D140875
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
Make it possible to disable building the decision forest ranking
model for clangd. To unbreak build of Clangd on PPC32 in gentoo, see
https://bugs.gentoo.org/829602
Based on D138520.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D139107
As pointed out in https://reviews.llvm.org/D119130#3829816, this
introduces a clang AST dependency to the clangToolingInclusions, which is used
by clang-format.
Since rest of the inclusion tooling doesn't depend on clang ast, moving this
into a separate library.
Differential Revision: https://reviews.llvm.org/D135245
This first version only uses bracket matching. We plan to extend this to
use DirectiveTree as well.
Also includes changes to Token to allow retrieving corresponding token
in token stream of original source file.
Differential Revision: https://reviews.llvm.org/D129648
Tracked down the crash, which was argument-evaluation-order UB
in the wrapping indexStandardLibrary().
Sorry for the churn!
This reverts commit 77533ea443aca6e9978d7c8a6822420f8345f6af.
This reverts commit ccdb56ac10eef3048135169a67d239328c2b1de6.
Still seeing windows failures on GN bots: http://45.33.8.238/win/58316/step_9.txt
Unfortunately I can't debug these at all - it's a bare unsymbolized
stacktrace, and I can't reproduce the failure.
This provides a nice "warm start" with all headers indexed, not just
those included so far.
The standard library is indexed after a preamble is parsed, using that
file's configuration. The result is pushed into the dynamic index.
If we later see a higher language version, we reindex it.
It's configurable as Index.StandardLibrary, off by default for now.
Based on D105177 by @kuhnel
Fixes https://github.com/clangd/clangd/issues/618
Differential Revision: https://reviews.llvm.org/D115232
This is the first patch in an ongoing attempt of Include Cleaner: unused/missing
headere diagnostics, an IWYU-like functionality implementation for clangd. The
work is split into (mostly) distinct and parallelizable pieces:
- Finding all referenced locations (this patch).
- Finding all referenced locations of macros.
- Building IncludeGraph and marking headers as unused, used and directly used.
- Making use of the introduced library and add an option to use in clangd.
---
* Adding support for standard library headers (possibly through mapping
genfiles).
Based on https://reviews.llvm.org/D100540.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D105426
This reduces the size of the dependency graph and makes incremental
development a little more pleasant (less rebuilding).
This introduces a bit of complexity/fragility as some tests verify
clang-tidy behavior. I attempted to isolate these and build/run as much
of the tests as possible in both configs to prevent rot.
Expectation is that (some) developers will use this locally, but
buildbots etc will keep testing clang-tidy.
Fixes https://github.com/clangd/clangd/issues/233
Differential Revision: https://reviews.llvm.org/D105679
For files directly under clangd/, -Iclang-tools-extra/clangd (and the
equivalent for generated files) are not required, as CMake/the compiler puts
these directories on the include path by default.
However this means each subdirectory needs to
include_directories(.. ${CMAKE_CURRENT_BINARY_DIR}/..) etc, and this
proved annoying and error-prone to maintain and debug.
Since include_directories is inherited by subdirectories, we just
configure this explicitly at the top level instead.
As pointed out in D96244, "Module" is already pretty overloaded to refer
to clang and llvm modules. (And clangd deals directly with the former).
FeatureModule is a bit of a mouthful but it's pretty self-descriptive.
I think it might be better than "Component" which doesn't really capture
the "common interface" aspect - it's IMO confusing to refer to
"components" but exclude CDB for example.
Differential Revision: https://reviews.llvm.org/D97950
The patch also does some cleanup on the interface of the entry
points from TargetFinder into the heuristic resolution code.
Since the heuristic resolver is created in a place where the
ASTContext is available, it can store the ASTContext and the
NameFactory hack can be removed.
Differential revision: https://reviews.llvm.org/D92290
Many useful signals can be derived from a valid AST which is regularly updated by
the ASTWorker. `runWithPreamble` does not have access to the ParsedAST
but it can be provided access to some signals derived from a (possibly
stale) AST.
Differential Revision: https://reviews.llvm.org/D94424
This diff addresses the issue of the ever increasing memory usage of clangd. The key to understand what happens is to use `malloc_stats()`: malloc arenas keep getting bigger, although the actual memory used does not. It seems some operations while bulding the indices (both dynamic and background) create this problem. Specifically, 'FileSymbols::update' and 'FileSymbols::buildIndex' seem especially affected.
This diff adds a call to `malloc_trim()` periodically in
ClangdLSPServer.
Fixes: https://github.com/clangd/clangd/issues/251
Fixes: https://github.com/clangd/clangd/issues/115
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D93452
Added some new ClangTidyOptionsProvider like classes designed for clangd work flow.
These providers are designed to source the options on the worker thread but in a thread safe manner.
This is done through making the options getter take a pointer to the filesystem used by the worker thread which natuarally is from a ThreadsafeFS.
Internal caching in the providers is also guarded.
The providers don't inherit from `ClangTidyOptionsProvider` instead they share a base class which is able to create a provider for the `ClangTidyContext` using a specific FileSystem.
This approach means one provider can be used for multiple contexts even though `ClangTidyContext` owns its provider.
Depends on D90531
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D91029
D71880 makes this dependency redundant and we can safely remove it. Tested for
both shared lib build and static lib build.
Reviewed By: hokein
Differential Revision: https://reviews.llvm.org/D91951
Put project-aware-index between command-line specified static index and
ClangdServer indexes.
This also moves remote-index dependency from clangDaemon to ClangdMain
in an attempt to prevent cyclic dependency between clangDaemon and
remote-index-marshalling.
Differential Revision: https://reviews.llvm.org/D91860
An index implementation that can dispatch to a variety of indexes
depending on the file path. Enables clangd to work with multiple indexes in the
same instance, configured via config files.
Depends on D90749, D90746
Differential Revision: https://reviews.llvm.org/D90750
This is a mass-market version of the "dump AST" tweak we have behind
-hidden-features.
I think in this friendlier form it'll be useful for people outside clang
developers, which would justify making it a real feature.
It could be useful as a step towards lightweight clang-AST tooling in clangd
itself (like matcher-based search).
Advantages over the tweak:
- simplified information makes it more accessible, likely somewhat useful
without learning too much clang internals
- can be shown in a tree view
- structured information gives some options for presentation (e.g.
icon + two text colors + tooltip in vscode)
- clickable nodes jump to the corresponding code
Disadvantages:
- a bunch of code to handle different node types
- likely missing some important info vs dump-ast due to brevity/oversight
- may end up chasing/maintaining support for the long tail of nodes
Demo with VSCode support: https://imgur.com/a/6gKfyIV
Differential Revision: https://reviews.llvm.org/D89571
RemoteIndexClient implementations only depends on clangdSupport for
logging functionality and has no dependence on clangDeamon itself. This clears
out that link time dependency and enables depending on it in clangDeamon itself,
so that we can have other index implementations that makes use of the
RemoteIndex.
Differential Revision: https://reviews.llvm.org/D90746
We intend to replace heuristics based code completion ranking with a Decision Forest Model.
This patch introduces a format for representing the model and an inference runtime that is code-generated at build time.
- Forest.json contains all the trees as an array of trees.
- Features.json describes the features to be used.
- Codegen file takes the above two files and generates CompletionModel containing Feature struct and corresponding Evaluate function.
The Evaluate function maps a feature to a real number describing the relevance of this candidate.
- The codegen is part of build system and these files are generated at build time.
- Proposes a way to test the generated runtime using a test model.
- Replicates the model structure in unittests.
- unittest tests both the test model (for correct tree traversal) and the real model (for sanity).
This reverts commit 549e55b3d5634870aa9d42135f51ad46a6a0e347.
