The patch reapply https://github.com/llvm/llvm-project/pull/173130.
This patch implement the following papers:
[P1857R3 Modules Dependency Discovery](https://wg21.link/p1857r3).
[P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1).
[CWG2947](https://cplusplus.github.io/CWG/issues/2947.html).
At the start of phase 4 an import or module token is treated as starting
a directive and are converted to their respective keywords iff:
- After skipping horizontal whitespace are
- at the start of a logical line, or
- preceded by an export at the start of the logical line.
- Are followed by an identifier pp token (before macro expansion), or
- <, ", or : (but not ::) pp tokens for import, or
- ; for module
Otherwise the token is treated as an identifier.
Additionally:
- The entire import or module directive (including the closing ;) must
be on a single logical line and for module must not come from an
#include.
- The expansion of macros must not result in an import or module
directive introducer that was not there prior to macro expansion.
- A module directive may only appear as the first preprocessing tokens
in a file (excluding the global module fragment.)
- Preprocessor conditionals shall not span a module declaration.
After this patch, we handle C++ module-import and module-declaration as
a real pp-directive in preprocessor. Additionally, we refactor module
name lexing, remove the complex state machine and read full module name
during module/import directive handling. Possibly we can introduce a
tok::annot_module_name token in the future, avoid duplicatly parsing
module name in both preprocessor and parser, but it's makes error
recovery much diffcult(eg. import a; import b; in same line).
This patch also introduce 2 new keyword `__preprocessed_module` and
`__preprocessed_import`. These 2 keyword was generated during `-E` mode.
This is useful to avoid confusion with `module` and `import` keyword in
preprocessed output:
```cpp
export module m;
struct import {};
#define EMPTY
EMPTY import foo;
```
Fixes https://github.com/llvm/llvm-project/issues/54047
The previous patch has an use-after-free issue in
Lexer::LexTokenInternal function. Since C++20, the `export`, `import`
and `module` identifiers may be an introducer of a C++ module
declaration/importing directive, and the directive will handled in
`LexIdentifierContinue`. Unfortunately, the EOF may be encountered in
`LexIdentifierContinue` and `CurLexer` might be destructed in
`HandleEndOfFile`, If the code after `LexIdentifierContinue` try to
access `LangOps` or other class members in this Lexer, it will hit
undefined behavior.
This patch also fix the use-after-free issue in Lexer by introduce a
mechanism to delay the destruction of `CurLexer` in `Preprocessor`
class.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
This PR implement the following papers:
[P1857R3 Modules Dependency Discovery](https://wg21.link/p1857r3).
[P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1).
[CWG2947](https://cplusplus.github.io/CWG/issues/2947.html).
At the start of phase 4 an import or module token is treated as starting
a directive and are converted to their respective keywords iff:
- After skipping horizontal whitespace are
- at the start of a logical line, or
- preceded by an export at the start of the logical line.
- Are followed by an identifier pp token (before macro expansion), or
- <, ", or : (but not ::) pp tokens for import, or
- ; for module
Otherwise the token is treated as an identifier.
Additionally:
- The entire import or module directive (including the closing ;) must
be on a single logical line and for module must not come from an
#include.
- The expansion of macros must not result in an import or module
directive introducer that was not there prior to macro expansion.
- A module directive may only appear as the first preprocessing tokens
in a file (excluding the global module fragment.)
- Preprocessor conditionals shall not span a module declaration.
After this patch, we handle C++ module-import and module-declaration as
a real pp-directive in preprocessor. Additionally, we refactor module
name lexing, remove the complex state machine and read full module name
during module/import directive handling. Possibly we can introduce a
tok::annot_module_name token in the future, avoid duplicatly parsing
module name in both preprocessor and parser, but it's makes error
recovery much diffcult(eg. import a; import b; in same line).
This patch also introduce 2 new keyword `__preprocessed_module` and
`__preprocessed_import`. These 2 keyword was generated during `-E` mode.
This is useful to avoid confusion with `module` and `import` keyword in
preprocessed output:
```cpp
export module m;
struct import {};
#define EMPTY
EMPTY import foo;
```
Fixes https://github.com/llvm/llvm-project/issues/54047
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
Signed-off-by: Wang, Yihan <yronglin777@gmail.com>
lldb's CPlusPlusNameParser is currently identifying keywords using it's
own map implemented using clang/Basic/TokenKinds.def. However, it does
not respect the language options so identifiers can incorrectly
determined to be keywords when using languages in which they are not
keywords.
Rather than implement the logic to enable/disable keywords in both
projects it makes sense for lldb to use clang's implementation.
See #164284 for more information
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
This adds a new diagnostic group, -Wc++-keyword, which is off by default
and grouped under -Wc++-compat. The diagnostic catches use of C++
keywords in C code.
This change additionally fixes an issue with -Wreserved-identifier not
diagnosing use of reserved identifiers in function parameter lists in a
function declaration which is not a definition.
Fixes https://github.com/llvm/llvm-project/issues/21898
It would be nice to see what our users think about this change, as this
is something that WG21/EWG quite wants to fix a handful of questionable
issues with UB. Depending on the outcome of this after being committed,
we might instead suggest EWG undeprecate this, and require a bit of
'magic' from the lexer.
Additionally, this patch makes it so we emit this diagnostic ALSO in
cases where the literal name is reserved. It doesn't make sense to limit
that.
---------
Co-authored-by: Vlad Serebrennikov <serebrennikov.vladislav@gmail.com>
Enabling __ptr32 keyword to support in Clang for z/OS. It is represented
by addrspace(1) in LLVM IR. Unlike existing implementation, __ptr32 is
not mangled into symbol names for z/OS.
This PR implement [P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1), and refactor the convoluted state
machines in module name lexical analysis.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
This commit implements the entirety of the now-accepted [N3017
-Preprocessor
Embed](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3017.htm) and
its sister C++ paper [p1967](https://wg21.link/p1967). It implements
everything in the specification, and includes an implementation that
drastically improves the time it takes to embed data in specific
scenarios (the initialization of character type arrays). The mechanisms
used to do this are used under the "as-if" rule, and in general when the
system cannot detect it is initializing an array object in a variable
declaration, will generate EmbedExpr AST node which will be expanded by
AST consumers (CodeGen or constant expression evaluators) or expand
embed directive as a comma expression.
This reverts commit
682d461d5a.
---------
Co-authored-by: The Phantom Derpstorm <phdofthehouse@gmail.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
Co-authored-by: H. Vetinari <h.vetinari@gmx.com>
This commit implements the entirety of the now-accepted [N3017 -
Preprocessor
Embed](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3017.htm) and
its sister C++ paper [p1967](https://wg21.link/p1967). It implements
everything in the specification, and includes an implementation that
drastically improves the time it takes to embed data in specific
scenarios (the initialization of character type arrays). The mechanisms
used to do this are used under the "as-if" rule, and in general when the
system cannot detect it is initializing an array object in a variable
declaration, will generate EmbedExpr AST node which will be expanded
by AST consumers (CodeGen or constant expression evaluators) or
expand embed directive as a comma expression.
---------
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
Co-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Podchishchaeva, Mariya <mariya.podchishchaeva@intel.com>
This patch expands notion of "interesting" in `IdentifierInto` it to
also cover ObjC keywords and builtins, which matches notion of
"interesting" in serialization layer. What was previously "interesting"
in `IdentifierInto` is now called "notable".
Beyond clearing confusion between serialization and the rest of the
compiler, it also resolved a naming problem: ObjC keywords, notable
identifiers, and builtin IDs are all stored in the same bit-field. Now
we can use "interesting" to name it and its corresponding type, instead
of `ObjCKeywordOrInterestingOrBuiltin` abomination.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
Prior to this, clang would always report
```
compile with '-ffixed-point' to enable fixed point types
```
whenever it sees `_Accum`, `_Fract`, or `_Sat` when fixed point
arithmetic is not enabled. This can break existing code that uses these
as variable names and doesn't use fixed point arithmetic like in some
microsoft headers
(https://github.com/llvm/llvm-project/pull/67750#issuecomment-1775264907).
Fixed point should not raise this error for these cases, so this removes
the error altogether and defaults to the usual error clang gives where
it can see these keywords as either unknown types or regular variables.
Refactor `uintptr_t` inside of `clang::Selector` that holds a pointer to `IdentifierInfo` or `MultiKeywordSelector` and `IdentifierInfoFlag` enum into `PointerIntPair`. This is a part of `PointerIntPair` migration outlined in https://github.com/llvm/llvm-project/issues/69835.
Unlike `uintpt_t`, `PointerIntPair` required pointee types to be complete, so I had to shuffle definitions of `MultiKeywordSelector` and `detail::DeclarationNameExtra` around to make them complete at `Selector`. Also, there were outdated specializations of `PointerLikeTypeTraits` for `IdentifierInfo *`, which are no longer needed, because `alignof` that primary template use works just fine. Not just that, but they declared that `IdentifierInfo *` has only 1 spare lower bit, but today they are 8-byte aligned.
This does the rename for most internal uses of C2x, but does not rename
or reword diagnostics (those will be done in a follow-up).
I also updated standards references and citations to the final wording
in the standard.
1. Teach -Wuser-defined-literals to warn on using double underscores in
literal suffix identifiers.
2. Add -Wdeprecated-literal-operator to warn about the use of the first
grammar production of literal-operator-id, which defaults to off for now.
Differential Revision: https://reviews.llvm.org/D152632
This reverts commit 0a532207b8696d81e46017f444bd2257347f129b.
This breaks several of our tests. Have given reproducers to author.
Reverting this until author can fix the issue.
This is more useful for debug/test than getNullabilitySpelling:
- default form has uglifying underscores
- non-default form crashes on NullableResult
- both return unhelpfully verbose strings for Unspecified
- operator<< works with gtest, formatv, etc
Differential Revision: https://reviews.llvm.org/D149650
As the diagnostic message shows, we should remove -fmodules-ts flag in
clang/llvm17. Since clang/llvm16 is already branched. We can remove the
depreacared flag now.
This is first part for support cbuffer/tbuffer.
The format for cbuffer/tbuffer is
BufferType [Name] [: register(b#)] { VariableDeclaration [: packoffset(c#.xyzw)]; ... };
More details at https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-constants
New keyword 'cbuffer' and 'tbuffer' are added.
New AST node HLSLBufferDecl is added.
Build AST for simple cbuffer/tbuffer without attribute support.
The special thing is variables declared inside cbuffer is exposed into global scope.
So isTransparentContext should return true for HLSLBuffer.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D129883
This addresses an accidental change in behavior from
41667a8b9b624e282e7c08fadf7091223728d1c1 to get the bots back to green.
However, I think there's an issue with LLDB assuming it's valid to
enable support for keywords in language modes that don't support the
keyword (as other parts of Clang are not expecting to be able to do
that).
This should fix (and others):
https://lab.llvm.org/buildbot/#/builders/68/builds/38374
The patch diagnoses an identifier as a future keyword if it exists in a
future language mode, such as:
int restrict;
in C modes earlier than C99. We now give a warning to the user that
such an identifier is a future keyword. Handles keywords from C as well
as C++.
Differential Revision: https://reviews.llvm.org/D131683
When doing that NFC refactor, I'd messed up how char8_t was reported,
which resulted in it being considered a 'future' keyword, without the
corresponding diagnostic, which lead to an assert. This patch corrects
the char8_t to ONLY be future in C++ mode.
The getKeywordStatus function is a horrible mess of inter-dependent 'if'
statements that depend significantly on the ORDER of the checks. This
patch removes the dependency on order by checking each set-flag only
once.
It does this by looping through each of the set bits, and checks each
individual flag for its effect, then combines them at the end.
This might slow down startup performance slightly, as there are only a
few hundred keywords, and a vast majority will only get checked 1x
still.
This patch ALSO removes the KEYWORD_CONCEPTS flag, because it has since
become synonymous with C++20.
Differential Revision: https://reviews.llvm.org/D131007
Update KEYALL to cover KEYCUDA. Introduce KEYMAX and
a generic way to update KEYALL.
Reviewed by: Dan Liew
Differential Revision: https://reviews.llvm.org/D125396
CUDA/HIP programs use __noinline__ like a keyword e.g.
__noinline__ void foo() {} since __noinline__ is defined
as a macro __attribute__((noinline)) in CUDA/HIP runtime
header files.
However, gcc and clang supports __attribute__((__noinline__))
the same as __attribute__((noinline)). Some C++ libraries
use __attribute__((__noinline__)) in their header files.
When CUDA/HIP programs include such header files,
clang will emit error about invalid attributes.
This patch fixes this issue by supporting __noinline__ as
a keyword, so that CUDA/HIP runtime could remove
the macro definition.
Reviewed by: Aaron Ballman, Artem Belevich
Differential Revision: https://reviews.llvm.org/D124866
Underscore-uglified identifiers are used in standard library implementations to
guard against collisions with macros, and they hurt readability considerably.
(Consider `push_back(Tp_ &&__value)` vs `push_back(Tp value)`.
When we're describing an interface, the exact names of parameters are not
critical so we can drop these prefixes.
This patch adds a new PrintingPolicy flag that can applies this stripping
when recursively printing pieces of AST.
We set it in code completion/signature help, and in clangd's hover display.
All three features also do a bit of manual poking at names, so fix up those too.
Fixes https://github.com/clangd/clangd/issues/736
Differential Revision: https://reviews.llvm.org/D116387
The original version of this was reverted, and @rjmcall provided some
advice to architect a new solution. This is that solution.
This implements a builtin to provide a unique name that is stable across
compilations of this TU for the purposes of implementing the library
component of the unnamed kernel feature of SYCL. It does this by
running the Itanium mangler with a few modifications.
Because it is somewhat common to wrap non-kernel-related lambdas in
macros that aren't present on the device (such as for logging), this
uniquely generates an ID for all lambdas involved in the naming of a
kernel. It uses the lambda-mangling number to do this, except replaces
this with its own number (starting at 10000 for readabililty reasons)
for lambdas used to name a kernel.
Additionally, this implements itself as constexpr with a slight catch:
if a name would be invalidated by the use of this lambda in a later
kernel invocation, it is diagnosed as an error (see the Sema tests).
Differential Revision: https://reviews.llvm.org/D103112
WG14 adopted N2645 and WG21 EWG has accepted P2334 in principle (still
subject to full EWG vote + CWG review + plenary vote), which add
support for #elifdef as shorthand for #elif defined and #elifndef as
shorthand for #elif !defined. This patch adds support for the new
preprocessor directives.
Warn when a declaration uses an identifier that doesn't obey the reserved
identifier rule from C and/or C++.
Differential Revision: https://reviews.llvm.org/D93095
_Nullable_result generally like _Nullable, except when being imported into a
swift async method. rdar://70106409
Differential revision: https://reviews.llvm.org/D92495
