This patch adds logic to canonicalize `-include-pch`'s input in the
frontend. This way, the `ASTWriter` always serializes the canonicalized
path to the included pch file whether the input is an absolute path or a
relative path.
Fixes rdar://168596546.
fixes#159438
This patch adds `MatrixElementExpr`, a new AST node for HLSL matrix
element and swizzle access (e.g. M._m00, M._11_22_33).
It introduces a shared `ElementAccessExprBase` used by both matrix and
vector swizzle expressions, updates Sema to parse and validate
zero-based and one-based accessors, detects duplicates for l-value
checks, and emits improved diagnostics. CodeGen is updated to lower
scalar and multi-element accesses consistently, and full AST
serialization, dumping, and tooling support is included. This
implementation reflects the updated
[RFC](https://github.com/llvm/wg-hlsl/pull/357/files) for HLSL matrix
accessor semantics.
(After changing the scope) This PR implements parsing the reflection
operator (^^) for primitive types. The goal is to keep the first PR
simple. In subsequent PRs, parsing for the remaining requirements will
be introduced.
This implementation is based on the fork of @katzdm.
Class `CXXReflectExpr` is introduced to represent the operand of the
reflection operator. For now, in this PR, the type std::meta::info is
not implemented yet, so when we construct an AST node CXXReflectExpr,
`VoidTy` is used as placeholder type for now.
The file `ParseReflect.cpp` is introduced, which for now only has the
function `ParseCXXReflectExpression`. It parses the operand of the
reflection operator.
---------
Co-authored-by: Shafik Yaghmour <shafik.yaghmour@intel.com>
Co-authored-by: Hubert Tong <hubert.reinterpretcast@gmail.com>
Co-authored-by: Sirraide <aeternalmail@gmail.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: Erich Keane <ekeane@nvidia.com>
When building a module, the PCM file is always written first and then
the validation timestamp gets created. Clang needs to first read the
validation timestamp and only then read the PCM file. Otherwise, it
could read an out-of-date PCM file and then read the validation
timestamp for its new up-to-date version. This would erroneously skip
validation with `-fmodules-validate-once-per-build-session`. I'm not
concerned about multiple Clang instances seeing different filesystem
contents from each other within a single build session, since that would
break the assumption `-fmodules-validate-once-per-build-session` relies
on.
For incremental multi-process/multi-thread compilation utilizing a build
session fix errors like
> fatal error: file '/path/to/Frmwrk.framework/Headers/Header.h' has been modified since the module file '/path/to/ModuleCache.noindex/XXX/Frmwrk-YYY.pcm' was built: mtime changed (was aaa, now bbb)
Another symptom of the bug is when you check Frmwrk.pcm the header's
mtime is correct, so it is confusing where "was aaa" is even coming from.
The main problem is that in case of a signature mismatch
`ModuleManager::addModule` returns `OutOfDate` but keeps .pcm buffer in
`InMemoryModuleCache`. So if later on it tries to use such a module, it
would have an outdated buffer in `InMemoryModuleCache`. If another
process/thread happens to rebuild such module and write a new validation
timestamp, the original process would skip the input file validation and
would keep using the outdated .pcm buffer hitting errors about
unexpected input modifications.
I want to note that in the current implementation the validation
timestamp can be still incorrect. When we are writing AST we don't
account for the case when an input file is modified after its mtime and
size are captured. We are relying on the input files not being modified
while building a module using these files.
rdar://165237499
Reverts #172430 (c560f1cf03aa06c0bdd00c5a9b558c16d882af6f).
Causes some failures like `error: static assertion expression is not an
integral constant expression` and `error: substitution into constraint
expression resulted in a non-constant expression` in modules builds.
Repro TBD.
Depends on #169603.
This is the `use_device_ptr` counterpart of #168905.
With OpenMP 6.1, a `fallback` modifier can be specified on the
`use_device_ptr` clause to control the behavior when a pointer lookup
fails, i.e. there is no device pointer to translate into.
The default is `fb_preserve` (i.e. retain the original pointer), while
`fb_nullify` means: use `nullptr` as the translated pointer.
Dependent PR: #173930.
This PR unifies the terminology for:
* "context hash" - previously ambiguously referred to as "module hash"
or as overly specific "module context hash"
* "specific module cache path" - previously referred to as just "module
cache path" - hard to distinguish from the command-line-provided module
cache path without the context hash
NFCI
While it is correct to assign a single fixed hash to all template
arguments, it can reduce the effectiveness of lazy loading and is
not actually needed: we are allowed to ignore parts that cannot be
handled because they will be analogously ignored by all hashings.
Reviewed as part of https://github.com/llvm/llvm-project/pull/133057
It is unclear (to me) why this needs to be done "for safety", but
this change significantly improves the effectiveness of lazy loading.
Reviewed as part of https://github.com/llvm/llvm-project/pull/133057
The code is applied from ODRHash::AddDecl with the reasoning given
in the comment, to reduce collisions. This was particularly visible
with STL types templated on std::pair where its template arguments
were not taken into account.
Reviewed as part of https://github.com/llvm/llvm-project/pull/133057
This reverts commit 1928c1ea9b57e9c44325d436bc7bb2f4585031f3.
We have at least one repro, but I won't be able to work on this until
next week. Also with Clang 22 cut upcoming, we probably need to revert
for now.
The `ModuleCache` class is currently reference-counted intrusively. As
explained in https://github.com/llvm/llvm-project/pull/139584, this is
problematic. This PR uses `std::shared_ptr` to reference-count
`ModuleCache` instead, which clarifies what happens to its lifetime when
constructing `CompilerInstance`, for example. This also makes the
reference in `ModuleManager` non-owning, simplifying the ownership
relationship further. The
`ASTUnit::transferASTDataFromCompilerInstance()` function now accounts
for that by taking care to keep it alive.
PCHs (but also modules generated from several implicit invocations like
swiftc) previously reported a confusing diagnostic about module caches
being mismatched by subdir. This is an implementation detail of the
module machinery, and not very useful to the end user. Instead, report
this case as a configuration mismatch when the compiler can confirm the
module cache was passed the same between the current TU & previously
compiled products.
Ideally, each argument that could result in this error would be uniquely
reported (e.g., O3), but as a starting point, providing something more
general is strictly better than pointing the user to the module cache.
This patch also includes NFCs for renaming variable names from Module to
AST and formatting cleanup in related areas.
resolves: rdar://167453135
fixes#166206
- Add swizzle support if row index is constant
- Add test cases
- Add new AST type
- Add new LValue for Matrix Row Type
- TODO: Make the new LValue a dynamic index version of ExtVectorElt
Change the code added in commit 0d490ae55f and modified in commit
5ee6cff90b to the pattern found in ASTReader::finishPendingActions()
that avoids the iterator returned from redecls(), which may become
invalid during iteration.
RISC-V vector intrinsic is generated dynamically at runtime, thus it's
note preserved in AST yet when using precompile header, neither do
information in SemaRISCV. We need to write these information to ast
record to be able to use precompile header for RISC-V.
Fixes#109634
## Problem
Given code such as `N::foo();`, we perform name look-up on `N`. In the
case where `N` is a namespace declared in imported modules, one
namespace decl (the "key declaration") for each module that declares a
namespace `foo` is loaded and stored. In large scales where there are
many such modules, (e.g., 1,500) and many uses (e.g., 500,000), this
becomes extremely inefficient because every look-up (500,000 of them)
return 1,500 results.
The following synthetic script demonstrates the problem:
```bash
#/usr/bin/env bash
CLANG=${CLANG:-clang++}
NUM_MODULES=${NUM_MODULES:-1500}
NUM_USES=${NUM_USES:-500000}
USE_MODULES=${USE_MODULES:-true}
TMPDIR=$(mktemp -d)
echo "Working in temp directory: $TMPDIR"
cd $TMPDIR
trap "rm -rf \"$TMPDIR\"" EXIT
echo "namespace N { inline void foo() {} }" > m1.h
for i in $(seq 2 $NUM_MODULES); do echo "namespace N {}" > m${i}.h; done
if $USE_MODULES; then
seq 1 $NUM_MODULES | xargs -I {} -P $(nproc) bash -c "$CLANG -std=c++20 -fmodule-header m{}.h"
fi
> a.cpp
if $USE_MODULES; then
for i in $(seq 1 $NUM_MODULES); do echo "import \"m${i}.h\";" >> a.cpp; done
else
for i in $(seq 1 $NUM_MODULES); do echo "#include \"m${i}.h\"" >> a.cpp; done
fi
echo "int main() {" >> a.cpp
for i in $(seq 1 $NUM_USES); do echo " N::foo();" >> a.cpp; done
echo "}" >> a.cpp
if $USE_MODULES; then
time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null \
$(for i in $(seq 1 $NUM_MODULES); do echo -n "-fmodule-file=m${i}.pcm "; done)
else
time $CLANG -std=c++20 -Wno-experimental-header-units -c a.cpp -o /dev/null
fi
```
As of 575d6892bcc5cef926cfc1b95225148262c96a15, without modules
(`USE_MODULES=false`) this takes about **4.5s**, whereas with modules
(`USE_MODULES=true`), this takes about **37s**.
With this PR, without modules there's no change (as expected) at 4.5s,
but with modules it improves to about **5.2s**.
## Approach
The approach taken here aims to maintain status-quo with respect to the
input and output of modules. That is, the `ASTReader` and `ASTWriter`
both read and write the same declarations as it did before. The
difference is in the middle part: the [`StoredDeclsMap` in
`DeclContext`](https://github.com/llvm/llvm-project/blob/release/21.x/clang/include/clang/AST/DeclBase.h#L2024-L2030).
The `StoredDeclsMap` is roughly a `map<DeclarationName,
StoredDeclsList>`. Currently, we read all of the external namespace
decls from `ASTReader`, they all get stored into the `StoredDeclsList`,
and the `ASTWriter` iterates through that list and writes out the
results.
This PR continues to read all of the external namespace decls from
`ASTReader`, but only stores one namespace decl in the
`StoredDeclsList`. This is okay since the reading of the decls handles
all of the merging and chaining of the namespace decls, and as long as
they're loaded and chained, returning one for look-up purposes is
sufficient.
The other half of the problem is to write out all of the external
namespaces that we used to store in `StoredDeclsList` but no longer. For
this, we take advantage of the
[`KeyDecls`](https://github.com/llvm/llvm-project/blob/release/21.x/clang/include/clang/Serialization/ASTReader.h#L1342-L1347)
data structure in `ASTReader`. `KeyDecls` is roughly a `map<Decl *,
vector<GlobalDeclID>>`, and it stores a mapping from the canonical decl
of a redeclarable decl to a list of `GlobalDeclID`s where each ID
represents a "key declaration" from each imported module. More to the
point, if we read external namespaces `N1`, `N2`, `N3` in `ASTReader`,
we'll either have `N1` mapped to `[N2, N3]`, or some newly local
canonical decl mapped to `[N1, N2, N3]`. Either way, we can visit `N1`,
`N2`, and `N3` by doing `ASTReader::forEachImportedKeyDecls(N1,
Visitor)`, and we leverage this to maintain the current behavior of
writing out all of the imported namespace decls in `ASTWriter`.
## Alternatives Attempted
- Tried reading fewer declarations on the `ASTReader` side, and writing
out fewer declarations on the `ASTWriter` side, and neither options
worked at all.
- Tried trying to split `StoredDeclsList` into two pieces, one with
non-namespace decls and one with only namespace decls, but that didn't
work well... I think because the order of the declarations matter
sometimes, and maybe also because the declaration replacement logic gets
more complicated.
- Tried to deduplicate at the `SemaLookup` level. Basically, retrieve
all the stored decls but deduplicate populating the `LookupResult`
[here](https://github.com/llvm/llvm-project/blob/release/21.x/clang/lib/Sema/SemaLookup.cpp#L1137-L1144).
This did improve things slightly, but not quite enough, and this
solution seemed cleaner in the end anyway.
This reverts commit
54a4da9df6.
MSVC supports an extension allowing to delete an array of objects via
pointer whose static type doesn't match its dynamic type. This is done
via generation of special destructors - vector deleting destructors.
MSVC's virtual tables always contain a pointer to the vector deleting
destructor for classes with virtual destructors, so not having this
extension implemented causes clang to generate code that is not
compatible with the code generated by MSVC, because clang always puts a
pointer to a scalar deleting destructor to the vtable. As a bonus the
deletion of an array of polymorphic object will work just like it does
with MSVC - no memory leaks and correct destructors are called.
This patch will cause clang to emit code that is compatible with code
produced by MSVC but not compatible with code produced with clang of
older versions, so the new behavior can be disabled via passing
-fclang-abi-compat=21 (or lower).
Fixes https://github.com/llvm/llvm-project/issues/19772
This PR introduces a new mechanism for enforcing a sandbox around
filesystem reads coming from the compiler. A fatal error is raised
whenever the `llvm::sys::fs`, `llvm::MemoryBuffer::getFile*()` APIs get
used directly instead of going through the "blessed" virtual interface
of `llvm::vfs::FileSystem`.
Fix the propagation added in commit 0d490ae55f to include all redecls,
not only previous ones. This fixes another instance of the assertion
"Cannot get layout of forward declarations" in getASTRecordLayout().
Kudos to Alexander Kornienko for providing an initial version of the
reproducer that I further simplified.
Fixes#170084
- CUDA's dynamic parallelism extension allows device-side kernel
launches, which share the identical syntax to host-side launches, e.g.,
kernel<<<Dg, Db, Ns, S>>>(arguments);
but differ from the code generation. That device-side kernel launches is
eventually translated into the following sequence
config = cudaGetParameterBuffer(alignment, size);
// setup arguments by copying them into `config`.
cudaLaunchDevice(func, config, Dg, Db, Ns, S);
- To support the device-side kernel launch, 'CUDAKernelCallExpr' is
reused but its config expr is set to a call to 'cudaLaunchDevice'.
During the code generation, 'CUDAKernelCallExpr' is expanded into the
sequence aforementioned.
- As the device-side kernel launch requires the source to be compiled as
relocatable device code and linked with '-lcudadevrt'. Linkers are
changed to pass relevant link options to 'nvlink'.
As described in section 2.14.6 of openmp spec, the patch implements
support for iterator in motion clauses.
---------
Co-authored-by: Shashwathi N <nshashwa@pe31.hpc.amslabs.hpecorp.net>
MSVC supports an extension allowing to delete an array of objects via
pointer whose static type doesn't match its dynamic type. This is done
via generation of special destructors - vector deleting destructors.
MSVC's virtual tables always contain a pointer to the vector deleting
destructor for classes with virtual destructors, so not having this
extension implemented causes clang to generate code that is not
compatible with the code generated by MSVC, because clang always puts a
pointer to a scalar deleting destructor to the vtable. As a bonus the
deletion of an array of polymorphic object will work just like it does
with MSVC - no memory leaks and correct destructors are called.
This patch will cause clang to emit code that is compatible with code
produced by MSVC but not compatible with code produced with clang of
older versions, so the new behavior can be disabled via passing
-fclang-abi-compat=21 (or lower).
This is yet another attempt to land vector deleting destructors support
originally implemented by
https://github.com/llvm/llvm-project/pull/133451.
This PR contains fixes for issues reported in the original PR as well as
fixes for issues related to operator delete[] search reported in several
issues like
https://github.com/llvm/llvm-project/pull/133950#issuecomment-2787510484https://github.com/llvm/llvm-project/issues/134265
Fixes https://github.com/llvm/llvm-project/issues/19772
Similar to previous no transitive changes to decls, types, identifiers
and source locations (
https://github.com/llvm/llvm-project/pull/92083https://github.com/llvm/llvm-project/pull/92085https://github.com/llvm/llvm-project/pull/92511https://github.com/llvm/llvm-project/pull/86912
)
This patch does the same thing for MacroID and PreprocessedEntityID.
---
### Some background
Previously we record different IDs linearly. That is, when writing a
module, if we have 17 decls in imported modules, the ID of decls in the
module will start from 18. This makes the contents of the BMI changes if
the we add/remove any decls, types, identifiers and source locations in
the imported modules.
This makes it hard for us to reduce recompilations with modules. We want
to skip recompilations as we think the modules can help us to remove
fake dependencies. This can be done by split the ID into <ModuleIndex,
LocalIndex> pairs.
This is ALREADY done for several different ID above. We call it
non-casacading changes
(https://clang.llvm.org/docs/StandardCPlusPlusModules.html#experimental-non-cascading-changes).
Our internal users have already used this feature and it works well for
years.
Now we want to extend this to MacroID and PreprocessedEntityID. This is
helpful for us in the downstream as we allowed named modules to export
macros. But I believe this is also helpful for header-like modules if
you'd like to explore the area.
And also I think this is a nice cleanup too.
---
Given the use of MacroID and PreprocessedEntityID are not as complicated
as other IDs in the above series, I feel the patch itself should be
good. I hope the vendors can test the patch to make sure it won't affect
existing users.
Close https://github.com/llvm/llvm-project/issues/166068
The cause of the problem is that we would import initializers and
pending implicit instantiations from other named module. This is very
bad and it may waste a lot of time.
And we didn't observe it as the weak symbols can live together and the
strong symbols would be removed by other mechanism. So we didn't observe
the bad behavior for a long time. But it indeeds waste compilation time.
This PR adds support for the `dyn_groupprivate` clause, which will be
part of OpenMP 6.1. This feature allows users to request dynamic shared
memory on target regions.
---------
Co-authored-by: Krzysztof Parzyszek <Krzysztof.Parzyszek@amd.com>
Fixes#165445.
Fixes a crash when `ASTWriter::GenerateNameLookupTable` processes enum
constants from C++20 header units.
The special handling for enum constants, introduced in fccc6ee, doesn't
account for declarations whose owning module is a C++20 header unit. It
calls `isNamedModule()` on the result of
`getTopLevelOwningNamedModule()`, which returns null for header units,
causing a null pointer dereference.
This implements the parts of
https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3457.htm which were
adopted at the recent meeting in Brno.
Clang already implemented `__COUNTER__`, but needed some changes for
conformance. Specifically, we now diagnose when the macro is expanded
more than 2147483647 times. Additionally, we now give the expected
extension and pre-compat warnings for the feature.
To support testing the limits, this also adds a -cc1-only option,
`-finitial-counter-value=`, which lets you specify the initial value the
`__COUNTER__` macro should expand to.
This PR refactors `ASTUnit::LoadFromASTFile()` to be easier to follow.
Conceptually, it tries to read an AST file, adopt the serialized
options, and set up `Sema` and `ASTContext` to deserialize the AST file
contents on-demand.
The implementation of this used to be spread across an
`ASTReaderListener` and the function in question. Figuring out what
listener method gets called when and how it's supposed to interact with
the rest of the functionality was very unclear. The `FileManager`'s VFS
was being swapped-out during deserialization, the options were being
adopted by `Preprocessor` and others just-in-time to pass `ASTReader`'s
validation checks, and the target was being initialized somewhere in
between all of this. This lead to a very muddy semantics.
This PR splits `ASTUnit::LoadFromASTFile()` into three distinct steps:
1. Read out the options from the AST file.
2. Initialize objects from the VFS to the `ASTContext`.
3. Load the AST file and hook it up with the compiler objects.
This should be much easier to understand, and I've done my best to
clearly document the remaining gotchas.
(This was originally motivated by the desire to remove
`FileManager::setVirtualFileSystem()` and make it impossible to swap out
VFSs from underneath `FileManager` mid-compile.)
This rename was made as part of
https://github.com/llvm/llvm-project/pull/147835 in order to ease
rebasing the PR, and give a nice window for other patches to get rebased
as well.
It has been a while already, so lets go ahead and rename it back.
This PR enhances the OpenMP `nowait` clause implementation by adding
support for optional argument in both parsing and semantic analysis
phases.
Reference:
1. OpenMP 6.0 Specification, page 481
#137363 was supposed to be NFC for the `CrossProcessModuleCache` (a.k.a
normal implicit module builds), but accidentally passed the wrong path
to `sys::fs::status`. Then, #141358 removed the correct path that
should've been passed instead. (The variable was flagged as unused.)
None of our existing tests caught this regression, we only found out due
to a SourceKit-LSP benchmark getting slower.
This PR re-implements the original behavior, adds new remark to Clang
for PCM input file validation, and uses it to create more reliable tests
of the `-fmodules-validate-once-per-build-session` flag.
This reverts commit
8d9aecce06.
Additionally, this refactors how we're doing the AST storage to put it
all in the trailing storage, which will hopefully prevent it from
leaking. The problem was that the AST doesn't call destructors on things
in ASTContext storage, so we weren't actually able to delete the
combiner
SmallVector (which I should have known...). This patch instead moves all
of that SmallVector data into trailing storage, which shouldn't have the
same
problem with leaking as before.
This is the first patch of a handful to get the reduction combiner
recipe lowering properly. THIS patch is NFC as it doesn't actually
change anything except the structure of the AST.
For each 'combiner' recipe we need a 'LHS' 'RHS' and expression to
represent the operation.
Each var-reference can have 1 or more combiners.
IF it is a plain scalar, or a struct with the proper operator, or an
array of either of those, there will be 1.
HOWEVER, aggregates without the proper operator are supposed to be
broken down and done from their elements (which can only be scalars). In
this case, we will represent 1 'combiner' recipe per field-decl.
This patch only puts the infrastructure in place to do so, future
patches wll do the work to fill this in.
