Handles clang::DiagnosticsEngine and clang::DiagnosticIDs.
For DiagnosticIDs, this mostly migrates from `new DiagnosticIDs` to
convenience method `DiagnosticIDs::create()`.
Part of cleanup https://github.com/llvm/llvm-project/issues/151026
Updated error message logic for undo function. Throws different errors
for the case of there being nothing to undo, and for the case of
requesting more undos than there are operations to undo.
Fixes https://github.com/llvm/llvm-project/issues/143668
The idea is to store a type-value pair in clang::Value which is updated
by the interpreter runtime. The class copies builtin types and boxes
non-builtin types to provide some lifetime control.
The patch enables default printers for C and C++ using a very
minimalistic approach. We handle enums, arrays and user types. Once we
land this we can focus on enabling user-defined pretty-printers which
take control over printing of types
The work started as part of https://reviews.llvm.org/D146809, then we
created a giant in https://github.com/llvm/llvm-project/pull/84769
This patch fixes an issue where Microsoft-specific layout attributes,
such as __declspec(empty_bases), were ignored during CUDA/HIP device
compilation on a Windows host. This caused a critical memory layout
mismatch between host and device objects, breaking libraries that rely
on these attributes for ABI compatibility.
The fix introduces a centralized hasMicrosoftRecordLayout() check within
the TargetInfo class. This check is aware of the auxiliary (host) target
and is set during TargetInfo::adjust if the host uses a Microsoft ABI.
The empty_bases, layout_version, and msvc::no_unique_address attributes
now use this centralized flag, ensuring device code respects them and
maintains layout consistency with the host.
Fixes: https://github.com/llvm/llvm-project/issues/146047
This removes ThreadSafeContext::Lock, ThreadSafeContext::getLock, and
ThreadSafeContext::getContext, and replaces them with a
ThreadSafeContext::withContextDo method (and const override).
The new method can be used to access an existing
ThreadSafeContext-wrapped LLVMContext in a safe way:
ThreadSafeContext TSCtx = ... ;
TSCtx.withContextDo([](LLVMContext *Ctx) {
// this closure has exclusive access to Ctx.
});
The new API enforces correct locking, whereas the old APIs relied on
manual locking (which almost no in-tree code preformed, relying instead
on incidental exclusive access to the ThreadSafeContext).
This PR introduces out-of-process (OOP) execution support for
Clang-Repl. With this enhancement, two new flags, oop-executor and
oop-executor-connect, are added to the Clang-Repl interface. These flags
enable the launch of an external executor (llvm-jitlink-executor), which
handles code execution in a separate process.
This reverts commit e2a885537f11f8d9ced1c80c2c90069ab5adeb1d. Build failures were fixed right away and reverting the original commit without the fixes breaks the build again.
The `DiagnosticOptions` class is currently intrusively
reference-counted, which makes reasoning about its lifetime very
difficult in some cases. For example, `CompilerInvocation` owns the
`DiagnosticOptions` instance (wrapped in `llvm::IntrusiveRefCntPtr`) and
only exposes an accessor returning `DiagnosticOptions &`. One would
think this gives `CompilerInvocation` exclusive ownership of the object,
but that's not the case:
```c++
void shareOwnership(CompilerInvocation &CI) {
llvm::IntrusiveRefCntPtr<DiagnosticOptions> CoOwner = &CI.getDiagnosticOptions();
// ...
}
```
This is a perfectly valid pattern that is being actually used in the
codebase.
I would like to ensure the ownership of `DiagnosticOptions` by
`CompilerInvocation` is guaranteed to be exclusive. This can be
leveraged for a copy-on-write optimization later on. This PR changes
usages of `DiagnosticOptions` across `clang`, `clang-tools-extra` and
`lldb` to not be intrusively reference-counted.
Check this error for more context
(https://github.com/compiler-research/CppInterOp/actions/runs/14749797085/job/41407625681?pr=491#step:10:531)
This fails with
```
* thread #1, name = 'CppInterOpTests', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x55500356d6d3)
* frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
frame #2: 0x00007fffee20917a libclangCppInterOp.so.21.0gitstd::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() + 58
frame #3: 0x00007fffee224796 libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 838
frame #4: 0x00007fffee22494d libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 13
frame #5: 0x00007fffed95ec62 libclangCppInterOp.so.21.0gitclang::IncrementalCUDADeviceParser::~IncrementalCUDADeviceParser() + 98
frame #6: 0x00007fffed9551b6 libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 102
frame #7: 0x00007fffed95598d libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 13
frame #8: 0x00007fffed9181e7 libclangCppInterOp.so.21.0gitcompat::createClangInterpreter(std::vector<char const*, std::allocator<char const*>>&) + 2919
```
Problem :
1) The destructor currently handles no clearance for the DeviceParser
and the DeviceAct. We currently only have this
9764938224/clang/lib/Interpreter/Interpreter.cpp (L416-L419)
2) The ownership for DeviceCI currently is present in
IncrementalCudaDeviceParser. But this should be similar to how the
combination for hostCI, hostAction and hostParser are managed by the
Interpreter. As on master the DeviceAct and DeviceParser are managed by
the Interpreter but not DeviceCI. This is problematic because :
IncrementalParser holds a Sema& which points into the DeviceCI. On
master, DeviceCI is destroyed before the base class ~IncrementalParser()
runs, causing Parser::reset() to access a dangling Sema (and as Sema
holds a reference to Preprocessor which owns PragmaNamespace) we see
this
```
* frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
```
This PR hides the reference-counted pointer that holds `TargetOptions`
from the public API of `CompilerInvocation`. This gives
`CompilerInvocation` an exclusive control over the lifetime of this
member, which will eventually be leveraged to implement a copy-on-write
behavior.
There are two clients that currently share ownership of that pointer:
* `TargetInfo` - This was refactored to hold a non-owning reference to
`TargetOptions`. The options object is typically owned by the
`CompilerInvocation` or by the new `CompilerInstance::AuxTargetOpts` for
the auxiliary target. This needed a bit of care in `ASTUnit::Parse()` to
keep the `CompilerInvocation` alive.
* `clangd::PreambleData` - This was refactored to exclusively own the
`TargetOptions` that get moved out of the `CompilerInvocation`.
`clang-repl --cuda` was previously crashing with a segmentation fault,
instead of reporting a clean error
```
(base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda
#0 0x0000000111da4fbc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x150fbc)
#1 0x0000000111da31dc llvm::sys::RunSignalHandlers() (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x14f1dc)
#2 0x0000000111da5628 SignalHandler(int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x151628)
#3 0x000000019b242de4 (/usr/lib/system/libsystem_platform.dylib+0x180482de4)
#4 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#5 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#6 0x0000000107f6bac8 clang::Interpreter::createWithCUDA(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x2173ac8)
#7 0x000000010206f8a8 main (/opt/local/libexec/llvm-20/bin/clang-repl+0x1000038a8)
#8 0x000000019ae8c274
Segmentation fault: 11
```
The underlying issue was that the `DeviceCompilerInstance` (used for
device-side CUDA compilation) was never initialized with a `Sema`, which
is required before constructing the `IncrementalCUDADeviceParser`.
89687e6f38/clang/lib/Interpreter/DeviceOffload.cpp (L32)89687e6f38/clang/lib/Interpreter/IncrementalParser.cpp (L31)
Unlike the host-side `CompilerInstance` which runs `ExecuteAction`
inside the Interpreter constructor (thereby setting up Sema), the
device-side CI was passed into the parser uninitialized, leading to an
assertion or crash when accessing its internals.
To fix this, I refactored the `Interpreter::create` method to include an
optional `DeviceCI` parameter. If provided, we know we need to take care
of this instance too. Only then do we construct the
`IncrementalCUDADeviceParser`.
**Currently we don't make use of the JIT for the wasm use cases so the
approach using the execution engine won't work in these cases.**
Rather if we use dlopen. We should be able to do the following
(demonstrating through a toy project)
1) Make use of LoadDynamicLibrary through the given implementation
```
extern "C" EMSCRIPTEN_KEEPALIVE int load_library(const char *name) {
auto Err = Interp->LoadDynamicLibrary(name);
if (Err) {
llvm::logAllUnhandledErrors(std::move(Err), llvm::errs(), "load_library error: ");
return -1;
}
return 0;
}
```
2) Add a button to call load_library once the library has been added in
our MEMFS (currently we have symengine built as a SIDE MODULE and we are
loading it)
This reverts commit 30ad53b92cec0cff9679d559edcc5b933312ba0c as it breaks
systems that don't have a systemwide libc++ or libstdc++ installed. It should
be rewritten to not invoke the system linker. In the meanwhile, reverting
to unblock the bots.
Apply the fix suggested by Lang Hames to address a crash in Clang-REPL
that occurs during the execution of `__run_exit_handlers` when using
dynamic libraries.
While running clang-repl in the browser, we would be interested in this
cc1 command
`
"" -cc1 -triple wasm32-unknown-emscripten -emit-obj -disable-free
-clear-ast-before-backend -disable-llvm-verifier -discard-value-names
-main-file-name "<<< inputs >>>" -mrelocation-model static
-mframe-pointer=none -ffp-contract=on -fno-rounding-math
-mconstructor-aliases -target-cpu generic -debugger-tuning=gdb
-fdebug-compilation-dir=/ -v -fcoverage-compilation-dir=/ -resource-dir
/lib/clang/19 -internal-isystem /include/wasm32-emscripten/c++/v1
-internal-isystem /include/c++/v1 -internal-isystem
/lib/clang/19/include -internal-isystem /include/wasm32-emscripten
-internal-isystem /include -std=c++17 -fdeprecated-macro -ferror-limit
19 -fvisibility=default -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf
-fcxx-exceptions -fexceptions -fincremental-extensions -o "<<< inputs
>>>.o" -x c++ "<<< inputs >>>"
`
As can be seen `shared` is anyway overwritten by `static` which is also
what would be provided by default. Hence we can get rid of the shared
flag here.
Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.
It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.
This patch improves the code reuse of the actions system and adds
several improvements for easier debugging via clang-repl
--debug-only=clang-repl.
The change inimproves the consistency of the TUKind when actions are
handled within a WrapperFrontendAction. In this case instead of falling
back to default TU_Complete, we forward to the TUKind of the ASTContext
which presumably was created by the intended action. This enables the
incremental infrastructure to reuse code.
This patch also clones the first llvm::Module because the first PTU now
can come from -include A.h and the presumption of llvm::Module being
empty does not hold. The changes are a first step to fix the issues with
`clang-repl --cuda`.
This PR introduces out-of-process (OOP) execution support for
Clang-Repl. With this enhancement, two new flags, `oop-executor` and
`oop-executor-connect`, are added to the Clang-Repl interface. These
flags enable the launch of an external executor
(`llvm-jitlink-executor`), which handles code execution in a separate
process.
This patch improves the design of the IncrementalParser and Interpreter
classes. Now the incremental parser is only responsible for building the
partial translation unit declaration and the AST, while the Interpreter
fills in the lower level llvm::Module and other JIT-related
infrastructure. Finally the Interpreter class now orchestrates the AST
and the LLVM IR with the IncrementalParser and IncrementalExecutor
classes.
The design improvement allows us to rework some of the logic that
extracts an interpreter value into the clang::Value object. The new
implementation simplifies use-cases which are used for out-of-process
execution by allowing interpreter to be inherited or customized with an
clang::ASTConsumer.
This change will enable completing the pretty printing work which is in
llvm/llvm-project#84769
This commit introduces support for running clang-repl and executing C++
code interactively inside a Javascript engine using WebAssembly when
built with Emscripten. This is achieved by producing WASM "shared
libraries" that can be loaded by the Emscripten runtime using dlopen()
More discussion is available in https://reviews.llvm.org/D158140
Co-authored-by: Anubhab Ghosh <anubhabghosh.me@gmail.com>
Until now the IncrExecutor was created lazily on the first execution
request. In order to process the PTUs that come from initialization, we
have to do it upfront implicitly.
This patch converts the enum into scoped enum, and moves it into its own header for the time being. It's definition is needed in `Sema.h`, and is going to be needed in upcoming `SemaObjC.h`. `Lookup.h` can't hold it, because it includes `Sema.h`.
The LLJITBuilder interface provides a very convenient way to configure
the ORCv2 JIT engine. IncrementalExecutor already used it internally to
construct the JIT, but didn't provide external access. This patch lifts
control of the creation process to the Interpreter and allows injection
of a custom instance through the extended interface. The Interpreter's
default behavior remains unchanged and the IncrementalExecutor remains
an implementation detail.
IncrementalExecutor is an implementation detail of the Interpreter. In
order to test extended features properly, we must be able to setup and
tear down the executor manually.
The Interpreter locks PTUs that originate from implicit runtime code and
initialization to prevent users from undoing them accidentally.
The previous implementation seemed hacky, because it required the reader
to be familiar with the internal workings of the PTU stack. The concept
itself is a pragmatic solution and not very surprising. This patch
introduces a function for it and adds a comment.
RuntimeInterfaceBuilder wires up JITed expressions with the hardcoded
Interpreter runtime. It's used only for value printing right now, but it
is not limited to that. The default implementation focuses on an
evaluation process where the Interpreter has direct access to the memory
of JITed expressions (in-process execution or shared memory).
We need a different approach to support out-of-process evaluation or
variations of the runtime. It seems reasonable to expose a minimal
interface for it. The new RuntimeInterfaceBuilder is an abstract base
class in the public header. For that, the TypeVisitor had to become a
component (instead of inheriting from it). FindRuntimeInterface() was
adjusted to return an instance of the RuntimeInterfaceBuilder and it can
be overridden from derived classes.
With out-of-process execution the target triple can be different from
the one on the host. We need an interface to configure it.
Relanding this with cleanup-fixes in the unittest.
Reverting because `clang-repl -Xcc -mcpu=arm1176jzf-s` isn't overwriting
this as I had expected. We need to check whether a specific CPU flag was
given by the user first.
Reverts llvm/llvm-project#77491
This patch brings back the basic support for C by inserting the required
for value printing runtime only when we are in C++ mode. Additionally,
it defines a new overload of operator placement new because we can't
really forward declare it in a library-agnostic way.
Fixes the issue described in llvm/llvm-project#69072.
We can pass `-mcpu=native` to the clang driver to let it consider the
host CPU when choosing the compile target for `clang-repl`. We can
already achieve this behavior with `clang-repl -Xcc -mcpu=native`, but
it seems like a reasonable default actually.
The trade-off between optimizing for a specific CPU and maximum
compatibility often leans towards the latter for static binaries,
because distributing many versions is cumbersome. However, when
compiling at runtime, we know the exact target CPU and we can use that
to optimize the generated code.
This patch makes a difference especially for "scattered" architectures
like ARM. When cross-compiling for a Raspberry Pi for example, we may
use a stock toolchain like arm-linux-gnueabihf-gcc. The resulting binary
will be compatible with all hardware versions. This is handy, but they
will all have `arm-linux-gnueabihf` as their host triple. Previously,
this caused the clang driver to select triple `armv6kz-linux-gnueabihf`
and CPU `arm1176jzf-s` as the REPL target. After this patch the default
triple and CPU on Raspberry Pi 4b will be `armv8a-linux-gnueabihf` and
`cortex-a72` respectively.
With this patch clang-repl matches the host detection in Orc.
This patch contains changes from
002d471a4a3cd8b429e4ca7c84fd54a642e50e4c, in
addition to a bug fix that added a virtual destructor to
`CompletionContextHandler`
The original changes in the orginal commit piggybacks on clang's
semantic modules to enable semantic completion. In particular, we use
`CodeCompletionContext` to differentiate two types of code completion.
We also
extract the relevant type information from it.
Original commit message:
"
This patch enabled code completion for ClangREPL. The feature was built upon
three existing Clang components: a list completer for LineEditor, a
CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
The first component serves as the main entry point of handling interactive inputs.
Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.
The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.
In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,
clang-repl> int foo = 42;
clang-repl> f<tab>
From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.
Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.
Differential revision: https://reviews.llvm.org/D154382
"
The new patch also fixes clangd and several memory issues that the bots reported
and upload the missing files.
Original commit message:
"
This patch enabled code completion for ClangREPL. The feature was built upon
three existing Clang components: a list completer for LineEditor, a
CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
The first component serves as the main entry point of handling interactive inputs.
Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.
The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.
In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,
clang-repl> int foo = 42;
clang-repl> f<tab>
From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.
Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.
Differential revision: https://reviews.llvm.org/D154382
"
The new patch also fixes clangd and several memory issues that the bots reported.
