While building llvm (clang, lld) for wasm using emscripten (recipe
hosted on emscripten-forge
https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/llvm)
I ended up with this error
```
│ │ wasm-ld: error: ../../../../lib/libclangInterpreter.a(Wasm.cpp.o): undefined symbol: lld::wasm::link(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm:
│ │ :raw_ostream&, bool, bool)
```
This is due to the link function here
a4819bd46d/clang/lib/Interpreter/Wasm.cpp (L25-L30)
This was added through this PR (https://github.com/llvm/llvm-project/pull/86402) as an attempt to support running clang-repl and executing C++ code interactively inside a Javascript engine using WebAssembly when built with Emscripten.
The definition for link is present in lldwasm and when building for the emscripten platform we should be linking against it.
This patch improves the design of the IncrementalParser and Interpreter
classes. Now the incremental parser is only responsible for building the
partial translation unit declaration and the AST, while the Interpreter
fills in the lower level llvm::Module and other JIT-related
infrastructure. Finally the Interpreter class now orchestrates the AST
and the LLVM IR with the IncrementalParser and IncrementalExecutor
classes.
The design improvement allows us to rework some of the logic that
extracts an interpreter value into the clang::Value object. The new
implementation simplifies use-cases which are used for out-of-process
execution by allowing interpreter to be inherited or customized with an
clang::ASTConsumer.
This change will enable completing the pretty printing work which is in
llvm/llvm-project#84769
When BPF object files are linked with bpftool, every symbol must be
accompanied by BTF info. Ensure that extern functions referenced by
global variable initializers are included in BTF.
The primary motivation is "static" initialization of PROG maps:
```c
extern int elsewhere(struct xdp_md *);
struct {
__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
__uint(max_entries, 1);
__type(key, int);
__type(value, int);
__array(values, int (struct xdp_md *));
} prog_map SEC(".maps") = { .values = { elsewhere } };
```
BPF backend needs debug info to produce BTF. Debug info is not
normally generated for external variables and functions. Previously, it
was solved differently for variables (collecting variable declarations
in ExternalDeclarations vector) and functions (logic invoked during
codegen in CGExpr.cpp).
This patch generalises ExternalDefclarations to include both function
and variable declarations. This change ensures that function references
are not missed no matter the context. Previously external functions
referenced in constant expressions lacked debug info.
This commit introduces support for running clang-repl and executing C++
code interactively inside a Javascript engine using WebAssembly when
built with Emscripten. This is achieved by producing WASM "shared
libraries" that can be loaded by the Emscripten runtime using dlopen()
More discussion is available in https://reviews.llvm.org/D158140
Co-authored-by: Anubhab Ghosh <anubhabghosh.me@gmail.com>
This patch is based on clang-tidy's modernize-make-unique but limited
to those cases where type names are mentioned twice like
`std::unique_ptr<Type>(new Type())`, which is a bit mouthful.
Original commit message:"
[clang-repl] Extend the C support. (#89804)
The IdResolver chain is the main way for C to implement lookup rules. Every new
partial translation unit caused clang to exit the top-most scope which in turn
cleaned up the IdResolver chain. That was not an issue for C++ because its
lookup is implemented on the level of declaration contexts.
This patch keeps the IdResolver chain across partial translation units
maintaining proper C-style lookup infrastructure.
"
It was reverted in dfdf1c5fe45a82b9c578306f3d7627fd251d63f8 because it broke the
bots of lldb. This failure was subtle to debug but the current model does not
work well with ObjectiveC support in lldb. This patch does cleans up the
partial translation units in ObjectiveC. In future if we want to support
ObjectiveC we need to understand what exactly lldb is doing when recovering from
errors...
Until now the IncrExecutor was created lazily on the first execution
request. In order to process the PTUs that come from initialization, we
have to do it upfront implicitly.
The IdResolver chain is the main way for C to implement lookup rules.
Every new partial translation unit caused clang to exit the top-most
scope which in turn cleaned up the IdResolver chain. That was not an
issue for C++ because its lookup is implemented on the level of
declaration contexts.
This patch keeps the IdResolver chain across partial translation units
maintaining proper C-style lookup infrastructure.
Original commit message: "
Clang's CodeGen is designed to work with a single llvm::Module. In many cases
for convenience various CodeGen parts have a reference to the llvm::Module
(TheModule or Module) which does not change when a new module is pushed.
However, the execution engine wants to take ownership of the module which does
not map well to CodeGen's design. To work this around we clone the module and
pass it down.
With some effort it is possible to teach CodeGen to ask the CodeGenModule for
its current module and that would have an overall positive impact on CodeGen
improving the encapsulation of various parts but that's not resilient to future
regression.
This patch takes a more conservative approach and keeps the first llvm::Module
empty intentionally and does not pass it to the Jit. That's also not bullet
proof because we have to guarantee that CodeGen does not write on the
blueprint. However, we have inserted some assertions to catch accidental
additions to that canary module.
This change will fixes a long-standing invalid memory access reported by
valgrind when we enable the TBAA optimization passes. It also unblock progress
on https://github.com/llvm/llvm-project/pull/84758.
"
This patch reverts adc4f6233df734fbe3793118ecc89d3584e0c90f and removes
the check of `named_metadata_empty` of the first llvm::Module because on darwin
clang inserts some harmless metadata which we can ignore.
Clang's CodeGen is designed to work with a single llvm::Module. In many
cases
for convenience various CodeGen parts have a reference to the
llvm::Module
(TheModule or Module) which does not change when a new module is pushed.
However, the execution engine wants to take ownership of the module
which does
not map well to CodeGen's design. To work this around we clone the
module and
pass it down.
With some effort it is possible to teach CodeGen to ask the
CodeGenModule for
its current module and that would have an overall positive impact on
CodeGen
improving the encapsulation of various parts but that's not resilient to
future
regression.
This patch takes a more conservative approach and keeps the first
llvm::Module
empty intentionally and does not pass it to the Jit. That's also not
bullet
proof because we have to guarantee that CodeGen does not write on the
blueprint. However, we have inserted some assertions to catch accidental
additions to that canary module.
This change will fixes a long-standing invalid memory access reported by
valgrind when we enable the TBAA optimization passes. It also unblock
progress
on https://github.com/llvm/llvm-project/pull/84758.
This patch converts the enum into scoped enum, and moves it into its own header for the time being. It's definition is needed in `Sema.h`, and is going to be needed in upcoming `SemaObjC.h`. `Lookup.h` can't hold it, because it includes `Sema.h`.
Using remove() on DeclContext::lookup_result list invalidates iterators.
This assertion failure was one (fortunate) symptom:
```
clang/include/clang/AST/DeclBase.h:1337: reference clang::DeclListNode::iterator::operator*() const: Assertion `Ptr && "dereferencing end() iterator"' failed.
```
The LLJITBuilder interface provides a very convenient way to configure
the ORCv2 JIT engine. IncrementalExecutor already used it internally to
construct the JIT, but didn't provide external access. This patch lifts
control of the creation process to the Interpreter and allows injection
of a custom instance through the extended interface. The Interpreter's
default behavior remains unchanged and the IncrementalExecutor remains
an implementation detail.
IncrementalExecutor is an implementation detail of the Interpreter. In
order to test extended features properly, we must be able to setup and
tear down the executor manually.
The Interpreter locks PTUs that originate from implicit runtime code and
initialization to prevent users from undoing them accidentally.
The previous implementation seemed hacky, because it required the reader
to be familiar with the internal workings of the PTU stack. The concept
itself is a pragmatic solution and not very surprising. This patch
introduces a function for it and adds a comment.
RuntimeInterfaceBuilder wires up JITed expressions with the hardcoded
Interpreter runtime. It's used only for value printing right now, but it
is not limited to that. The default implementation focuses on an
evaluation process where the Interpreter has direct access to the memory
of JITed expressions (in-process execution or shared memory).
We need a different approach to support out-of-process evaluation or
variations of the runtime. It seems reasonable to expose a minimal
interface for it. The new RuntimeInterfaceBuilder is an abstract base
class in the public header. For that, the TypeVisitor had to become a
component (instead of inheriting from it). FindRuntimeInterface() was
adjusted to return an instance of the RuntimeInterfaceBuilder and it can
be overridden from derived classes.
With out-of-process execution the target triple can be different from
the one on the host. We need an interface to configure it.
Relanding this with cleanup-fixes in the unittest.
Reverting because `clang-repl -Xcc -mcpu=arm1176jzf-s` isn't overwriting
this as I had expected. We need to check whether a specific CPU flag was
given by the user first.
Reverts llvm/llvm-project#77491
This patch brings back the basic support for C by inserting the required
for value printing runtime only when we are in C++ mode. Additionally,
it defines a new overload of operator placement new because we can't
really forward declare it in a library-agnostic way.
Fixes the issue described in llvm/llvm-project#69072.
We can pass `-mcpu=native` to the clang driver to let it consider the
host CPU when choosing the compile target for `clang-repl`. We can
already achieve this behavior with `clang-repl -Xcc -mcpu=native`, but
it seems like a reasonable default actually.
The trade-off between optimizing for a specific CPU and maximum
compatibility often leans towards the latter for static binaries,
because distributing many versions is cumbersome. However, when
compiling at runtime, we know the exact target CPU and we can use that
to optimize the generated code.
This patch makes a difference especially for "scattered" architectures
like ARM. When cross-compiling for a Raspberry Pi for example, we may
use a stock toolchain like arm-linux-gnueabihf-gcc. The resulting binary
will be compatible with all hardware versions. This is handy, but they
will all have `arm-linux-gnueabihf` as their host triple. Previously,
this caused the clang driver to select triple `armv6kz-linux-gnueabihf`
and CPU `arm1176jzf-s` as the REPL target. After this patch the default
triple and CPU on Raspberry Pi 4b will be `armv8a-linux-gnueabihf` and
`cortex-a72` respectively.
With this patch clang-repl matches the host detection in Orc.
This patch fixes:
clang/lib/Interpreter/CodeCompletion.cpp:126:35: error: 'startswith'
is deprecated: Use starts_with instead
[-Werror,-Wdeprecated-declarations]
clang/lib/Interpreter/CodeCompletion.cpp:189:42: error: 'startswith'
is deprecated: Use starts_with instead
[-Werror,-Wdeprecated-declarations]
This patch contains changes from
002d471a4a3cd8b429e4ca7c84fd54a642e50e4c, in
addition to a bug fix that added a virtual destructor to
`CompletionContextHandler`
The original changes in the orginal commit piggybacks on clang's
semantic modules to enable semantic completion. In particular, we use
`CodeCompletionContext` to differentiate two types of code completion.
We also
extract the relevant type information from it.
A few symbols within libclangInterpreter have got explicit dllexport
attributes, in order to make them exported (and thus visible at runtime)
in any build, not only when they are part of e.g. a DLL libclang-cpp,
but also when they are part of a plain .exe.
Due to the explicit dllexports, these symbols would sidestep the regular
MinGW logic of exporting all symbols if there are no dllexports.
Therefore, for libclang-cpp, a separate fix was made in
592e935e115ffb451eb9b782376711dab6558fe0, to pass --export-all-symbols
to the build of libclang-cpp.
If building with BUILD_SHARED_LIBS enabled, then the same issue appears
in libclangInterpreter; pass the same flag --export-all-symbols there as
well, to make sure all symbols are visible, not only the ones that are
explicitly marked as dllexport.
This re-applies db51e572893, which was reverted in 05b1a2cb3e6 due to bot
failures. The DebuggerSupportPlugin now depends on DWARF, so it has been moved
to the new OrcDebugging library (as has the enableDebuggerSupport API).
This re-applies e1a5bb59b91, which was reverted in e5f169f91a8 due to LSan
failures on some bots (see https://github.com/llvm/llvm-project/issues/67586).
The LSan failures were not caused by this patch (just exposed by it), so LSan
was disabled for the failing test in 47625fea5e3. This should be safe to
re-land now.
This change means that debugger support only needs to be linked in if it's
used. The code size of debugger support is expected to increase as we improve
it (e.g. pulling in DWARF parsing), so making it an optional extra is useful
for controlling final binary sizes.
This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.
This matches other nearby enums.
For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::
Original commit message:"
ORC splits into separate dylibs symbols coming from the process and symbols
materialized in the Jit. This patch adapts intent of the existing interface and
adds a regression test to make sure both Jit'd and compiled symbols can be found.
Differential revision: https://reviews.llvm.org/D159115
"
This patch disables the test statement on windows as it seems we might have a
bug in the way we model dllimports.
ORC splits into separate dylibs symbols coming from the process and symbols
materialized in the Jit. This patch adapts intent of the existing interface and
adds a regression test to make sure both Jit'd and compiled symbols can be found.
Differential revision: https://reviews.llvm.org/D159115
Original commit message:
"
This patch enabled code completion for ClangREPL. The feature was built upon
three existing Clang components: a list completer for LineEditor, a
CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
The first component serves as the main entry point of handling interactive inputs.
Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.
The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.
In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,
clang-repl> int foo = 42;
clang-repl> f<tab>
From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.
Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.
Differential revision: https://reviews.llvm.org/D154382
"
The new patch also fixes clangd and several memory issues that the bots reported
and upload the missing files.
Original commit message:
"
This patch enabled code completion for ClangREPL. The feature was built upon
three existing Clang components: a list completer for LineEditor, a
CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
The first component serves as the main entry point of handling interactive inputs.
Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.
The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.
In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,
clang-repl> int foo = 42;
clang-repl> f<tab>
From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.
Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.
Differential revision: https://reviews.llvm.org/D154382
"
The new patch also fixes clangd and several memory issues that the bots reported.
This patch enabled code completion for ClangREPL. The feature was built upon
three existing Clang components: a list completer for LineEditor, a
CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method.
The first component serves as the main entry point of handling interactive inputs.
Because a completion point for a compiler instance has to be unchanged once it
is set, an incremental compiler instance is created for each code
completion. Such a compiler instance carries over AST context source from the
main interpreter compiler in order to obtain declarations or bindings from
previous input in the same REPL session.
The most important API codeComplete in Interpreter/CodeCompletion is a thin
wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as
a code completion point and a ReplCompletionConsumer, which communicates
completion results from SemaCodeCompletion back to the list completer for the
REPL.
In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels
were added so that SemaCodeCompletion can treat top level statements like
expression statements at the REPL. For example,
clang-repl> int foo = 42;
clang-repl> f<tab>
From a parser's persective, the cursor is at a top level. If we used code
completion without any changes, PCC_Namespace would be supplied to
Sema::CodeCompleteOrdinaryName, and thus the completion results would not
include foo.
Currently, the way we use PCC_TopLevelOrExpression and
CCC_TopLevelOrExpression is no different from the way we use PCC_Statement
and CCC_Statement respectively.
Differential revision: https://reviews.llvm.org/D154382
