Make the interfaces part of lldbPluginScriptInterpreterPython instead of
putting them into their own static library. This avoids the need for an
extra static archive and more importantly a bunch of code duplication
between the two CMakeLists.txt.
Avoid directly including `ScriptInterpreterPython.h` in `SBHostOS`, and
instead go through the plugin interface to obtain the scripting path.
This also deprecates the ``SBHostOS::GetLLDBPythonPath`` method in favor
of the more generic GetScriptPath variant.
The header guard is redundant because
`source/Plugins/ScriptInterpreter/CMakeLists.txt` already gates the
entire `Python/` subdirectory behind `LLDB_ENABLE_PYTHON`.
This revert #181334 and its follow-up PRs (including #181488, #181492,
#181493, #181494 and #181498) as well as Ismail's documentation changes
(#181594, #181717). The original commit causes a test failure in CI
(https://github.com/llvm/llvm-project/issues/181938) but the more I look
at the patch, the more I'm convinced it was not ready to land. It will
be easier to iterate on the feedback by re-landing this than by using
post-commit review.
## Summary
Use the standard `// LLDB Python header must be included first.` comment
to match every other Python interface `.cpp` file in this directory, as
suggested by @JDevlieghere.
## Test plan
NFC - comment only change.
Co-authored-by: Rahul Reddy Chamala <rachamal@meta.com>
## Summary
Move the definition of `ScriptInterpreter::MakeSBModuleSpec` from
`ScriptInterpreter.cpp` to `ScriptedPythonInterface.cpp`.
`MakeSBModuleSpec` constructs an `SBModuleSpec`, whose symbols live in
the API library (`liblldb`). `ScriptInterpreter.cpp` is part of
`lldbInterpreter`, which is also linked into `lldb-server` — and
`lldb-server` does not link the API library. On Windows, this causes
`LNK2019: unresolved external symbol` for `SBModuleSpec`'s constructor
and destructor.
`ScriptedPythonInterface.cpp` is part of the Python plugin library,
which only links into `liblldb` where the API symbols are available. The
method retains friend access to `SBModuleSpec` since it is still a
member of `ScriptInterpreter` regardless of which `.cpp` file defines
it.
Fixes Windows linker failure from #181334.
## Test plan
- [ ] Existing ScriptedSymbolLocator tests pass
Co-authored-by: Rahul Reddy Chamala <rachamal@meta.com>
## Summary
Include `lldb-python.h` as the first include inside the
`LLDB_ENABLE_PYTHON` block in `ScriptInterpreterPythonInterfaces.cpp`,
matching the pattern used by every other Python interface `.cpp` file in
this directory.
On Windows, `lldb-python.h` defines `NO_PID_T` before including
`Python.h`. This prevents `PosixApi.h` (transitively included via
`lldb-private.h`) from redefining `pid_t` with a conflicting type
(`uint32_t` vs `int`).
The issue was introduced by #181334 (ScriptedSymbolLocator plugin),
which added a new header whose include chain transitively reaches
`PosixApi.h`.
Fixes Windows build failures on lldb-aarch64-windows, lldb-x86_64-win,
and lldb-remote-linux-win.
## Test plan
- [ ] lldb-aarch64-windows build passes
- [ ] lldb-x86_64-win build passes
- [ ] lldb-remote-linux-win build passes
Co-authored-by: Rahul Reddy Chamala <rachamal@meta.com>
## Summary
- Move `#include "lldb/Core/PluginManager.h"` after `#include
"ScriptInterpreterPythonInterfaces.h"` so Python's `pyconfig.h` defines
`pid_t` before `PosixApi.h` gets included. This fixes the `C2371:
'pid_t': redefinition; different basic types` error on all Windows
builders.
Fixes CI failures from #181334 / #181488.
Co-authored-by: Rahul Reddy Chamala <rachamal@meta.com>
## Summary
Based on discussion from
[RFC](https://discourse.llvm.org/t/rfc-python-callback-for-source-file-resolution/83545),
this PR adds a new `SymbolLocatorScripted` plugin that allows Python
scripts to implement custom symbol and source file resolution logic.
This enables downstream users to build custom symbol servers, source
file remapping, and build artifact resolution entirely in Python.
### Changes
- Adds `LocateSourceFile()` to the SymbolLocator plugin interface,
called during source path resolution with a fully loaded `ModuleSP`, so
the plugin has access to the module's UUID, file paths, and symbols.
- Adds `SymbolLocatorScripted` plugin that delegates all four
SymbolLocator methods (`LocateExecutableObjectFile`,
`LocateExecutableSymbolFile`, `DownloadObjectAndSymbolFile`,
`LocateSourceFile`) to a user-provided Python class.
- Adds `ScriptedSymbolLocatorPythonInterface` to bridge C++ calls to
Python, with proper GIL management and error handling.
- Results for `LocateSourceFile` are cached per (module UUID, source
file) pair.
- The Python class is configured via: `settings set
plugin.symbol-locator.scripted.script-class module.ClassName`
### Python class interface
```python
class MyLocator:
def __init__(self, exe_ctx, args): ...
def locate_source_file(self, module, original_source_file):
...
def locate_executable_object_file(self, module_spec): ...
def locate_executable_symbol_file(self, module_spec,
default_search_paths): ...
def download_object_and_symbol_file(self, module_spec,
force_lookup, copy_executable): ...
```
### Test plan
```
Added TestScriptedSymbolLocator.py with 3 test cases:
- test_locate_source_file — verifies the locator resolves source
files, receives a valid SBModule with UUID, and remaps paths correctly
- test_locate_source_file_none_fallthrough — verifies returning
None falls through to default LLDB resolution, and that having no script
class set works normally
- test_invalid_script_class — verifies graceful handling of
invalid class names without crashing
```
Co-authored-by: Rahul Reddy Chamala <rachamal@fb.com>
Fix LLDB header guards using clang-tidy's llvm-header-guard check. A
bunch of headers have been moved or renamed and we often forget to
update the header guard.
This patch adds plumbing to support the implementations of StackFrame::Get{*}Variable{*} on ScriptedFrame. The major pieces required are:
- A modification to ScriptedFrameInterface, so that we can actually call the python methods.
- A corresponding update to the python implementation to call the python methods.
- An implementation in ScriptedFrame that can get the variable list on construction inside ScriptedFrame::Create, and pass that list into the ScriptedFrame so it can get those values on request.
There is a major caveat, which is that if the values from the python side don't have variables attached, right now, they won't be passed into the scripted frame to be stored in the variable list. Future discussions around adding support for 'extended variables' when printing frame variables may create a reason to change the VariableListSP into a ValueObjectListSP, and generate the VariableListSP on the fly, but that should be addressed at a later time.
This patch also adds tests to the frame provider test suite to prove these changes all plumb together correctly.
Related radar: rdar://165708771
This patch adds support for:
- PyObject -> SBValueList (which was surprisingly not there before!)
- PyObject -> SBValue
- SBValue -> ValueObjectSP using the ScriptInterpreter
These three are the main remaining plumbing changes necessary before we can get to the meat of actually using ScriptedFrame to provide values to the printer/etc. Future patches build off this change in order to allow ScriptedFrames to provide variables and get values for variable expressions.
This patch adds `get_priority()` support to synthetic frame providers to
enable priority-based selection when multiple providers match a thread.
This is the first step toward supporting frame provider chaining for
visualizing coroutines, Swift async tasks, and et al.
Priority ordering follows Unix nice convention where lower numbers
indicate higher priority (0 = highest). Providers without explicit
priority return `std::nullopt`, which maps to UINT32_MAX (lowest
priority), ensuring backward compatibility with existing providers.
The implementation adds `GetPriority()` as a virtual method to
`SyntheticFrameProvider` base class, implements it through the scripting
interface hierarchy (`ScriptedFrameProviderInterface` and
`ScriptedFrameProviderPythonInterface`), and updates
`Thread::GetStackFrameList()` to sort applicable providers by priority
before attempting to load them.
Python frame providers can now specify priority:
```python
@staticmethod
def get_priority():
return 10 # Or return None for default priority.
```
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This adds a new virtual method `GetScriptedModulePath()` to
`ScriptedInterface` that allows retrieving the file path of the Python
module containing the scripted object implementation.
The Python implementation acquires the GIL and walks through the
object's `__class__.__module__` to find the module's `__file__`
attribute. This will be used by ScriptedFrame to populate the module and
compile unit for frames pointing to Python source files.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Extract `__func__` attribute from staticmethod/classmethod descriptors
before treating them as callables. Python's `@staticmethod` and
`@classmethod` decorators wrap methods in descriptor objects that are
not directly usable as PythonCallable, when calling PyCallable_Check.
The actual callable function is stored in the `__func__` attribute of
these descriptors, so we need to unwrap them to properly validate and
invoke the decorated methods in scripted interfaces.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This fixes a typo in ScriptedPythonInterface and changes
`AbstrackMethodCheckerPayload` to `AbstractMethodCheckerPayload`.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch extends ScriptedFrame to work with real (non-scripted)
threads,
enabling frame providers to synthesize frames for native processes.
Previously, ScriptedFrame only worked within
ScriptedProcess/ScriptedThread
contexts. This patch decouples ScriptedFrame from ScriptedThread,
allowing
users to augment or replace stack frames in real debugging sessions for
use
cases like custom calling conventions, reconstructing corrupted frames
from
core files, or adding diagnostic frames.
Key changes:
- ScriptedFrame::Create() now accepts ThreadSP instead of requiring
ScriptedThread, extracting architecture from the target triple rather
than ScriptedProcess.arch
- Added SBTarget::RegisterScriptedFrameProvider() and
ClearScriptedFrameProvider() APIs, with Target storing a
SyntheticFrameProviderDescriptor template for new threads
- Added "target frame-provider register/clear" commands for CLI access
- Thread class gains LoadScriptedFrameProvider(),
ClearScriptedFrameProvider(),
and GetFrameProvider() methods for per-thread frame provider management
- New SyntheticStackFrameList overrides FetchFramesUpTo() to lazily
provide
frames from either the frame provider or the real stack
This enables practical use of the SyntheticFrameProvider infrastructure
in
real debugging workflows.
rdar://161834688
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
parameters when defining the scripting interfaces.
We try to count the parameters to make sure the user has defined them
correctly, but this throws the counting off.
I'm not adding a test for this because then it would seem like we
thought this was a good idea. I'd actually rather not support it
altogether, but we added the parameter checking pretty recently so there
are extant implementations that we broke. I only want to support them,
not suggest anyone else do this going forward.
This patch implements the base and python interface for the
ScriptedFrameProvider class.
This is necessary to call python APIs from the ScriptedFrameProvider
that will come in a follow-up.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Modify the python wrapper to return uint32_t,
which prevents incorrect child name-to-index mapping and avoids
performing redundant operations on non-existent SBValues.
This patch adds the notion of "Facade" locations which can be reported
from a ScriptedResolver instead of the actual underlying breakpoint
location for the breakpoint. Also add a "was_hit" method to the scripted
resolver that allows the breakpoint to say which of these "Facade"
locations was hit, and "get_location_description" to provide a
description for the facade locations.
I apologize in advance for the size of the patch. Almost all of what's
here was necessary to (a) make the feature testable and (b) not break
any of the current behavior.
The motivation for this feature is given in the "Providing Facade
Locations" section that I added to the python-reference.rst so I won't
repeat it here.
rdar://152112327
This patch introduces a new scripting affordance in lldb:
`ScriptedFrame`.
This allows user to produce mock stackframes in scripted threads and
scripted processes from a python script.
With this change, StackFrame can be synthetized from different sources:
- Either from a dictionary containing a load address, and a frame index,
which is the legacy way.
- Or by creating a ScriptedFrame python object.
One particularity of synthezising stackframes from the ScriptedFrame
python object, is that these frame have an optional PC, meaning that
they don't have a report a valid PC and they can act as shells that just
contain static information, like the frame function name, the list of
variables or registers, etc. It can also provide a symbol context.
rdar://157260006
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
The current implementation tries to (1) patch the existing readline
module definition if it's already present in the inittab and (2) append
our patched readline module to the inittab. The former (1) uses the
non-stable Python API and I can't find a situation where this is
necessary.
We do this work before initialization, so for the readline
module to exist, it either needs to be added by Python itself (which
doesn't seem to be the case), or someone would have had to have added it
without initializing.
Use `PyThread_get_thread_ident`, which is part of the Stable API,
instead of accessing a member of the PyThreadState, which is opaque when
using the Stable API.
This conditionally reimplements PythonString::AsUTF8 using
PyUnicode_AsUTF8String instead of PyUnicode_AsUTF8AndSize.
PyUnicode_AsUTF8AndSize caches the UTF-8 representation of the string in
the Unicode object, which makes it more efficient and ties the lifetime
of the data to the Python string. However, it was only added to the
Stable API in Python 3.10. Older versions that want to use the Stable
API must use PyUnicode_AsUTF8String in combination with ConstString.
PyConfig and friends are not part of the stable API. We could switch
back to Py_SetPythonHome, which has been deprecated, but still part of
the stable API. For now, limit the use of PyConfig to when
LLDB_EMBED_PYTHON_HOME is enabled, which essentially means Windows.
Changing the order doesn't seem to matter.
Looking at the implementation of `pylifecycle.c` in cpython, finalizing
and initialized are set at the same time. Therefore we can eliminate the
call to `Py_IsFinalizing` and only check `Py_IsInitialized`, which is
part of the stable API.
I converted the check to an assert and confirmed that during my test
suite runs, we never got into the if block. Because we check before
taking the lock, there is an opportunity for a race, but that exact same
race exists with the original code.
PyMemoryView_FromMemory is part of stable ABI but the flag constants
such as PyBUF_READ are not. This was fixed in Python 3.11 [1], but still
requires this workaround when using older versions.
[1] https://github.com/python/cpython/issues/98680
Binary I/O (also called buffered I/O) expects bytes-like objects and
produces bytes objects [1]. Switch from using a Python buffer to using
Python bytes to read the data. This eliminates calls to functions that
aren't part of the Python stable C API.
[1] https://docs.python.org/3/library/io.html#binary-i-o
This adds a CMake option (which defaults to OFF) to force building
against the Python limited API. This makes iterating on #151617 easier
and eventually will prevent us from regressing this configuration.
Eliminate calls to PyGILState_Check, which is not part of the Python
Limited C API. In the Locker, we can use PyGILState_Ensure directly. We
could do something similar to replace the assert, but I don't think it's
worth it. We don't assert that we hold the GIL anywhere else.
The behavior of thread initialization changed in Python 3.7. The minimum
supported Python version is now 3.8. That means that
`PyEval_ThreadsInitialized` always returns true and `PyEval_InitThreads`
is never called.
The helper function existed to coordinate initializing the threads and
acquiring the GIL, which is no longer necessary. With that, there's no
point in having the helper at all. This PR eliminates the function and
inlines to GIL acquisition into the caller.
This is a continuation of 68fd102, which did the same thing but only for
StopInfo. Using make_shared is both safer and more efficient:
- With make_shared, the object and the control block are allocated
together, which is more efficient.
- With make_shared, the enable_shared_from_this base class is properly
linked to the control block before the constructor finishes, so
shared_from_this() will be safe to use (though still not recommended
during construction).
If we're not touching them, we don't need to do anything special to pass
them along -- with one important caveat: due to how cmake arguments
work, the implicitly passed arguments need to be specified before
arguments that we handle.
This isn't particularly nice, but the alternative is enumerating all
arguments that can be used by llvm_add_library and the macros it calls
(it also relies on implicit passing of some arguments to
llvm_process_sources).
