In #168245, I attempted to dump the available settings to Markdown. That
required a full build of LLDB. However, to build the docs, only the swig
wrappers should need to be compiled. The comment was that we should be
able to use the definitions from the TableGen files.
Currently, the property definitions in don't have information about the
path where they will be available. They only contain a `Definition`
which groups properties, so they can be added to
`OptionValueProperties`.
With this PR, I'm adding the path for each property definition. For
example, `symbols.enable-external-lookup` would have `Name =
enable-external-lookup, Path = symbols`. In LLDB itself, we don't need
this path, we only need it for the documentation. To avoid mismatches
between the actual path and the declared one, I added a debug-only check
when a property group is added to a parent
(`OptionValueProperties::AppendProperty`).
The TableGen emitter for the properties now additionally emits
`g_{definition}_properties_def`, which includes both the array of
properties and the expected path. This constant has to be used to
initialize a `OptionValueProperties`.
I couldn't test this for everything (e.g. IntelPT or ProcessKDP), but
the necessary changes are simple: (1) set the `Path` in the TableGen
file, (2) update `initialize` to use `_def`.
Taking advantage of a few new SPI in macOS 26.4 libdyld, it is possible
for lldb to load binaries out of a shared cache binary blob, instead of
needing discrete files on disk. lldb has had one special case where it
has done this for years -- if the debugee process and lldb itself are
using the same shared cache, it could create ObjectFiles based on its
own memory contents. This new method requires only the shared cache on
disk, not depending on it being mapped into lldb's address space
already.
In HostInfoMacOSX.mm, we create an array of binaries in lldb's shared
cache, by one of two methods depending on the availability of SPI/SDKs.
This PR adds a new third method for loading lldb's shared cache off disk
as a proof of concept. It will prefer this new method when the needed
SPI are available at runtime. There is also a user setting to disable
this new method in case we uncover a problem as it is deployed.
I did change the internal store of the shared cache files from a single
array, to being organized by shared cache UUIDs, so we can have multiple
shared caches indexed in the future.
In HostInfoBase.h's SharedCacheImageInfo class, you can now create an
ImageInfo with a DataExtractorSP or a void* baton. I added GetUUID and
GetExtractor methods, and the latter will use the libdyld SPI to map the
segments for a specific binary into lldb's memory and return a
DataExtractorSP.
The setting is currently called symbols.shared-cache-binary-loading.
In DynamicLoaderDarwin::FindTargetModuleForImageInfo there was an
ordering mistake where we would always consult the HostInfoMacOSX.mm
shared cache provider, instead of checking lldb's own global module
cache first when looking for a binary, resulting in creating a new
Module repeatedly for shared cache binaries with the new method, parsing
the symbol table repeatedly. I fixed the ordering so we look at existing
Modules before we check the shared cache for one.
In ObjectFileMachOTest, it tests a TEXT and a DATA symbol, checking that
the contents of the function/data object match the bytes we got from the
shared cache. The test was using a DATA_DIRTY symbol, which was fine
when using lldb's own shared cache memory, but when we worked on the
shared cache binary on-disk directly, we were seeing different values
for the bytes because of relocations in there. I changed this to a
constant DATA symbol.
rdar://148939795
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Co-authored-by: Alex Langford <nirvashtzero@gmail.com>
I noticed that Module::GetMemoryObjectFile populates a Status object
upon error but it's effectively dropped on the floor. Instead, the
clients can report the error as desired.
At the moment, all clients are either (1) consuming the error because
it's only trying to find a module, or (2) log the error and bail out
early. I tried to preserve existing behavior as faithfully as possible.
We have gotten reports of an occasional deadlock on shutdown. The
Driver::MainLoop thread is shutting down, and gets stuck here:
Thread 0x3c017c
start
main
Driver::MainLoop()
lldb::SBDebugger::RunCommandInterpreter(bool, bool)
lldb_private::CommandInterpreter::RunCommandInterpreter(lldb_private::CommandInterpreterRunOptions&)
lldb_private::Debugger::RunIOHandlers()
lldb_private::Debugger::PopIOHandler(std::__1::shared_ptr<lldb_private::IOHandler>
const&)
lldb_private::Editline::Cancel()
lldb_private::LockableStreamFile::Lock()
std::__1::recursive_mutex::lock()
It has acquired the IO Handler list lock (in PopIOHandler) and is stuck
trying to get the debugger output stream lock.
Meanwhile, the event-handler thread is doing:
Thread 0x3c01d3
thread_start
_pthread_start
lldb_private::HostThreadMacOSX::ThreadCreateTrampoline(void*)
lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(void*)
std::__1::__function::__func<lldb_private::Debugger::StartEventHandlerThread()::$_0,
void* ()>::operator()()
lldb_private::Debugger::DefaultEventHandler()
lldb_private::Statusline::~Statusline()
lldb_private::Statusline::UpdateScrollWindow(lldb_private::Statusline::ScrollWindowMode)
lldb_private::Debugger::RefreshIOHandler()
std::__1::recursive_mutex::lock()
UpdateScrollWindow gets the debugger output stream lock, and sends some
data to the output, then it calls RefreshIOHandler while still holding
the Debugger output stream lock. That's the problem, since if it gets
that lock before Editline::Cancel() completes, we'll get this deadlock.
The solution is simple, there's no reason why UpdateScrollWindow should
hold the debugger output lock when it calls RefreshIOHandler. If that
refresh ends up needing to write to the debugger output, it can take the
lock more narrowly at that point. This fixed the deadlock because after
yielding the output lock, the second thread waits on the first to get
the IO Handler lock. Meanwhile the first thread can now acquire the
debugger output lock and finish the Cancel, and exit PopIOHandler which
will allow the second thread to make progress in turn.
I couldn't figure out any way to test this...
In a PR last month I changed the ObjectFile CreateInstance etc methods
to accept an optional DataExtractorSP instead of a DataBufferSP, and
retain the extractor in a shared pointer internally in all of the
ObjectFile subclasses. This is laying the groundwork for using a
VirtualDataExtractor for some Mach-O binaries on macOS, where the
segments of the binary are out-of-order in actual memory, and we add a
lookup table to make it appear that the TEXT segment is at offset 0 in
the Extractor, etc. Working on the actual implementation, I realized we
were still using DataBufferSP's in ModuleSpec and Module, as well as in
ObjectFile::GetModuleSpecifications.
I originally was making a much larger NFC change where I had all
ObjectFile subclasses operating on DataExtractors throughout their
implementation, as well as in the DWARF parser. It was a very large
patchset. Many subclasses start with their DataExtractor, then create
smaller DataExtractors for parts of the binary image - the string table,
the symbol table, etc., for processing.
After consideration and discussion with Jonas, we agreed that a
segment/section of a binary will never require a lookup table to access
the bytes within it, so I changed
VirtualDataExtractor::GetSubsetExtractorSP to (1) require that the
Subset be contained within a single lookup table entry, and (2) return a
simple DataExtractor bounded on that byte range. By doing this, I was
able to remove all of my very-invasive changes to the ObjectFile
subclass internals; it's only when they are operating on the entire
binary image that care is needed.
One pattern that subclasses like ObjectFileBreakpad use is to take an
ArrayRef of the DataBuffer for a binary, then create a StringRef of
that, then look for strings in it. With a VirtualDataExtractor and
out-of-order binary segments, with gaps between them, this allows us to
search the entire buffer looking for a string, and segfault when it gets
to an unmapped region of the buffer. I added a
VirtualDataExtractor::GetSubsetExtractorSP(0) which gets the largest
contiguous memory region starting at offset 0 for this use case, and I
added a comment about what was being done there because I know it is not
obvious, and people not working on macOS wouldn't be familiar with the
requirement. (when we have a ModuleSpec with a DataExtractor, any of the
ObjectFile subclasses get a shot at Creating, so they all have to be
able to iterate on these)
rdar://148939795
## Description
Contribution to this topic [Rich Disassembler for
LLDB](https://discourse.llvm.org/t/rich-disassembler-for-lldb/76952),
this part.
```
The rich disassembler output should be exposed as structured data and made available through LLDB’s scripting API so more tooling could be built on top of this
```
----
This pr replaces #174847
As was suggested in [this
comment](https://github.com/llvm/llvm-project/pull/174847#issuecomment-3757015552),
implement access to variable annotations from `SBInstruction` class
itself.
Notes:
- ✅ did run black formatter on the python file;
## Testing
Run test with
```sh
./build/bin/lldb-dotest -v -p TestVariableAnnotationsDisassembler.py lldb/test/API/functionalities/disassembler-variables
```
all tests (9 existing + 1 newly added) are passing
<details>
<summary>screenshot 2026-01-23</summary>
build from the latest commit 08f00730b5768a8e3f7039d810084fabaaa24470
<img width="1506" height="562" alt="image"
src="https://github.com/user-attachments/assets/69516353-3432-47df-ae45-c40b51ec14c4"
/>
</details>
<details>
<summary>screenshot 2026-01-29</summary>
build from the latest commit f48a1a2c10f96a457ca6844be2ccc9406d3d57a0
<img width="1232" height="740" alt="image"
src="https://github.com/user-attachments/assets/9d104ce6-36c3-430b-98fe-f028f83a6b6d"
/>
</details>
---------
Signed-off-by: Nikita B <n2h9z4@gmail.com>
This patch enables ELF core files to be loaded and still show
executables and shared libraries. Functionality includes:
- Load executable and shared libraries from memory if ELF headers are
available
- Create placeholder for missing shared libraries and executable.
Previously you just wouldn't get anything in the "image list" if no
executable was provided.
During target creation for launch mode, the locate module callback was
not invoked for the main executable because the Target doesn't exist yet
when ModuleList::GetSharedModule is called. This prevented custom module
location logic from being applied to the main executable.
This change adds a PlatformSP to ModuleSpec so that GetSharedModule can
call the locate module callback using the Platform directly, even when
Target is not available. The Platform is now set on the ModuleSpec in
TargetList::CreateTargetInternal before calling ResolveExecutable.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
Writing testcases for error handling that depends on specific DWARF is
very difficult and the resulting tests tend not to be very portable.
This patch adds a new setting to inject an error into variable
materialization, thus making it possible to write very simple source
code based tests for error handling instead.
This is primarily motivated by the Swift language plugin, but
functionality is generic and should be useful for other languages as
well.
No caller sets this to `true`. Initially added for and set by
`SymbolFileDWARFDebugMap` (see
`616f490777a4f35269a23abee851680134050065`). This was then removed
shortly after in:
```
commit 762f7135e290696595b6c7233245581f59eeb07c
Author: Greg Clayton <gclayton@apple.com>
Date: Sun Sep 18 18:59:15 2011 +0000
Don't put modules for .o files into the global shared module list. We
used to do this because we needed to find the shared pointer for a .o
file when the .o file's module was needed in a SymbolContext since the
module in a symbol context was a shared pointer. Now that we are using
intrusive pointers we don't have this limitation anymore since any
instrusive shared pointer can be made from a pointer to an object
all on its own.
```
At this point it's more of a foot-gun, because forcing it to true has
potentially significant performance implications (e.g.,
a5eaa05dce)
**This patch adds a marker to make hidden frames more explicit.**
---
Hidden frames can be confusing for some users, who see that the indexes
of the frames in a backtrace are not contiguous. This patch aims to
lessen the confusion by adding a delimiter for the first and last non
hidden frame, i.e the boundaries.
IDE's like Xcode and VSCode represent those in the UI by having the
hidden frames either greyed out or collapsed.
It's not possible to do this in the CLI, therefore, this patch makes use
of 2 unicode characters to mark the beginning and end of the hidden
frames range.
This patch depends on:
- https://github.com/llvm/llvm-project/pull/168603
# Examples
In the example below, frame `#2` to `#7` are is hidden, and therefore,
frame `#1` is the first non hidden frame of the range while frame `#8`
is the last non hidden frame:
<img width="488" height="112" alt="Screenshot 2025-11-18 at 18 41 11"
src="https://github.com/user-attachments/assets/a21431da-9729-4cf0-a6bc-024aa306fc45"
/>
If the selected frame is one of the 2 boundary frames, we replace the
delimiter character with the select character (`*`).
<img width="487" height="111" alt="Screenshot 2025-11-18 at 18 41 03"
src="https://github.com/user-attachments/assets/5616fa81-6db6-457d-9d1e-bbe46e710c26"
/>
<img width="488" height="111" alt="Screenshot 2025-11-18 at 18 40 55"
src="https://github.com/user-attachments/assets/93dfa6cf-0956-4718-b31c-f965ec72b56d"
/>
Depends on:
* https://github.com/llvm/llvm-project/pull/174618
If a format entity calls back into `Format` and passes it a format
entity type that we're already in the process of parsing, we are likely
going to run into infinite recursion and blow the stack. I think this is
only an issue when a format entity calls Format on a format string
provided by the user (otherwise we're in control of the recursion). An
example of this can be seen in the test-case adjusted by this patch.
This seems to be causing actual crashes in the field, so this patch adds
basic tracking to `Formatter::Format` that checks whether we're
recursively parsing the same entity. This may very well be intended by
some entities (e.g., `Root` and `Scope`), so there is an escape hatch
for those. There's also a special case where `Variable` causes a
recursive format (which I pointed out in a source comment).
We could narrow the scope of what kind of recursion is allowed by adding
a `UserProvidedFormatChild` (or similar) flag to `Entry`, and only
disallow recursing on those kinds of entries. For now I just use an
exemption list in `IsInvalidRecursiveFormat`.
Adding a unit-test for this is unfortunately tricky because the only
format entity that currently suffers from this is
`${function.name-with-args}`, which requires a language plugin and valid
target. If we really wanted to we could probably mock all of those, but
the shell test provides test coverage for the previously crashing case.
rdar://166890120
This patch creates a new `FormatEntity::Formatter` class and moves
`FormatEntity::Format` (and related APIs) into it. Most of the
parameters to `Format` are immutable across all recursive calls, so I
made them `const` member variables of `Formatter`. The main changes are
just mechanical renaming of:
```
FormatEntity::Format(...)
```
to
```
FormatEntity::Formatter(...).Format(stream, entry, valobj)
```
and making use of the member variables from inside `Format`.
We can probably make most of the parameters to the `Formatter`
constructor defaulted, but I chose not to in this patch to keep the diff
smaller.
The motivation for this is that I'm planning on adding logic to detect
recursive format entities (which would crash LLDB). That requires some
state, which in my opinion is best kept inside the `Formatter` class
instead of another parameter to `Format`.
The patch should be entirely NFC.
I know this is required for at least one feature, because
TestSectionAPI.py has failures if zlib isn't enabled. So I think it's
useful for users to be able to check.
Now that it's in the config, I have also used it to make a test
annotation so we don't get the failure in TestSectionAPI.py when zlib is
disabled.
Which for future reference was:
Traceback (most recent call last):
File
"/home/davspi01/llvm-project/lldb/packages/Python/lldbsuite/test/decorators.py",
line 452, in wrapper
return func(self, *args, **kwargs)
File
"/home/davspi01/llvm-project/lldb/test/API/python_api/section/TestSectionAPI.py",
line 67, in test_compressed_section_data
self.assertEqual(section_data, [0x20, 0x30, 0x40, 0x50, 0x60, 0x70,
0x80, 0x90])
AssertionError: Lists differ: [] != [32, 48, 64, 80, 96, 112, 128, 144]
As it failed to decode the compressed section.
Even if a file has a valid modification time, it's possible that reading
the data fails. The SourceManager wasn't accounting for that, which
would result in a crash due to an unchecked read from a null `data_sp`.
We were seeing the issue when trying to read from a buggy virtual file
system, but presumably the same thing can happen with a poorly timed
unmount of a drive.
rdar://166414707
This patch is a follow-up to 96c733e to fix a missing space in the
frame.pc format entity. This space was intended to be prepended to the
module format entity scope but if the module is not valid, which is
often the case for python pc-less scripted frames, the space between the
pc and the function name is missing.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
The current behavior will pick the first symbol that contains the
address, this causes LLDB to pick the wrong symbol when looking for
swift reflection metadata on Linux, as in that case it is valid for a
symbol to completely encompass another one.
Instead, this function should prefer the symbol which is an exact, if it
exists. As a bonus, this should also be faster in the vast majority of
the cases, as we probably query symbols by their exact address most of
the time.
rdar://166344740
The ObjectFile plugin interface accepts an optional DataBufferSP
argument. If the caller has the contents of the binary, it can provide
this in that DataBufferSP. The ObjectFile subclasses in their
CreateInstance methods will fill in the DataBufferSP with the actual
binary contents if it is not set.
ObjectFile base class creates an ivar DataExtractor from the
DataBufferSP passed in.
My next patch will be a caller that creates a VirtualDataExtractor with
the binary data, and needs to pass that in to the ObjectFile plugin,
instead of the bag-of-bytes DataBufferSP. It builds on the previous
patch changing ObjectFile's ivar from DataExtractor to DataExtractorSP
so I could pass in a subclass in the shared ptr. And it will be using
the VirtualDataExtractor that Jonas added in
https://github.com/llvm/llvm-project/pull/168802
No behavior is changed by the patch; we're simply moving the creation of
the DataExtractor to the caller, instead of a DataBuffer that is
immediately used to set up the ObjectFile DataExtractor. The patch is a
bit complicated because all of the ObjectFile subclasses have to
initialize their DataExtractor to pass in to the base class.
I ran the testsuite on macOS and on AArch64 Ubutnu. (btw David, I ran it
under qemu on my M4 mac with SME-no-SVE again, Ubuntu 25.10, checked
lshw(1) cpu capabilities, and qemu doesn't seem to be virtualizing the
SME, that explains why the testsuite passes)
rdar://148939795
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Currently if we are debugging an app that was compiled against an SDK
that we don't know about on the host, then every time we evaluate an
expression we get following spam on the console:
```
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
error: Error while searching for Xcode SDK: Unrecognized SDK type: <some SDK>
```
It is not a fatal error but the way we spam it pretty much ruins the
debugging experience.
This patch makes it so we only log this error once per debugger session.
Not sure how to best test it since we'd need to build a program with a
particular SDK and then make it unrecognized by LLDB. Confirmed manually
that the error only gets reported once after this patch.
Correct the use_editline check in IOHandlerEditline to prevent a crash
when we have an output and/or error file, but no stream. This fixes a
regression introduced by 58279d1 that results in a crash when calling
el_init with a NULL stream.
The original code was checking the stream: GetOutputFILE and
GetErrorFILE.
```
use_editline = GetInputFILE() && GetOutputFILE() && GetErrorFILE() &&
m_input_sp && m_input_sp->GetIsRealTerminal();
```
The new code is checking the file: `m_output_sp` and `m_error_sp`.
```
use_editline = m_input_sp && m_output_sp && m_error_sp &&
m_input_sp->GetIsRealTerminal();
```
The correct check is:
```
use_editline =
m_input_sp && m_input_sp->GetIsRealTerminal() &&
m_output_sp && m_output_sp->GetUnlockedFile().GetStream() &&
m_error_sp && m_error_sp->GetUnlockedFile().GetStream();
```
We don't need to update the check for the input, because we're handling
the missing stream there correctly in the call to the constructor:
```
m_editline_up = std::make_unique<Editline>(
editline_name, m_input_sp ? m_input_sp->GetStream() : nullptr,
m_output_sp, m_error_sp, m_color);
```
We can't do the same for the output and error because we need to pass
the file, not the stream (to do proper locking).
As I was debugging this I added some more assertions. They're generally
useful so I'm keeping them.
Fixes#170891
Add a verbose option to the version command and include the "build
configuration" in the command output. This allows users to quickly
identify if their version of LLDB was built with support for
xml/curl/python/lua etc. This data is already available through the SB
API using SBDebugger::GetBuildConfiguration, but this makes it more
discoverable.
```
(lldb) version -v
lldb version 22.0.0git (git@github.com:llvm/llvm-project.git revision 21a2aac5e5456f9181384406f3b3fcad621a7076)
clang revision 21a2aac5e5456f9181384406f3b3fcad621a7076
llvm revision 21a2aac5e5456f9181384406f3b3fcad621a7076
editline_wchar: yes
lzma: yes
curses: yes
editline: yes
fbsdvmcore: yes
xml: yes
lua: yes
python: yes
targets: [AArch64, AMDGPU, ARM, AVR, BPF, Hexagon, Lanai, LoongArch, Mips, MSP430, NVPTX, PowerPC, RISCV, Sparc, SPIRV, SystemZ, VE, WebAssembly, X86, XCore]
curl: yes
```
Resolves#170727
Inside the LLVM codebase, const vector& should just be ArrayRef, as this
more general API works both with vectors, SmallVectors and
SmallVectorImpl, as well as with single elements.
This commit replaces two uses introduced in
https://github.com/llvm/llvm-project/pull/168797 .
Scripted frames that materialize Python functions or other non-native
code are PC-less by design, meaning they don't have valid program
counter values. Previously, these frames would display invalid addresses
(`0xffffffffffffffff`) in backtrace output.
This patch updates `FormatEntity` to detect and suppress invalid address
display for PC-less frames, adds fallback to frame methods when symbol
context is unavailable, and modifies `StackFrame::GetSymbolContext` to
skip PC-based symbol resolution for invalid addresses.
The changes enable PC-less frames to display cleanly with proper
function names, file paths, and line numbers, and allow for source
display of foreign sources (like Python). Includes comprehensive test
coverage demonstrating frames pointing to Python source files.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
## Description
Contribution to this topic [Rich Disassembler for
LLDB](https://discourse.llvm.org/t/rich-disassembler-for-lldb/76952),
this part.
```
The rich disassembler output should be exposed as structured data and made available through LLDB’s scripting API so more tooling could be built on top of this
```
----
This pr introduces new method `AnnotateStructured` in
`VariableAnnotator` class, which returns the result as a vector of
`VariableAnnotation` structured data, compared to original `Annotate`.
Additionally structured data is enhanced with information inferred from
`DWARFExpressionEntry` and variable declaration data.
I have moved this part of functionality form a bigger pr
https://github.com/llvm/llvm-project/pull/165163 to make it easier to
review, deliver smaller chunk faster in an incremental way.
## Testing
Run test with
```sh
./build/bin/lldb-dotest -v -p TestVariableAnnotationsDisassembler.py lldb/test/API/functionalities/disassembler-variables
```
all tests (9 existing) are passing.
<details>
<summary>screenshot 2025-11-24</summary>
<img width="1344" height="875" alt="screenshot"
src="https://github.com/user-attachments/assets/863e0fca-1e3e-43dc-bfa3-4b78ce287ae6"
/>
</details>
<details>
<summary>screenshot 2025-11-26</summary>
<img width="1851" height="865" alt="image"
src="https://github.com/user-attachments/assets/d47dacee-a679-4a49-ab22-efb5a16fe29c"
/>
</details>
<details>
<summary>screenshot 2025-12-03</summary>
<img width="1592" height="922" alt="Screenshot From 2025-12-03 22-11-30"
src="https://github.com/user-attachments/assets/957ded3d-bea1-43d0-8241-d342dfc2c7b0"
/>
</details>
---------
Signed-off-by: Nikita B <n2h9z4@gmail.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Some months ago, the LookupInfo constructor logic was refactored to not
depend on language specific logic, and use languages plugins instead. In
this refactor, when the language type is unknown, a single LookupInfo
object will handle multiple languages. This doesn't work well, as
multiple languages might want to configure the LookupInfo object in
different ways. For example, different languages might want to set the
m_lookup_name differently from each other, but the previous
implementation would pick the first name a language provided, and
effectively ignored every other language. Other fields of the LookupInfo
object are also configured in incompatible ways.
This approach doesn't seem to be a problem upstream, since only the
C++/Objective-C language plugins are available, but it broke downstream
on the Swift fork, as adding Swift to the list of default languages when
the language type is unknown breaks C++ tests.
This patch makes it so instead of building a single LookupInfo object
for multiple languages, one LookupInfo object is built per language
instead.
rdar://159531216
This will allow the instruction emulation unwinder to reason about
instructions that prevent the subsequent instruction from executing.
Part of a sequence of PRs:
[lldb][NFCI] Rewrite UnwindAssemblyInstEmulation in terms of a CFG visit
#169630
[lldb][NFC] Rename forward_branch_offset to branch_offset in
UnwindAssemblyInstEmulation #169631
[lldb] Add DisassemblerLLVMC::IsBarrier API #169632
[lldb] Handle backwards branches in UnwindAssemblyInstEmulation #169633
commit-id:bb5df4aa
This change makes StackFrame methods virtual to enable subclass
overrides and introduces BorrowedStackFrame, a wrapper that presents an
existing StackFrame with a different frame index.
This enables creating synthetic frame views or renumbering frames
without copying the underlying frame data, which is useful for frame
manipulation scenarios.
This also adds a new borrowed-info format entity to show what was the
original frame index of the borrowed frame.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Currently, disabling the statusline with `settings set show-statusline
false` leaves LLDB in a broken state. The same is true when trying to
toggle the setting again.
The issue was that setting the scroll window to 0 is apparently not
identical to setting it to the correct number of rows, even though some
documentation online incorrectly claims so.
Fixes#166608
This patch fixes and eliminates the possibility of SupportFileSP ever
being nullptr. The support file was originally treated like a value
type, but became a polymorphic type and therefore has to be stored and
passed around as a pointer.
To avoid having all the callers check the validity of the pointer, I
introduced the invariant that SupportFileSP is never null and always
default constructed. However, without enforcement at the type level,
that's fragile and indeed, we already identified two crashes where
someone accidentally broke that invariant.
This PR introduces a NonNullSharedPtr to prevent that. NonNullSharedPtr
is a smart pointer wrapper around std::shared_ptr that guarantees the
pointer is never null. If default-constructed, it creates a
default-constructed instance of the contained type. Note that I'm using
private inheritance because you shouldn't inherit from standard library
classes due to the lack of virtual destructor. So while the new
abstraction looks like a `std::shared_ptr`, it is in fact **not** a
shared pointer. Given that our destructor is trivial, we could use
public inheritance, but currently there's no need for it.
rdar://164989579
If we open a `NativeFile` with a `FILE*`, the OpenOptions default to
`eOpenOptionReadOnly`. This is an issue in python scripts if you try to
write to one of the files like `print("Hi",
file=lldb.debugger.GetOutputFileHandle())`.
To address this, we need to specify the access mode whenever we create a
`NativeFile` from a `FILE*`. I also added an assert on the `NativeFile`
that validates the file is opened with the correct access mode and
updated `NativeFile::Read` and `NativeFile::Write` to check the access
mode.
Before these changes:
```
$ lldb -b -O 'script lldb.debugger.GetOutputFileHandle().write("abc")'
(lldb) script lldb.debugger.GetOutputFileHandle().write("abc")
Traceback (most recent call last):
File "<input>", line 1, in <module>
io.UnsupportedOperation: not writable
```
After:
```
$ lldb -b -O 'script lldb.debugger.GetOutputFileHandle().write("abc")'
(lldb) script lldb.debugger.GetOutputFileHandle().write("abc")
abc3
```
Fixes#122387
Another attempt at resolving the deadlock issue @GeorgeHuyubo discovered
(his previous
[attempt](https://github.com/llvm/llvm-project/pull/160225)).
This change can be summarized as the following:
* Plumb through a boolean flag to force no preload in
`GetOrCreateModules` all the way through to `LoadModuleAtAddress`.
* Parallelize `Module::PreloadSymbols` separately from
`Target::GetOrCreateModule` and its caller `LoadModuleAtAddress` (this
is what avoids the deadlock).
These changes roughly maintain the performance characteristics of the
previous implementation of parallel module loading. Testing on targets
with between 5000 and 14000 modules, I saw similar numbers as before,
often more than 10% faster in the new implementation across multiple
trials for these massive targets. I think it's because we have less lock
contention with this approach.
# The deadlock
See [bt.txt](https://github.com/user-attachments/files/22524471/bt.txt)
for a sample backtrace of LLDB when the deadlock occurs.
As @GeorgeHuyubo explains in his
[PR](https://github.com/llvm/llvm-project/pull/160225), the deadlock
occurs from an ABBA deadlock that happens when a thread context-switches
out of `Module::PreloadSymbols`, goes into `Target::GetOrCreateModule`
for another module, possibly entering this block:
```
if (!module_sp) {
// The platform is responsible for finding and caching an appropriate
// module in the shared module cache.
if (m_platform_sp) {
error = m_platform_sp->GetSharedModule(
module_spec, m_process_sp.get(), module_sp, &search_paths,
&old_modules, &did_create_module);
} else {
error = Status::FromErrorString("no platform is currently set");
}
}
```
`Module::PreloadSymbols` holds a module-level mutex, and then
`GetSharedModule` *attempts* to hold the mutex of the global shared
`ModuleList`. So, this thread holds the module mutex, and waits on the
global shared `ModuleList` mutex.
A competing thread may execute `Target::GetOrCreateModule`, enter the
same block as above, grabbing the global shared `ModuleList` mutex.
Then, in `ModuleList::GetSharedModule`, we eventually call
`ModuleList::FindModules` which eventually waits for the `Module` mutex
held by the first thread (via `Module::GetUUID`). Thus, we deadlock.
## Reproducing the deadlock
It might be worth noting that I've never been able to observe this
deadlock issue during live debugging (e.g. launching or attaching to
processes), however we were able to consistently reproduce this issue
with coredumps when using the following settings:
```
(lldb) settings set target.parallel-module-load true
(lldb) settings set target.preload-symbols true
(lldb) settings set symbols.load-on-demand false
(lldb) target create --core /some/core/file/here
# deadlock happens
```
## How this change avoids this deadlock
This change avoids concurrent executions of `Module::PreloadSymbols`
with `Target::GetOrCreateModule` by waiting until after the
`Target::GetOrCreateModule` executions to run `Module::PreloadSymbols`
in parallel. This avoids the ordering of holding a Module lock *then*
the ModuleList lock, as `Target::GetOrCreateModule` executions maintain
the ordering of the shared ModuleList lock first (from what I've read
and tested).
## Why not read-write lock?
Some feedback in https://github.com/llvm/llvm-project/pull/160225 was to
modify mutexes used in these components with read-write locks. This
might be a good idea overall, but I don't think it would *easily*
resolve this specific deadlock. `Module::PreloadSymbols` would probably
need a write lock to Module, so even if we had a read lock in
`Module::GetUUID` we would still contend. Maybe the `ModuleList` lock
could be a read lock that converts to a write lock if it chooses to
update the module, but it seems likely that some thread would try to
update the shared module list and then the write lock would contend
again.
Perhaps with deeper architectural changes, we could fix this issue?
# Other attempts
One downside of this approach (and the former approach of parallel
module loading) is that each DYLD would need to implement this pattern
themselves. With @clayborg's help, I looked at a few other approaches:
* In `Target::GetOrCreateModule`, backgrounding the
`Module::PreloadSymbols` call by adding it directly to the thread pool
via `Debugger::GetThreadPool().async()`. This required adding a lock to
`Module::SetLoadAddress` (probably should be one there already) since
`ObjectFileELF::SetLoadAddress` is not thread-safe (updates sections).
Unfortunately, during execution, this causes the preload symbols to run
synchronously with `Target::GetOrCreateModule`, preventing us from truly
parallelizing the execution.
* In `Module::PreloadSymbols`, backgrounding the `symtab` and `sym_file`
`PreloadSymbols` calls individually, but similar issues as the above.
* Passing a callback function like
https://github.com/swiftlang/llvm-project/pull/10746 instead of the
boolean I use in this change. It's functionally the same change IMO,
with some design tradeoffs:
* Pro: the caller doesn't need to explicitly call
`Module::PreloadSymbols` itself, and can instead call whatever function
is passed into the callback.
* Con: the caller needs to delay the execution of the callback such that
it occurs after the `GetOrCreateModule` logic, otherwise we run into the
same issue. I thought this would be trickier for the caller, requiring
some kinda condition variable or otherwise storing the calls to execute
afterwards.
# Test Plan:
```
ninja check-lldb
```
---------
Co-authored-by: Tom Yang <toyang@fb.com>
Currently `OutputBuffer::GtIsGt` is used to tell us if we're inside
template arguments and have printed a '(' without a closing ')'. If so,
we don't need to quote '<' when printing it as part of a binary
expression inside a template argument. Otherwise we need to. E.g.,
```
foo<a<(b < c)>> // Quotes around binary expression needed.
```
LLDB's `TrackingOutputBuffer` has heuristics that rely on checking
whether we are inside template arguments, regardless of the current
parentheses depth. We've been using `isGtInsideTemplateArgs` for this,
but that isn't correct. Resulting in us incorrectly tracking the
basename of function like:
```
void func<(foo::Enum)1>()
```
Here `GtIsGt > 0` despite us being inside template arguments (because we
incremented it when seeing '(').
This patch adds a `isInsideTemplateArgs` API which LLDB will use to more
accurately track parts of the demangled name.
To make sure this API doesn't go untested in the actual libcxxabi
test-suite, I changed the existing `GtIsGt` logic to use it. Also
renamed the various variables/APIs involved to make it (in my opinion)
more straightforward to understand what's going on. But happy to rename
it back if people disagree.
Also adjusted LLDB to use the newly introduced API (and added a
unit-test that would previously fail).
Recently I've been deep diving ELF cores in LLDB, aspiring to move LLDB
closer to GDB in capability. One issue I encountered was a system lib
losing it's unwind plan when loading the debuginfo. The reason for this
was the debuginfo has the eh_frame section stripped and the main
executable did not.
The root cause of this was this line in
[ObjectFileElf](163933e9e7/lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp (L1972))
```
// For eTypeDebugInfo files, the Symbol Vendor will take care of updating the
// unified section list.
if (GetType() != eTypeDebugInfo)
unified_section_list = *m_sections_up;
```
This would always be executed because CalculateType can never return an
eTypeDebugInfo
```
ObjectFile::Type ObjectFileELF::CalculateType() {
switch (m_header.e_type) {
case llvm::ELF::ET_NONE:
// 0 - No file type
return eTypeUnknown;
case llvm::ELF::ET_REL:
// 1 - Relocatable file
return eTypeObjectFile;
case llvm::ELF::ET_EXEC:
// 2 - Executable file
return eTypeExecutable;
case llvm::ELF::ET_DYN:
// 3 - Shared object file
return eTypeSharedLibrary;
case ET_CORE:
// 4 - Core file
return eTypeCoreFile;
default:
break;
}
return eTypeUnknown;
}
```
This makes sense as there isn't a explicit sh_type to denote that this
file is a debuginfo. After some discussion with @clayborg and
@GeorgeHuyubo we settled on joining the exciting unified section list
with whatever new sections were being added. Adding each new unique
section, or taking the section with the maximum file size. We picked
this strategy to pick the section with the most information. In most
scenarios, LHS should be SHT_NOBITS and RHS would be SHT_PROGBITS.
Here is a diagram documenting the existing vs proposed new way.
<img width="1666" height="1093" alt="image"
src="https://github.com/user-attachments/assets/73ba9620-c737-439e-9934-ac350d88a3b5"
/>
A global offset table is a section that holds the address of functions
that are dynamically linked. The Swift plugin needs to know if sections
are a global offset table or not.
Main executables were bypassing the locate module callback that shared
libraries use, preventing custom symbol file location logic from working
consistently.
This PR fix this by
* Adding target context to ModuleSpec
* Leveraging that context to use target search path and platform's
locate module callback in ModuleList::GetSharedModule
This ensures both main executables and shared libraries get the same
callback treatment for symbol file resolution.
---------
Co-authored-by: George Hu <hyubo@meta.com>
Co-authored-by: George Hu <georgehuyubo@gmail.com>
This patch introduces a new way to reconstruct the thread stackframe
list.
New `SyntheticFrameProvider` classes can lazy fetch a StackFrame at
index using a provided StackFrameList.
In can either be the real unwinder StackFrameList or we could also chain
SyntheticFrameProviders to each others.
This is the foundation work to implement ScriptedFrameProviders, which
will come in a follow-up patch.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Reading a source file might take a while, for example because it's
located on a virtual file system that's fetching the data on demand.
This PR emits a progress event to convey this to the user when reading
the file exceeds a certain threshold (500ms). Although it doesn't speed
up the operation, it still greatly improves the user experience by
helping them understand what's going on.
rdar://163750392
While testing baremetal lldb, I came across a situation that if an
instruction could not be disassembled, lldb will print nothing as an
output which might be a bit strange. I added at least printing warning
in this case.
This PR adds support for emitting the OSC `9;4` sequences to show a GUI
native progress bar.
There's a limited number of terminal emulators that support this, so for
now this requires explicit opt-in through a setting. I'm reusing the
existing `show-progress` setting, which became a NOOP with the
introduction of the statusline. The option now defaults to off.
Implements #160369
This fixes a potential use after free where
ModuleList::RemoveSharedModuleIfOrphaned ->
SharedModuleList::RemoveIfOrphaned -> SharedModuleList::RemoveFromMap
would potentially dereference a freed pointer. This fixes it by not
calling ModuleList::RemoveSharedModuleIfOrphaned at all if the pointer
was just freed.
