AddressFunctionScope was always returning the first address range of the
function (assuming it was the only one). This doesn't work for
RegisterContextUnwind (it's only caller), when the function doesn't
start at the lowest address because it throws off the 'how many bytes
"into" a function I am' computation. This patch replaces the result with
a call to (recently introduced)
SymbolContext::GetFunctionOrSymbolAddress.
Follow-up to #138892 fixing breakage on windows. Calling
ClearAllLoadedSections earlier is necessary to avoid throwing out the
work done by the windows process plugin.
Minidump files contain explicit information about load addresses of
modules, so it can load them itself. This works on other platforms, but
fails on darwin because DynamicLoaderDarwin nukes the loaded module list
on initialization (which happens after the core file plugin has done its
work).
This used to work until #109477, which enabled the dynamic loader
plugins for minidump files in order to get them to provide access to
TLS.
Clearing the load list makes sense, but I think we could do it earlier
in the process, so that both Process and DynamicLoader plugins get a
chance to load modules. This patch does that by calling the function
early in the launch/attach/load core flows.
This fixes TestDynamicValue.py:test_from_core_file on darwin.
Re-landing this patch with small tweaks to address CI bot failures
as it was run on many different configurations. I think the test
may run on aarch64 Linux systems now.
When a frameless function faults or is interrupted asynchronously, the
UnwindPlan MAY have no register location rule for the return address
register (lr on arm64); the value is simply live in the lr register when
it was interrupted, and the frame below this on the stack -- e.g.
sigtramp on a Unix system -- has the full register context, including
that register.
RegisterContextUnwind::SavedLocationForRegister, when asked to find the
caller's pc value, will first see if there is a pc register location. If
there isn't, on a Return Address Register architecture like
arm/mips/riscv, we rewrite the register request from "pc" to "RA
register", and search for a location.
On frame 0 (the live frame) and an interrupted frame, the UnwindPlan may
have no register location rule for the RA Reg, that is valid. A
frameless function that never calls another may simply keep the return
address in the live register the whole way. Our instruction emulation
unwind plans explicitly add a rule (see Pavel's May 2024 change
https://github.com/llvm/llvm-project/pull/91321 ), but an UnwindPlan
sourced from debug_frame may not.
I've got a case where this exactly happens - clang debug_frame for arm64
where there is no register location for the lr in a frameless function.
There is a fault in the middle of this frameless function and we only
get the lr value from the fault handler below this frame if lr has a
register location of `IsSame`, in line with Pavel's 2024 change.
Similar to how we see a request of the RA Reg from frame 0 after failing
to find an unwind location for the pc register, the same style of
special casing is needed when this is a function that was interrupted.
Without this change, we can find the pc of the frame that was executing
when it was interrupted, but we need $lr to find its caller, and we
don't descend down to the trap handler to get that value, truncating the
stack.
rdar://145614545
This test is failing on the LLDB Incremental bot (arm64), which is
running an older set of tools (Xcode 15.2) and OS (macOS 14.1) and
the CFI directives must not be emitted correctly by either the tools
or the OS. I will need to reproduce how this is compiling on that
older setup and see what the issue is. Reverting for now so the
bots are not blocked.
This reverts commit e897cb139ee6ef5c145fed5394c4d96baa658e6b.
When a frameless function faults or is interrupted asynchronously, the
UnwindPlan MAY have no register location rule for the return address
register (lr on arm64); the value is simply live in the lr register when
it was interrupted, and the frame below this on the stack -- e.g.
sigtramp on a Unix system -- has the full register context, including
that register.
RegisterContextUnwind::SavedLocationForRegister, when asked to find the
caller's pc value, will first see if there is a pc register location. If
there isn't, on a Return Address Register architecture like
arm/mips/riscv, we rewrite the register request from "pc" to "RA
register", and search for a location.
On frame 0 (the live frame) and an interrupted frame, the UnwindPlan may
have no register location rule for the RA Reg, that is valid. A
frameless function that never calls another may simply keep the return
address in the live register the whole way. Our instruction emulation
unwind plans explicitly add a rule (see Pavel's May 2024 change
https://github.com/llvm/llvm-project/pull/91321 ), but an UnwindPlan
sourced from debug_frame may not.
I've got a case where this exactly happens - clang debug_frame for arm64
where there is no register location for the lr in a frameless function.
There is a fault in the middle of this frameless function and we only
get the lr value from the fault handler below this frame if lr has a
register location of `IsSame`, in line with Pavel's 2024 change.
Similar to how we see a request of the RA Reg from frame 0 after failing
to find an unwind location for the pc register, the same style of
special casing is needed when this is a function that was interrupted.
Without this change, we can find the pc of the frame that was executing
when it was interrupted, but we need $lr to find its caller, and we
don't descend down to the trap handler to get that value, truncating the
stack.
rdar://145614545
This patch uses the previously build infrastructure to parse multiple
FDE entries into a single unwind plan. There is one catch though: we
parse only one FDE entry per unwind range. This is not fully correct
because lldb coalesces adjecant address ranges, which means that
something that originally looked like two separate address ranges (and
two FDE entries) may get merged into one because if the linker decides
to put the two ranges next to each other. In this case, we will ignore
the second FDE entry.
It would be more correct to try to parse another entry when the one we
found turns out to be short, but I'm not doing this (yet), because:
- this is how we've done things so far (although, monolithic functions
are unlikely to have more than one FDE entry)
- in cases where we don't have debug info or (full) symbol tables, we
can end up with "symbols" which appear to span many megabytes
(potentially, the whole module). If we tried to fill short FDE entries,
we could end up parsing the entire eh_frame section in a single go. In a
way, this would be more correct, but it would also probably be very
slow.
I haven't quite decided what to do about this case yet, though it's not
particularly likely to happen in the "production" cases as typically the
functions are split into two parts (hot/cold) instead of one part per
basic block.
We're reading from the object's vtable to determine the pointer to the
full object. The vtable is normally in the "rodata" section of the
executable, which is often not included in the core file because it's
not supposed to change and the debugger can extrapolate its contents
from the executable file. We weren't doing that.
This patch changes the read operation to use the target class (which
falls back onto the executable module as expected) and adds the missing
ReadSignedIntegerFromMemory API. The fix is tested by creating a core
(minidump) file which deliberately omits the vtable pointer.
Custom regions in Process::GetUserSpecifiedCoreFileSaveRanges originally
used `FindEntryThatContains`. This made sense on my first attempt, but
what we really want are *intersecting* regions. This is so the user can
specify arbitrary memory, and if it's available we output it to the core
(Minidump or MachO).
This reverts commit daa4061d61216456baa83ab404e096200e327fb4.
Original PR https://github.com/llvm/llvm-project/pull/129092.
I have restricted the test to X86 Windows because it turns out the only
reason that `expr x.get()` would change m_memory_id is that on x86 we
have to write the return address to the stack in ABIWindows_X86_64::PrepareTrivialCall:
```
// Save return address onto the stack
if (!process_sp->WritePointerToMemory(sp, return_addr, error))
return false;
```
This is not required on AArch64 so m_memory_id was not changed:
```
(lldb) expr x.get()
(int) $0 = 0
(lldb) process status -d
Process 15316 stopped
* thread #1, stop reason = Exception 0x80000003 encountered at address 0x7ff764a31034
frame #0: 0x00007ff764a31038 TestProcessModificationIdOnExpr.cpp.tmp`main at TestProcessModificationIdOnExpr.cpp:35
32 __builtin_debugtrap();
33 __builtin_debugtrap();
34 return 0;
-> 35 }
36
37 // CHECK-LABEL: process status -d
38 // CHECK: m_stop_id: 2
ProcessModID:
m_stop_id: 3
m_last_natural_stop_id: 0
m_resume_id: 0
m_memory_id: 0
```
Really we should find a better way to force a memory write here, but
I can't think of one right now.
And a follow up warning fix.
This reverts commit 6aa963f780d63d4c8fa80de97dd79c932bc35f4e
and 2bff80f25d51e24d3c552e033a2863dd36ef648b.
This is failing on Windows on Arm: https://lab.llvm.org/buildbot/#/builders/141/builds/8375
Seems to produce the line the test wants but not in the right place.
Reverting while I investigate.
stop-hooks are supposed to trigger every time the process stops, but as
initially implemented they would only fire when control was returned to
the user. So for instance when a process was launched the stop hook
would only trigger when the process hit a breakpoint or crashed.
However, it would be really useful to be able to trigger a stop hook
when lldb first gains control over the process. One way to do that would
be to implement general "target lifecycle events" and then send process
created events that users could bind actions to.
OTOH, extending the stop hooks to fire when lldb first gains control
over the process is a pretty natural extension to the notion of a stop
hook. So this patch takes the shorter route to that ability by making
stop-hooks fire when lldb first gains control over the process.
I also added the ability to specify whether to trigger the stop hook "on
gaining control". I'm on the fence about whether to set the default to
be "trigger on gaining control" or "don't trigger on gaining control".
Since I think it's a generally useful feature, I've set the default to
"trigger on gaining control".
This patch fixes:
lldb/source/Target/Memory.cpp:438:3: error: 'lock_guard' may not
intend to support class template argument deduction
[-Werror,-Wctad-maybe-unsupported]
This change adds a setting `target.process.track-memory-cache-changes`.
Disabling this setting prevents invalidating and updating values in
`ValueObject::UpdateValueIfNeeded` when only "internal" debugger memory
is updated. Writing to "internal" debugger memory happens when, for
instance, expressions are evaluated by visualizers (pretty printers).
One of the examples when cache invalidation has a particularly heavy
impact is visualizations of some collections: in some collections
getting collection size is an expensive operation (it requires traversal
of the collection).
At the same time evaluating user expression with side effects (visible
to target, not only to debugger) will still bump memory ID because:
- If expression is evaluated via interpreter: it will cause write to
"non-internal" memory
- If expression is JIT-compiled: then to call the function LLDB will
write to "non-internal" stack memory
The downside of disabled `target.process.track-memory-cache-changes`
setting is that convenience variables won't reevaluate synthetic
children automatically.
---------
Co-authored-by: Mikhail Zakharov <mikhail.zakharov@jetbrains.com>
The function was always trying to dereference both the synthetic and
non-synthetic view of the object. This is wrong as the caller should be
able to determine which view of the object it wants to access, as is
done e.g. for child member access.
This patch removes the nonsynthetic->synthetic fallback, which is the
more surprising path, and fixes the callers to try both versions of the
object (when appropriate). I also snuck in simplification of the member
access code path because it was possible to use the same helper function
for that, and I wanted to be sure I understand the logic correctly.
I've left the synthetic->nonsynthetic fallback in place. I think we may
want to keep that one as we often have synthetic child providers for
pointer types. They usually don't provide an explicit dereference
operation but I think users would expect that a dereference operation on
those objects would work. What we may want to do is to try the
*synthetic* operation first in this case, so that the nonsynthetic case
is really a fallback.
---------
Co-authored-by: Ilia Kuklin <kuklin.iy@mail.ru>
Completes the ToJSON function for `OptionValue` types and make the interface function pure virtual
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Show assembly code when the source code for a frame is not available in
the debugger machine
Edit: this functionality will work only when using
`stop-disassembly-display = no-source` in the settings
Fix#136492
After the fix:
[Screencast From 2025-04-20
18-00-30.webm](https://github.com/user-attachments/assets/1ce41715-cf4f-42a1-8f5c-6196b9d685dc)
Identical PR to: https://github.com/llvm/llvm-project/pull/134563
Previous PR was approved and landed but broke the build due to bad
merge.
Manually resolve the merge conflict and try to land again.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
This reverts commit 070a4ae2f9bcf6967a7147ed2972f409eaa7d3a6.
Multiple buildbot failures have been reported:
https://github.com/llvm/llvm-project/pull/134563
The build fails with:
lldb/source/Target/Statistics.cpp:75:39: error: use of undeclared
identifier 'num_symbols_loaded'
Fix a [test
failure](https://github.com/llvm/llvm-project/pull/136236#issuecomment-2819772879)
in #136236, apply a minor renaming of statistics, and remerge. See
details below.
# Changes in #136236
Currently, `DebuggerStats::ReportStatistics()` calls
`Module::GetSymtab(/*can_create=*/false)`, but then the latter calls
`SymbolFile::GetSymtab()`. This will load symbols if haven't yet. See
stacktrace below.
The problem is that `DebuggerStats::ReportStatistics` should be
read-only. This is especially important because it reports stats for
symtab parsing/indexing time, which could be affected by the reporting
itself if it's not read-only.
This patch fixes this problem by adding an optional parameter
`SymbolFile::GetSymtab(bool can_create = true)` and receiving the
`false` value passed down from `Module::GetSymtab(/*can_create=*/false)`
when the call is initiated from `DebuggerStats::ReportStatistics()`.
---
Notes about the following stacktrace:
1. This can be reproduced. Create a helloworld program on **macOS** with
dSYM, add `settings set target.preload-symbols false` to `~/.lldbinit`,
do `lldb a.out`, then `statistics dump`.
2. `ObjectFile::GetSymtab` has `llvm::call_once`. So the fact that it
called into `ObjectFileMachO::ParseSymtab` means that the symbol table
is actually being parsed.
```
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = step over
frame #0: 0x0000000124c4d5a0 LLDB`ObjectFileMachO::ParseSymtab(this=0x0000000111504e40, symtab=0x0000600000a05e00) at ObjectFileMachO.cpp:2259:44
* frame #1: 0x0000000124fc50a0 LLDB`lldb_private::ObjectFile::GetSymtab()::$_0::operator()(this=0x000000016d35c858) const at ObjectFile.cpp:761:9
frame #5: 0x0000000124fc4e68 LLDB`void std::__1::__call_once_proxy[abi:v160006]<std::__1::tuple<lldb_private::ObjectFile::GetSymtab()::$_0&&>>(__vp=0x000000016d35c7f0) at mutex:652:5
frame #6: 0x0000000198afb99c libc++.1.dylib`std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) + 196
frame #7: 0x0000000124fc4dd0 LLDB`void std::__1::call_once[abi:v160006]<lldb_private::ObjectFile::GetSymtab()::$_0>(__flag=0x0000600003920080, __func=0x000000016d35c858) at mutex:670:9
frame #8: 0x0000000124fc3cb0 LLDB`void llvm::call_once<lldb_private::ObjectFile::GetSymtab()::$_0>(flag=0x0000600003920080, F=0x000000016d35c858) at Threading.h:88:5
frame #9: 0x0000000124fc2bc4 LLDB`lldb_private::ObjectFile::GetSymtab(this=0x0000000111504e40) at ObjectFile.cpp:755:5
frame #10: 0x0000000124fe0a28 LLDB`lldb_private::SymbolFileCommon::GetSymtab(this=0x0000000104865200) at SymbolFile.cpp:158:39
frame #11: 0x0000000124d8fedc LLDB`lldb_private::Module::GetSymtab(this=0x00000001113041a8, can_create=false) at Module.cpp:1027:21
frame #12: 0x0000000125125bdc LLDB`lldb_private::DebuggerStats::ReportStatistics(debugger=0x000000014284d400, target=0x0000000115808200, options=0x000000014195d6d1) at Statistics.cpp:329:30
frame #13: 0x0000000125672978 LLDB`CommandObjectStatsDump::DoExecute(this=0x000000014195d540, command=0x000000016d35d820, result=0x000000016d35e150) at CommandObjectStats.cpp:144:18
frame #14: 0x0000000124f29b40 LLDB`lldb_private::CommandObjectParsed::Execute(this=0x000000014195d540, args_string="", result=0x000000016d35e150) at CommandObject.cpp:832:9
frame #15: 0x0000000124efbd70 LLDB`lldb_private::CommandInterpreter::HandleCommand(this=0x0000000141b22f30, command_line="statistics dump", lazy_add_to_history=eLazyBoolCalculate, result=0x000000016d35e150, force_repeat_command=false) at CommandInterpreter.cpp:2134:14
frame #16: 0x0000000124f007f4 LLDB`lldb_private::CommandInterpreter::IOHandlerInputComplete(this=0x0000000141b22f30, io_handler=0x00000001419b2aa8, line="statistics dump") at CommandInterpreter.cpp:3251:3
frame #17: 0x0000000124d7b5ec LLDB`lldb_private::IOHandlerEditline::Run(this=0x00000001419b2aa8) at IOHandler.cpp:588:22
frame #18: 0x0000000124d1e8fc LLDB`lldb_private::Debugger::RunIOHandlers(this=0x000000014284d400) at Debugger.cpp:1225:16
frame #19: 0x0000000124f01f74 LLDB`lldb_private::CommandInterpreter::RunCommandInterpreter(this=0x0000000141b22f30, options=0x000000016d35e63c) at CommandInterpreter.cpp:3543:16
frame #20: 0x0000000122840294 LLDB`lldb::SBDebugger::RunCommandInterpreter(this=0x000000016d35ebd8, auto_handle_events=true, spawn_thread=false) at SBDebugger.cpp:1212:42
frame #21: 0x0000000102aa6d28 lldb`Driver::MainLoop(this=0x000000016d35ebb8) at Driver.cpp:621:18
frame #22: 0x0000000102aa75b0 lldb`main(argc=1, argv=0x000000016d35f548) at Driver.cpp:829:26
frame #23: 0x0000000198858274 dyld`start + 2840
```
# Changes in this PR top of the above
Fix a [test
failure](https://github.com/llvm/llvm-project/pull/136236#issuecomment-2819772879)
in `TestStats.py`. The original version of the added test checks that
all modules have symbol count zero when `target.preload-symbols ==
false`. The test failed on macOS. Due to various reasons, on macOS,
symbols can be loaded for dylibs even with that setting, but not for the
main module. For now, the fix of the test is to limit the assertion to
only the main module. The test now passes on macOS. In the future, when
we have a way to control a specific list of plug-ins to be loaded, there
may be a configuration that this test can use to assert that all modules
have symbol count zero.
Apply a minor renaming of statistics, per the
[suggestion](https://github.com/llvm/llvm-project/pull/136226#issuecomment-2825080275)
in #136226 after merge.
# New stats
The following stats are added and are available in both "statistics
dump" command and in python API.
1. In summary:
1. Add `totalSymbolsLoaded`. The total number of symbols loaded in all
modules.
2. Add `totalSymbolTablesLoaded `. The total number symbol tables loaded
in all modules.
2. In each module's stats:
1. Add `symbolsLoaded`. The number of symbols loaded in the current
module.
# Example
Example `statistics dump` output:
```
(lldb) statistics dump
{
...,
"modules": [
{
"path": "/Users/<username>/demo/simple/a.out",
"symbolsLoaded": 6,
...
},
...
],
...
"totalSymbolTablesLoaded": 42,
"totalSymbolsLoaded": 32198
}
```
# Tests
**Manual test**: Built and ran lldb on a helloworld program. Ran
`statistics dump`. Verified the above stats.
**Unit test**: Ran the following tests:
```
$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
...
Ran 18 tests in 192.676s
OK (skipped=3)
```
The function QueueThreadPlanForStepOutNoShouldStop has the semantics of
"go this parent frame"; ThreadPlanStepOut needs to respect that, not
skipping over any frames it finds uninteresting. This commit creates a
constructor that respects such instruction.
A subsequent commit will create a new constructor for ThreadPlanStepOut,
which needs to reuse much of the same logic of the existing constructor.
This commit places all of that reusable logic into a separate function.
A future patch will need to create a new constructor for this class, and
extracting code out of its sole existing constructor will make this
easier.
This commit creates a helper function for the code computing the target
frame to step out to.
This is necessary so that LLDB does not select (or show the stop reason
for) a thread which stopped at an internal breakpoint.
Other than manual testing/inspection, which I've done, this does not
seem to lend itself to API testing, as we cannot set internal
breakpoints through the SBAPI.
I traced the issue reported by Caroline and Pavel in #134757 back to the
call to ProcessRunLock::TrySetRunning. When that fails, we get a
somewhat misleading error message:
> process resume at entry point failed: Resume request failed - process
still running.
This is incorrect: the problem was not that the process was in a running
state, but rather that the RunLock was being held by another thread
(i.e. the Statusline). TrySetRunning would return false in both cases
and the call site only accounted for the former.
Besides the odd semantics, the current implementation is inherently
race-y and I believe incorrect. If someone is holding the RunLock, the
resume call should block, rather than give up, and with the lock held,
switch the running state and report the old running state.
This patch removes ProcessRunLock::TrySetRunning and updates all callers
to use ProcessRunLock::SetRunning instead. To support that,
ProcessRunLock::SetRunning (and ProcessRunLock::SetStopped, for
consistency) now report whether the process was stopped or running
respectively. Previously, both methods returned true unconditionally.
The old code has been around pretty much pretty much forever, there's
nothing in the git history to indicate that this was done purposely to
solve a particular issue. I've tested this on both Linux and macOS and
confirmed that this solves the statusline issue.
A big thank you to Jim for reviewing my proposed solution offline and
trying to poke holes in it.
Both the `CPlusPlusLanguage` plugins and the Swift language plugin
already assume the `sc != nullptr`. And all `FormatEntity` callsites of
`GetFunctionDisplayName` already check for nullptr before passing `sc`.
This patch makes this pre-condition explicit by changing the parameter
to `const SymbolContext &`. This will help with some upcoming changes in
this area.
I recently received an internal error report that LLDB was OOM'ing when
creating a Minidump. In my 64b refactor we made a decision to acquire
buffers the size of the largest memory region so we could read all of
the contents in one call. This made error handling very simple (and
simpler coding for me!) but had the trade off of large allocations if
huge pages were enabled.
This patch is one I've had on the back burner for awhile, but we can
read and write the Minidump memory sections in discrete chunks which we
already do for writing to disk.
I had to refactor the error handling a bit, but it remains the same. We
make a best effort attempt to read as much of the memory region as
possible, but fail immediately if we receive an error writing to disk. I
did not add new tests for this because our existing test suite is quite
good, but I did manually verify a few Minidumps couldn't read beyond the
red_zone.
```
(lldb) reg read $sp
rsp = 0x00007fffffffc3b0
(lldb) p/x 0x00007fffffffc3b0 - 128
(long) 0x00007fffffffc330
(lldb) memory read 0x00007fffffffc330
0x7fffffffc330: 60 c3 ff ff ff 7f 00 00 60 cd ff ff ff 7f 00 00 `.......`.......
0x7fffffffc340: 60 c3 ff ff ff 7f 00 00 65 e6 26 00 00 00 00 00 `.......e.&.....
(lldb) memory read 0x00007fffffffc329
error: could not parse memory info (Success!)
```
I'm not sure how to quantify the memory improvement other than we would
allocate the largest size regardless of the size. So a 2gb unreadable
region would cause a 2gb allocation even if we were reading 4096 kb. Now
we will take the range size or the max chunk size of 128 mb.
This reverts commit 094904303d50e0ab14bc5f2586a602f79af95953, reapplying
d7afafdbc464e65c56a0a1d77bad426aa7538306 (#133247).
The failure ought to be fixed by
0509932bb6a291ba11253f30c465ab3ad164ae08.
Add the Data Inspection Language (DIL) implementation pieces for
handling plain local and global variable names.
See https://discourse.llvm.org/t/rfc-data-inspection-language/69893 for
information about DIL.
This change includes the basic AST, Lexer, Parser and Evaluator pieces,
as well as some tests.
This NFC patch simplifies the main loop in HandleProcessStateChanged
event by moving duplicated code into the StopInfo class, also allowing
StopInfo subclasses to override behavior.
More specifically, two functions are created:
* ShouldShow: should a Thread with such StopInfo should be printed when
the debugger stops? Currently, no StopInfo subclasses override this, but
a subsequent patch will fix a bug by making StopInfoBreakpoint check
whether the breakpoint is internal.
* ShouldSelect: should a Thread with such a StopInfo be selected? This
is currently overridden by StopInfoUnixSignal but will, in the future,
be overridden by StopInfoBreakpoint.
When using `SBFrame::EvaluateExpression` on a frame that's not the
currently selected frame, we would sometimes run into errors such as:
```
error: error: The context has changed before we could JIT the expression!
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
```
During expression parsing, we call `RunStaticInitializers`. On our
internal fork this happens quite frequently because any usage of, e.g.,
function pointers, will inject ptrauth fixup code into the expression.
The static initializers are run using `RunThreadPlan`. The
`ExecutionContext::m_frame_sp` going into the `RunThreadPlan` is the
`SBFrame` that we called `EvaluateExpression` on. LLDB then tries to
save this frame to restore it after the thread-plan ran (the restore
occurs by unconditionally overwriting whatever is in
`ExecutionContext::m_frame_sp`). However, if the `selected_frame_sp` is
not the same as the `SBFrame`, then `RunThreadPlan` would set the
`ExecutionContext`'s frame to a different frame than what we started
with. When we `PrepareToExecuteJITExpression`, LLDB checks whether the
`ExecutionContext` frame changed from when we initially
`EvaluateExpression`, and if did, bails out with the error above.
One such test-case is attached. This currently passes regardless of the
fix because our ptrauth static initializers code isn't upstream yet. But
the plan is to upstream it soon.
This patch addresses the issue by saving/restoring the frame of the
incoming `ExecutionContext`, if such frame exists. Otherwise, fall back
to using the selected frame.
rdar://147456589
SetExitStatus can be called the second time when we reap the debug
server process. This shouldn't be interesting as at that point, we've
already told everyone that the process has exited.
I believe/hope this will also help with sporadic shutdown crashes that
have cropped up recently. They happen because the debug server is
monitored from a detached thread, so this code can be called after main
returns (and starts destroying everything). This isn't a real fix for
that though, as the situation can still happen (it's just that it
usually happens after the exit status has already been set). I think the
real fix for that is to make sure these threads terminate before we
start shutting everything down.
The tests for the
[intel-pt](3483740289/lldb/docs/use/intel_pt.rst)
trace plugin were failing for multiple reasons.
On machines where tracing is supported many of the tests were crashing
because of a nullptr dereference. It looks like the `core_file`
parameter in `ProcessTrace::CreateInstance` was once ignored, but was
changed to always being dereferenced. This caused the tests to fail even
when tracing was supported.
On machines where tracing is not supported we would still run tests that
attempt to take a trace. These would obviously fail because the required
hardware is not present. Note that some of the tests simply read
serialized json as trace files which does not require any special
hardware.
This PR fixes these two issues by guarding the pointer dereference and
then skipping unsupported tests on machines. With these changes the
trace tests pass on both types of machines.
We also add a new unit test to validate that a process can be created
with a nullptr core_file through the generic process trace plugin path.
This patch improves LLDB launch time on Linux machines for **preload
scenarios**, particularly for executables with a lot of shared library
dependencies (or modules). Specifically:
* Launching a binary with `target.preload-symbols = true`
* Attaching to a process with `target.preload-symbols = true`.
It's completely controlled by a new flag added in the first commit
`plugin.dynamic-loader.posix-dyld.parallel-module-load`, which *defaults
to false*. This was inspired by similar work on Darwin #110646.
Some rough numbers to showcase perf improvement, run on a very beefy
machine:
* Executable with ~5600 modules: baseline 45s, improvement 15s
* Executable with ~3800 modules: baseline 25s, improvement 10s
* Executable with ~6650 modules: baseline 67s, improvement 20s
* Executable with ~12500 modules: baseline 185s, improvement 85s
* Executable with ~14700 modules: baseline 235s, improvement 120s
A lot of targets we deal with have a *ton* of modules, and unfortunately
we're unable to convince other folks to reduce the number of modules, so
performance improvements like this can be very impactful for user
experience.
This patch achieves the performance improvement by parallelizing
`DynamicLoaderPOSIXDYLD::RefreshModules` for the launch scenario, and
`DynamicLoaderPOSIXDYLD::LoadAllCurrentModules` for the attach scenario.
The commits have some context on their specific changes as well --
hopefully this helps the review.
# More context on implementation
We discovered the bottlenecks by via `perf record -g -p <lldb's pid>` on
a Linux machine. With an executable known to have 1000s of shared
library dependencies, I ran
```
(lldb) b main
(lldb) r
# taking a while
```
and showed the resulting perf trace (snippet shown)
```
Samples: 85K of event 'cycles:P', Event count (approx.): 54615855812
Children Self Command Shared Object Symbol
- 93.54% 0.00% intern-state libc.so.6 [.] clone3
clone3
start_thread
lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(void*) r
std::_Function_handler<void* (), lldb_private::Process::StartPrivateStateThread(bool)::$_0>::_M_invoke(std::_Any_data const&)
lldb_private::Process::RunPrivateStateThread(bool) n
- lldb_private::Process::HandlePrivateEvent(std::shared_ptr<lldb_private::Event>&)
- 93.54% lldb_private::Process::ShouldBroadcastEvent(lldb_private::Event*)
- 93.54% lldb_private::ThreadList::ShouldStop(lldb_private::Event*)
- lldb_private::Thread::ShouldStop(lldb_private::Event*) *
- 93.53% lldb_private::StopInfoBreakpoint::ShouldStopSynchronous(lldb_private::Event*) t
- 93.52% lldb_private::BreakpointSite::ShouldStop(lldb_private::StoppointCallbackContext*) i
lldb_private::BreakpointLocationCollection::ShouldStop(lldb_private::StoppointCallbackContext*) k
lldb_private::BreakpointLocation::ShouldStop(lldb_private::StoppointCallbackContext*) b
lldb_private::BreakpointOptions::InvokeCallback(lldb_private::StoppointCallbackContext*, unsigned long, unsigned long) i
DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit(void*, lldb_private::StoppointCallbackContext*, unsigned long, unsigned lo
- DynamicLoaderPOSIXDYLD::RefreshModules() O
- 93.42% DynamicLoaderPOSIXDYLD::RefreshModules()::$_0::operator()(DYLDRendezvous::SOEntry const&) const u
- 93.40% DynamicLoaderPOSIXDYLD::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, bools
- lldb_private::DynamicLoader::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, boos
- 83.90% lldb_private::DynamicLoader::FindModuleViaTarget(lldb_private::FileSpec const&) o
- 83.01% lldb_private::Target::GetOrCreateModule(lldb_private::ModuleSpec const&, bool, lldb_private::Status*
- 77.89% lldb_private::Module::PreloadSymbols()
- 44.06% lldb_private::Symtab::PreloadSymbols()
- 43.66% lldb_private::Symtab::InitNameIndexes()
...
```
We saw that majority of time was spent in `RefreshModules`, with the
main culprit within it `LoadModuleAtAddress` which eventually calls
`PreloadSymbols`.
At first, `DynamicLoaderPOSIXDYLD::LoadModuleAtAddress` appears fairly
independent -- most of it deals with different files and then getting or
creating Modules from these files. The portions that aren't independent
seem to deal with ModuleLists, which appear concurrency safe. There were
members of `DynamicLoaderPOSIXDYLD` I had to synchronize though: namely
`m_loaded_modules` which `DynamicLoaderPOSIXDYLD` maintains to map its
loaded modules to their link addresses. Without synchronizing this, I
ran into SEGFAULTS and other issues when running `check-lldb`. I also
locked the assignment and comparison of `m_interpreter_module`, which
may be unnecessary.
# Alternate implementations
When creating this patch, another implementation I considered was
directly background-ing the call to `Module::PreloadSymbol` in
`Target::GetOrCreateModule`. It would have the added benefit of working
across platforms generically, and appeared to be concurrency safe. It
was done via `Debugger::GetThreadPool().async` directly. However, there
were a ton of concurrency issues, so I abandoned that approach for now.
# Testing
With the feature active, I tested via `ninja check-lldb` on both Debug
and Release builds several times (~5 or 6 altogether?), and didn't spot
additional failing or flaky tests.
I also tested manually on several different binaries, some with around
14000 modules, but just basic operations: launching, reaching main,
setting breakpoint, stepping, showing some backtraces.
I've also tested with the flag off just to make sure things behave
properly synchronously.
The function had special handling for -1, but that is incompatible with
functions whose entry point is not the first address. Use std::nullopt
instead.
