Add a "shared_cache_path" key-value to the jGetSharedCacheInfo response,
if we can fetch the shared cache path.
If debugserver and the inferior process are running with the same shared
cache UUID, there is a simple SPI to get debugserver's own shared cache
filepath and we will return that.
On newer OSes, there are SPI we can use to get the inferior process'
shared cache filepath, use that if necessary and the SPI are available.
The response for the jGetSharedCacheInfo packet will now look like
{"shared_cache_base_address":6609256448,"shared_cache_uuid":"B69FF43C-DBFD-3FB1-B4FE-A8FE32EA1062","no_shared_cache":false,"shared_cache_private_cache":false,"shared_cache_path":"/System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/dyld_shared_cache_arm64e"}
when we have the full information about the shared cache in the
inferior. There are three possible types of responses:
1. inferior has not yet mapped in a shared cache (read: when stopped at
dyld_start and dyld hasn't started executing yet). In this case, no
"shared_cache_path" is listed. ("shared_cache_base_address" will be 0,
"shared_cache_uuid" will be all-zeroes uuid)
2. inferior has a shared cache, but it is different than debugserver's
and we do not have the new SPI to query the shared cache filepath. No
"shared_cache_path" is listed.
3. We were able to find the shared cache filepath, and it is included in
the response, as above.
I'm not using this information in lldb yet, but changes that build on
this will be forthcoming.
rdar://148939795
This was getting joined in ShutDownExcecptionThread (sic) but not
cleared. So this function was not safe to call twice, since you aren't
supposed to join a thread twice. Sadly, this was called in
MachTask::Clear and MachProcess::Destroy, which are both called when you
tell debugserver to detach.
This didn't seem to cause problems IRL, but the most recent ASAN detects
this as an error and calls ASAN::Die, which was causing all the tests
that ran detach to fail.
I fixed that by moving the clear & test for m_exception_thread to
ShutDownExceptionThread. I also fixed the spelling of that routine. And
that routine was claiming to return a kern_return_t which no one was
checking. It actually returns a kern_return_t if there was a Mach
failure and a Posix error if there was a join failure. Since there's
really nothing you can do but exit if this fails, which is always what
you are in the process of doing when you call this, and since we have
already done all the useful logging in ShutDownExceptionThread, I just
removed the return value.
Remove the unnecessary sleep in MachProcess::AttachForDebug. The
preceding comment makes it seem like it's necessary for synchronization,
though I don't believe that's the case (see below), and even if it were,
sleeping is not a reliable way to achieve that.
The reason I don't believe it's necessary is because after we return, we
synchronize with the exception thread on a state change. The latter will
call and update the process state, which is exactly what we synchronize
on. I was able to verify that this is the first time we change the
process state: i.e., `GetState` doesn't return a different value before
and after the sleep.
On top of that, there are 3 more places where we call ptrace attach
(`PosixSpawnChildForPTraceDebugging`, `SBLaunchForDebug`, and
`BoardServiceLaunchForDebug`) where we don't sleep.
rdar://163952037
I was looking at the calls to `usleep` in debugserver and noticed that
these default arguments are never overwritten. I converted them to
constants in the function, which makes it easier to reason about.
The qSpeedTest packet is used for experiments to determine the optimal
packet size for a given communication medium, e.g. to transfer 10MB of
memory, is it faster to send a hundred 100KB packets or ten 1MB packets.
It creates a packet of the requested size in a stack allocation, but is
not checking that its buffer is large enough for the requested size.
Change this allocation to be on heap, and impose a maximum size that can
be tested (4MB, for now).
rdar://158630250
Review of diffs from lldb's internal debugserver and llvm.org main found
two orphaned changes that should be upstreamed.
First is in MachTask::ExceptionThread where we want to confirm that a
mach exception messages is from the correct process before we process
it.
Second is that we want to run the arm64 register context through
thread_convert_thread_state() after thread_get_state, and before
thread_set_state, to re-sign fp/sp/lr/pc appropriately for ptrauth
(arm64e) processes.
These are almost all for internal-developer-users only so "look at
debugserver.cpp" wasn't unreasonable, but we rarely add any new options
so a simple list of all recognized options isn't a burden to throw in
the help method.
This is a continuation of 68fd102, which did the same thing but only for
StopInfo. Using make_shared is both safer and more efficient:
- With make_shared, the object and the control block are allocated
together, which is more efficient.
- With make_shared, the enable_shared_from_this base class is properly
linked to the control block before the constructor finishes, so
shared_from_this() will be safe to use (though still not recommended
during construction).
Increase specificity by using the correct unit sizes. KBytes is an
abbreviation for kB, 1000 bytes, and the hardware industry as well as
several operating systems have now switched to using 1000 byte kBs.
If this change is acceptable, sometimes GitHub mangles merges to use the
original email of the account. $dayjob asks contributions have my work
email. Thanks!
A few files of lldb dir & few other files had duplicate headers
included. This patch removes those redundancies.
---------
Co-authored-by: Akash Agrawal <akashag@qti.qualcomm.com>
CMake 4 no longer sets the `CMAKE_OSX_SYSROOT` variable by default. If
you've updated to CMake 4 on macOS (e.g. with brew) and try building
LLDB with CMake/ninja, this will yield an error when building
debugserver that clang is unable to run since it tries to compile files
that don't exist.
These files are supposed to be generated by the `mig` process. `mig`
needs the `CMAKE_OSX_SYSROOT` variable in order to work and without it,
it silently fails to generate the files that later on need to be
compiled.
This commit sets this SDK path for mig and will fatal error out of config
when building debugserver without having set CMAKE_OSX_SYSROOT.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
PThreadEvent away from PThreadMutex in preparation for removing it.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
DNBTimer away from that class in preparation for removing PThreadMutex.
On the buildbots:
```
user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/tools/debugserver/source/DNBLog.cpp
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/tools/debugserver/source/DNBLog.cpp:66:15: error: no type named 'recursive_mutex' in namespace 'std'
static std::recursive_mutex g_LogThreadedMutex;
~~~~~^
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/tools/debugserver/source/DNBLog.cpp:67:8: error: no member named 'lock_guard' in namespace 'std'
std::lock_guard<std::recursive_mutex> guard(g_LogThreadedMutex);
~~~~~^
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/tools/debugserver/source/DNBLog.cpp:67:24: error: no member named 'recursive_mutex' in namespace 'std'
std::lock_guard<std::recursive_mutex> guard(g_LogThreadedMutex);
~~~~~^
```
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
MachProcess away from PThreadMutex in preparation for removing it.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
RNBRemote away from PThreadMutex in preparation for removing it.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
MachThread away from PThreadMutex in preparation for removing it.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
MachThreadList away from that class in preparation for removing
PThreadMutex.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
DNBLog away from that class in preparation for removing PThreadMutex.
The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
DNBTimer away from that class in preparation for removing PThreadMutex.
debugserver takes the address of a watchpoint exception and calculates
which watchpoint was responsible for it. There was an off-by-one error
in the range calculation which causes two watchpoints on consecutive
ranges to not correctly identify hits to the second watchpoint. The
result is that lldb wouldn't show the second watchpoint as ever being
hit.
Re-landing this test with a modification to only require two
watchpoints in the test, instead of four. If four watchpoints can
be set, it will test them.
rdar://145107575
This reverts commit 21d912121c9f41385b165a736be787527f5bd7c2.
Failure on the aarch64 ubuntu bot when setting the 4th watchpoint;
may be a hardware limitation on that bot. I thought creating four
watchpoints would be generally safe, but I don't need to do that
for my test, will re-land without it.
debugserver takes the address of a watchpoint exception and calculates
which watchpoint was responsible for it. There was an off-by-one error
in the range calculation which causes two watchpoints on consecutive
ranges to not correctly identify hits to the second watchpoint. The
result is that lldb wouldn't show the second watchpoint as ever being
hit.
rdar://145107575
debugserver isn't saving and restoring the SVE/SME register state around
inferior function calls.
Making arbitrary function calls while in Streaming SVE mode is generally
a poor idea because a NEON instruction can be hit and crash the
expression execution, which is how I missed this, but they should be
handled correctly if the user knows it is safe to do.
Re-landing this change after fixing an incorrect behavior on systems
without SME support.
rdar://146886210
This reverts commit 4e40c7c4bd66d98f529a807dbf410dc46444f4ca.
arm64 CI is getting a failure in
lldb-api.tools/lldb-server.TestGdbRemoteRegisterState.py
with this commit, need to investigate and re-land.
debugserver isn't saving and restoring the SVE/SME register state around
inferior function calls.
Making arbitrary function calls while in Streaming SVE mode is generally
a poor idea because a NEON instruction can be hit and crash the
expression execution, which is how I missed this, but they should be
handled correctly if the user knows it is safe to do.
rdar://146886210
We've been dealing with UBSAN issues around this code for some time now
(see `9c36859b33b386fbfa9599646de1e2ae01158180` and
`1a2122e9e9d1d495fdf337a4a9445b61ca56df6f`). On recent macOS versions, a
UBSAN-enabled debugserver will crash when performing a `memcpy` of the
input `mach_exception_data_t`. The pointer to the beginning of the
exception data may not be aligned on a doubleword boundary, leading to
UBSAN failures such as:
```
$ ./bin/debugserver 0.0.0.0:5555 /Volumes/SSD/llvm-builds/llvm-worktrees/clang-work/build-sanitized-release/tools/lldb/test/Shell/Recognizer/Output/verbose_trap.test.tmp.out
/Volumes/SSD/llvm-builds/llvm-worktrees/clang-work/lldb/tools/debugserver/source/MacOSX/MachException.cpp:35:12: runtime error: store to misaligned address 0x00016ddfa634 for type 'mach_exception_data_type_t *' (aka 'long long *'), which requires 8 byte alignment
0x00016ddfa634: note: pointer points here
02 00 00 00 03 00 01 00 00 00 00 00 11 00 00 00 00 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00
^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Volumes/SSD/llvm-builds/llvm-worktrees/clang-work/lldb/tools/debugserver/source/MacOSX/MachException.cpp:35:12
```
Work around these failures by pretending the input data is a `char*`
buffer.
Drive-by changes:
* I factored out some duplicated code into a static
`AppendExceptionData` and made the types consistent
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
In 2013 we added the QSaveRegisterState and QRestoreRegisterState
packets to checkpoint a thread's register state while executing an
inferior function call, instead of using the g packet to read all
registers into lldb, then the G packet to set them again after the func
call.
Since then, lldb has not sent g/G (except as a bug) - it either asks for
registers individually (p/P) or or asks debugserver to save and restore
the entire register set with these lldb extensions.
Felipe recently had a codepath that fell back to using g/G and found
that it does not work with the modern signed fp/sp/pc/lr registers that
we can get -- it sidesteps around the clearing of the non-addressable
bits that we do when reading/writing them, and results in a crash. (
https://github.com/llvm/llvm-project/pull/132079 )
Instead of fixing that issue, I am removing g/G from debugserver because
it's not needed by lldb, and it will would be easy for future bugs to
creep in to this packet that lldb should not use, but it can
accidentally fall back to and result in subtle bugs.
This does mean that a debugger using debugserver on darwin which doesn't
use QSaveRegisterState/QRestoreRegisterState will need to fall back to
reading & writing each register individually. I'm open to re-evaluating
this decision if this proves to be needed, and supporting these lldb
extensions is onerous.
This fixes an uncommon bug with debugserver controlling an inferior
process that is hitting an internal breakpoint & continuing when
multiple interrupts are sent by SB API to lldb -- with the result being
that lldb never stops the inferior process, ignoring the interrupt/stops
being sent by the driver layer (Xcode, in this case).
In the reproducing setup (which required a machine with unique timing
characteristics), lldb is sent SBProcess::Stop and then shortly after,
SBProcess::SendAsyncInterrupt. The driver process only sees that the
inferior is publicly running at this point, even though it's hitting an
internal breakpoint (new dylib being loaded), disabling the bp, step
instructioning, re-enabling the breakpoint, then continuing.
The packet sequence lldb sends to debugserver looks like
1. vCont;s // instruction step
2. ^c // async interrupt
3. Z.... // re-enable breakpoint
4. c // resume inferior execution
5. ^c // async interrupt
When debugserver needs to interrupt a running process
(`MachProcess::Interrupt`), the main thread in debugserver sends a
SIGSTOP posix signal to the inferior process, and notes that it has sent
this signal by setting `m_sent_interrupt_signo`.
When we send the first async interrupt while instruction stepping, the
signal is sent (probably after the inferior has already stopped) but
lldb can only *receive* the mach exception that includes the SIGSTOP
when the process is running. So at the point of step (3), we have a
SIGSTOP outstanding in the kernel, and
`m_sent_interrupt_signo` is set to SIGSTOP.
When we resume the inferior (`c` in step 4), debugserver sees that
`m_sent_interrupt_signo` is still set for an outstanding SIGSTOP, but at
this point we've already stopped so it's an unnecessary stop. It records
that (1) we've got a SIGSTOP still coming that debugserver sent and (2)
we should ignore it by also setting `m_auto_resume_signo` to the same
signal value.
Once we've resumed the process, the mach exception thread
(`MachTask::ExceptionThread`) receives the outstanding mach exception,
adds it to a queue to be processed
(`MachProcess::ExceptionMessageReceived`) and when we've collected all
outstanding mach exceptions, it calls
`MachProcess::ExceptionMessageBundleComplete` top evaluate them.
`MachProcess::ExceptionMessageBundleComplete` halts the process (without
updating the MachProcess `m_state`) while evaluating them. It sees that
this incoming SIGSTOP was meant to be ignored (`m_auto_resume_signo` is
set), so it `MachProcess::PrivateResume`'s the process again.
At the same time `MachTask::ExceptionThread` is receiving and processing
the ME, `MachProcess::Interrupt` is called with another interrupt that
debugserver received. This method checks that we're still eStateRunning
(we are) but then sees that we have an outstanding SIGSTOP already
(`m_sent_interrupt_signo`) and does nothing, assuming that we will stop
shortly from that one. It then returns to call
`RNBRemote::HandlePacket_last_signal` to print the status -- but because
the process is still `eStateRunning`, this does nothing.
So the first ^c (resulting in a pending SIGSTOP) is received and we
resume the process silently. And the second ^c is ignored because we've
got one interrupt already being processed.
The fix was very simple. In `MachProcess::Interrupt` when we detect that
we have a SIGSTOP out in the wild (`m_sent_interrupt_signo`), we need to
clear `m_auto_resume_signo` which is used to indicate that this SIGSTOP
is meant to be ignored, because it was from before our most recent
resume.
MachProcess::Interrupt holds the `m_exception_and_signal_mutex` mutex
already (after Jonas's commit last week), and all of
`MachProcess::ExceptionMessageBundleComplete` holds that same mutex, so
we know we can modify `m_auto_resume_signo` here and it will be handled
correctly when the outstanding mach exception is finally processed.
rdar://145872120
The mutex in RNBRemote::CommDataReceived protects m_rx_packets and
should extend to the end of the function to cover the read where we
check if the list is empty.
This PR fixes a race condition in debugserver where the main thread
calls MachProcess::Interrupt, setting `m_sent_interrupt_signo` while the
exception monitoring thread is checking the value of the variable.
I was on the fence between introducing a new mutex and reusing the
existing exception mutex. With the notable exception of
MachProcess::Interrupt, all the other places where we were already
locking this mutex before accessing the variable. I renamed the mutex to
make it clear that it's now protecting more than the exception messages.
Jason, while investigating a real issue, had a suspicion there was race
condition related to interrupts and I was able to narrow it down by
building debugserver with TSan.
Recognize the visionOS Triple::OSType::XROS os type. Some of these have
already been landed on main, but I reviewed the downstream sources and
there were a few that still needed to be landed upstream.
**Note:** The register reading and writing depends on new register
flavor support in thread_get_state/thread_set_state in the kernel, which
will be first available in macOS 15.4.
The Apple M4 line of cores includes the Scalable Matrix Extension (SME)
feature. The M4s do not implement Scalable Vector Extension (SVE),
although the processor is in Streaming SVE Mode when the SME is being
used. The most obvious side effects of being in SSVE Mode are that (on
the M4 cores) NEON instructions cannot be used, and watchpoints may get
false positives, the address comparisons are done at a lowered
granularity.
When SSVE mode is enabled, the kernel will provide the Streaming Vector
Length register, which is a maximum of 64 bytes with the M4. Also
provided are SVCR (with bits indicating if SSVE mode and SME mode are
enabled), TPIDR2, SVL. Then the SVE registers Z0..31 (SVL bytes long),
P0..15 (SVL/8 bytes), the ZA matrix register (SVL*SVL bytes), and the M4
supports SME2, so the ZT0 register (64 bytes).
When SSVE/SME are disabled, none of these registers are provided by the
kernel - reads and writes of them will fail.
Unlike Linux, lldb cannot modify the SVL through a thread_set_state
call, or change the processor state's SSVE/SME status. There is also no
way for a process to request a lowered SVL size today, so the work that
David did to handle VL/SVL changing while stepping through a process is
not an issue on Darwin today. But debugserver should be providing
everything necessary so we can reuse all of David's work on resizing the
register contexts in lldb if it happens in the future. debugbserver
sends svl, svcr, and tpidr2 in the expedited registers when a thread
stops, if SSVE|SME mode are enabled (if the kernel allows it to read the
ARM_SME_STATE register set).
While the maximum SVL is 64 bytes on M4, the AArch64 maximum possible
SVL is 256; this would give us a 64k ZA register. If debugserver sized
all of its register contexts assuming the largest possible SVL, we could
easily use 2MB more memory for the register contexts of all threads in a
process -- and on iOS et al, processes must run within a small memory
allotment and this would push us over that.
Much of the work in debugserver was changing the arm64 register context
from being a static compile-time array of register sets, to being
initialized at runtime if debugserver is running on a machine with SME.
The ZA is only created to the machine's actual maximum SVL. The size of
the 32 SVE Z registers is less significant so I am statically allocating
those to the architecturally largest possible SVL value today.
Also, debugserver includes information about registers that share the
same part of the register file. e.g. S0 and D0 are the lower parts of
the NEON 128-bit V0 register. And when running on an SME machine, v0 is
the lower 128 bits of the SVE Z0 register. So the register maps used
when defining the VFP registers must differ depending on the
capabilities of the cpu at runtime.
I also changed register reading in debugserver, where formerly when
debugserver was asked to read a register, and the thread_get_state read
of that register failed, it would return all zero's. This is necessary
when constructing a `g` packet that gets all registers - because there
is no separation between register bytes, the offsets are fixed. But when
we are asking for a single register (e.g. Z0) when not in SSVE/SME mode,
this should return an error.
This does mean that when you're running on an SME capabable machine, but
not in SME mode, and do `register read -a`, lldb will report that 48 SVE
registers were unavailable and 5 SME registers were unavailable. But
that's only when `-a` is used.
The register reading and writing depends on new register flavor support
in thread_get_state/thread_set_state in the kernel, which is not yet in
a release. The test case I wrote is skipped on current OSes. I pilfered
the SME register setup from some of David's existing SME test files;
there were a few Linux specific details in those tests that they weren't
easy to reuse on Darwin.
rdar://121608074
