The debugserver code predates modern C++, but with C++11 and later
there's no need to have something like PThreadMutex. This migrates
MachProcess away from PThreadMutex in preparation for removing it.
This fixes an uncommon bug with debugserver controlling an inferior
process that is hitting an internal breakpoint & continuing when
multiple interrupts are sent by SB API to lldb -- with the result being
that lldb never stops the inferior process, ignoring the interrupt/stops
being sent by the driver layer (Xcode, in this case).
In the reproducing setup (which required a machine with unique timing
characteristics), lldb is sent SBProcess::Stop and then shortly after,
SBProcess::SendAsyncInterrupt. The driver process only sees that the
inferior is publicly running at this point, even though it's hitting an
internal breakpoint (new dylib being loaded), disabling the bp, step
instructioning, re-enabling the breakpoint, then continuing.
The packet sequence lldb sends to debugserver looks like
1. vCont;s // instruction step
2. ^c // async interrupt
3. Z.... // re-enable breakpoint
4. c // resume inferior execution
5. ^c // async interrupt
When debugserver needs to interrupt a running process
(`MachProcess::Interrupt`), the main thread in debugserver sends a
SIGSTOP posix signal to the inferior process, and notes that it has sent
this signal by setting `m_sent_interrupt_signo`.
When we send the first async interrupt while instruction stepping, the
signal is sent (probably after the inferior has already stopped) but
lldb can only *receive* the mach exception that includes the SIGSTOP
when the process is running. So at the point of step (3), we have a
SIGSTOP outstanding in the kernel, and
`m_sent_interrupt_signo` is set to SIGSTOP.
When we resume the inferior (`c` in step 4), debugserver sees that
`m_sent_interrupt_signo` is still set for an outstanding SIGSTOP, but at
this point we've already stopped so it's an unnecessary stop. It records
that (1) we've got a SIGSTOP still coming that debugserver sent and (2)
we should ignore it by also setting `m_auto_resume_signo` to the same
signal value.
Once we've resumed the process, the mach exception thread
(`MachTask::ExceptionThread`) receives the outstanding mach exception,
adds it to a queue to be processed
(`MachProcess::ExceptionMessageReceived`) and when we've collected all
outstanding mach exceptions, it calls
`MachProcess::ExceptionMessageBundleComplete` top evaluate them.
`MachProcess::ExceptionMessageBundleComplete` halts the process (without
updating the MachProcess `m_state`) while evaluating them. It sees that
this incoming SIGSTOP was meant to be ignored (`m_auto_resume_signo` is
set), so it `MachProcess::PrivateResume`'s the process again.
At the same time `MachTask::ExceptionThread` is receiving and processing
the ME, `MachProcess::Interrupt` is called with another interrupt that
debugserver received. This method checks that we're still eStateRunning
(we are) but then sees that we have an outstanding SIGSTOP already
(`m_sent_interrupt_signo`) and does nothing, assuming that we will stop
shortly from that one. It then returns to call
`RNBRemote::HandlePacket_last_signal` to print the status -- but because
the process is still `eStateRunning`, this does nothing.
So the first ^c (resulting in a pending SIGSTOP) is received and we
resume the process silently. And the second ^c is ignored because we've
got one interrupt already being processed.
The fix was very simple. In `MachProcess::Interrupt` when we detect that
we have a SIGSTOP out in the wild (`m_sent_interrupt_signo`), we need to
clear `m_auto_resume_signo` which is used to indicate that this SIGSTOP
is meant to be ignored, because it was from before our most recent
resume.
MachProcess::Interrupt holds the `m_exception_and_signal_mutex` mutex
already (after Jonas's commit last week), and all of
`MachProcess::ExceptionMessageBundleComplete` holds that same mutex, so
we know we can modify `m_auto_resume_signo` here and it will be handled
correctly when the outstanding mach exception is finally processed.
rdar://145872120
This PR fixes a race condition in debugserver where the main thread
calls MachProcess::Interrupt, setting `m_sent_interrupt_signo` while the
exception monitoring thread is checking the value of the variable.
I was on the fence between introducing a new mutex and reusing the
existing exception mutex. With the notable exception of
MachProcess::Interrupt, all the other places where we were already
locking this mutex before accessing the variable. I renamed the mutex to
make it clear that it's now protecting more than the exception messages.
Jason, while investigating a real issue, had a suspicion there was race
condition related to interrupts and I was able to narrow it down by
building debugserver with TSan.
**Note:** The register reading and writing depends on new register
flavor support in thread_get_state/thread_set_state in the kernel, which
will be first available in macOS 15.4.
The Apple M4 line of cores includes the Scalable Matrix Extension (SME)
feature. The M4s do not implement Scalable Vector Extension (SVE),
although the processor is in Streaming SVE Mode when the SME is being
used. The most obvious side effects of being in SSVE Mode are that (on
the M4 cores) NEON instructions cannot be used, and watchpoints may get
false positives, the address comparisons are done at a lowered
granularity.
When SSVE mode is enabled, the kernel will provide the Streaming Vector
Length register, which is a maximum of 64 bytes with the M4. Also
provided are SVCR (with bits indicating if SSVE mode and SME mode are
enabled), TPIDR2, SVL. Then the SVE registers Z0..31 (SVL bytes long),
P0..15 (SVL/8 bytes), the ZA matrix register (SVL*SVL bytes), and the M4
supports SME2, so the ZT0 register (64 bytes).
When SSVE/SME are disabled, none of these registers are provided by the
kernel - reads and writes of them will fail.
Unlike Linux, lldb cannot modify the SVL through a thread_set_state
call, or change the processor state's SSVE/SME status. There is also no
way for a process to request a lowered SVL size today, so the work that
David did to handle VL/SVL changing while stepping through a process is
not an issue on Darwin today. But debugserver should be providing
everything necessary so we can reuse all of David's work on resizing the
register contexts in lldb if it happens in the future. debugbserver
sends svl, svcr, and tpidr2 in the expedited registers when a thread
stops, if SSVE|SME mode are enabled (if the kernel allows it to read the
ARM_SME_STATE register set).
While the maximum SVL is 64 bytes on M4, the AArch64 maximum possible
SVL is 256; this would give us a 64k ZA register. If debugserver sized
all of its register contexts assuming the largest possible SVL, we could
easily use 2MB more memory for the register contexts of all threads in a
process -- and on iOS et al, processes must run within a small memory
allotment and this would push us over that.
Much of the work in debugserver was changing the arm64 register context
from being a static compile-time array of register sets, to being
initialized at runtime if debugserver is running on a machine with SME.
The ZA is only created to the machine's actual maximum SVL. The size of
the 32 SVE Z registers is less significant so I am statically allocating
those to the architecturally largest possible SVL value today.
Also, debugserver includes information about registers that share the
same part of the register file. e.g. S0 and D0 are the lower parts of
the NEON 128-bit V0 register. And when running on an SME machine, v0 is
the lower 128 bits of the SVE Z0 register. So the register maps used
when defining the VFP registers must differ depending on the
capabilities of the cpu at runtime.
I also changed register reading in debugserver, where formerly when
debugserver was asked to read a register, and the thread_get_state read
of that register failed, it would return all zero's. This is necessary
when constructing a `g` packet that gets all registers - because there
is no separation between register bytes, the offsets are fixed. But when
we are asking for a single register (e.g. Z0) when not in SSVE/SME mode,
this should return an error.
This does mean that when you're running on an SME capabable machine, but
not in SME mode, and do `register read -a`, lldb will report that 48 SVE
registers were unavailable and 5 SME registers were unavailable. But
that's only when `-a` is used.
The register reading and writing depends on new register flavor support
in thread_get_state/thread_set_state in the kernel, which is not yet in
a release. The test case I wrote is skipped on current OSes. I pilfered
the SME register setup from some of David's existing SME test files;
there were a few Linux specific details in those tests that they weren't
easy to reuse on Darwin.
rdar://121608074
In DNBArchImplARM64.cpp I'm doing
```
#if __has_feature(ptrauth_calls) && defined(__LP64__)
```
And the preprocessor warns that this is not defined behavior. This
checks if ptrauth_calls is available and if this is being compiled
64-bit (i.e. arm64e), and defines a single DEBUGSERVER_IS_ARM64E when
those are both true.
I did have to duplicate one DNBLogThreaded() call which itself is a
macro, and using an ifdef in the middle of macro arguments also got me a
warning from the preprocessor.
While testing this for all the different targets, I found a DNBError
initialization that accepts a c-string but I'm passing in a printf-style
formatter c-string and an argument. Create the string before the call
and pass in the constructed string.
rdar://127129242
There's a bad interaction between the macOS 14 dyld and the "dyld_sim"
shim that comes from older (iOS 15) simulator downloads that results in
dyld reporting some modules twice in the return from the dyld callback
to list modules. The records were identical, but lldb wasn't happy with
seeing the duplicates...
Since it's not possible to load two different modules at the same
address, this change just picks the first instance of any entries that
have the same load address.
There really isn't a good way to test this patch.
When debugserver is attaching to a process, it first task_for_pid()'s
and then ptrace(PT_ATTACHEXC)'s. When that ptrace() fails to complete,
we are in a semi-attached state that we need to give up from, and
our error reporting isn't ideal -- we can even claim that the process
is already being debugged (by ourselves).
Differential Revision: https://reviews.llvm.org/D155037
rdar://101152233
jGetLoadedDynamicLibrariesInfos has a mode where it will list
every binary in the process - the load address and filepath from dyld
SPI, and the mach-o header and load commands from a scan by debugserver
for perf reasons. With a large enough number of libraries, creating
that StructuredData representation of all of this, and formatting it
into an ascii string to send up to lldb, can grow debugserver's heap
size too large for some environments.
This patch adds a new report_load_commands:false boolean to the
jGetLoadedDynamicLibrariesInfos packet, where debugserver will now
only report the dyld SPI load address and filepath for all of the
binaries. lldb can then ask for the detailed information on
the process binaries in smaller chunks, and avoid debugserver
having ever growing heap use as the number of binaries inevitably
increases.
This patch also removes a version of jGetLoadedDynamicLibrariesInfos
for pre-iOS 10 and pre-macOS 10.12 systems where we did not use
dyld SPI. We can't back compile to those OS builds any longer
with modern Xcode.
Finally, it removes a requirement in DynamicLoaderMacOS that the
JSON reply from jGetLoadedDynamicLibrariesInfos include the
mod_date field for each binary. This has always been reported as
0 in modern dyld, and is another reason for packet growth in
the reply. debugserver still puts the mod_date field in its replies
for interop with existing lldb's, but we will be able to remove it
the field from debugserver's output after the next release cycle
when this patch has had time to circulate.
I'll add lldb support for requesting the load addresses only
and splitting the request up into chunks in a separate patch.
Differential Revision: https://reviews.llvm.org/D150158
rdar://107848326
Have debugserver parse the watchpoint flags out of the exception
syndrome register when we get a watchpoint mach exception. Relay
those fields up to lldb in the stop reply packet, if the watchpoint
number was reported by the hardware, use the address from that as
the watchpoint address.
Change how watchpoints are reported to lldb from using the mach
exception data, to using the `reason:watchpoint` and `description:asciihex`
method that lldb-server uses, which can relay the actual trap address
as well as the address of a watched memory region responsible for
the trap, so lldb can step past it.
Have debugserver look for the nearest watchpoint that it has set
when it gets a watchpoint trap, so accesses that are reported as
starting before the watched region are associated with the correct
watchpoint to lldb. Add a test case for this specific issue.
Differential Revision: https://reviews.llvm.org/D147820
rdar://83996471
After an exec, the inferior is a new process and none of these memory
regions are still allocated. Clear them out.
Differential Revision: https://reviews.llvm.org/D140253
The dynamic linker on Darwin, dyld, can provide status of
the process state for a few significant points early on,
most importantly, when libSystem has been initialized and it
is safe to call functions behind the scenes. Pipe this
information up from debugserver to DynamicLoaderMacOS, for
the DynamicLoader::IsFullyInitialized() method, then have
Thread::SafeToCallFunctions use this information. Finally,
for the two utility functions in the AppleObjCRuntimeV2
LanguageRuntime plugin that I was fixing, call this method
before running our utility functions to collect the list of
objc classes registered in the runtime.
User expressions will still be allowed to run any time -
we assume the user knows what they are doing - but these
two additional utility functions that they are unaware of
will be limited by this state.
Differential Revision: https://reviews.llvm.org/D139054
rdar://102436092
can probably make function calls.
Change debugserver's posix_spawn() to spawn an inferior process in
its own process group, so signals from the terminal like control-c
are passed to the inferior process instead of debugserer. In lldb's
own native-host launching, there is a LaunchInfo option
LaunchInSeparateProcessGroup, and this mirrors the most common
setting of that on Darwin systems.
Patch by Alessandro Arzilli <alessandro.arzilli@gmail.com>.
Differential Revision: https://reviews.llvm.org/D128504
debugserver parses the Mach-O header & load commands of
binaries; if it does this with a binary whose LC_BUILD
platform enum it does not recognize, it will currently crash.
This patch changes MachProcss::GetPlatformString to return
an optional platform string, and updates the callers to
do the right thing when this optional could not be
provided.
Differential Revision: https://reviews.llvm.org/D136719
rdar://100452994
jGetLoadedDynamicLibrariesInfos normally checks with dyld to find
the list of binaries loaded in the inferior, and getting the filepath,
before trying to parse the Mach-O binary in inferior memory.
This allows for debugserver to parse a Mach-O binary present in memory,
but not yet registered with dyld. This patch also adds some simple
sanity checks that we're reading a Mach-O header before we begin
stepping through load commands, because we won't have the sanity check
of consulting dyld for the list of loaded binaries before parsing.
Also adds a testcase.
[This patch was reverted after causing a testsuite failure on a CI bot;
I haven't been able to repro the failure outside the CI, but I have a
theory that my sanity check on cputype which only matched arm64 and
x86_64 - and the CI machine may have a watch simulator that is still
using i386.]
Differential Revision: https://reviews.llvm.org/D128956
rdar://95737734
jGetLoadedDynamicLibrariesInfos normally checks with dyld to find
the list of binaries loaded in the inferior, and getting the filepath,
before trying to parse the Mach-O binary in inferior memory.
This allows for debugserver to parse a Mach-O binary present in memory,
but not yet registered with dyld. This patch also adds some simple
sanity checks that we're reading a Mach-O header before we begin
stepping through load commands, because we won't have the sanity check
of consulting dyld for the list of loaded binaries before parsing.
Also adds a testcase.
Differential Revision: https://reviews.llvm.org/D128956
rdar://95737734
should not receive as exceptions (some will get converted to BSD
signals instead). This is really the only stable way to ensure that
a Mach exception gets converted to it's equivalent BSD signal. For
programs that rely on BSD signal handlers, this has to happen or you
can't even get the program to invoke the signal handler when under
the debugger.
This builds on a previous solution to this problem which required you
start debugserver with the -U flag. This was not very discoverable
and required lldb be the one to launch debugserver, which is not always
the case.
Differential Revision: https://reviews.llvm.org/D125434
The dyld SPI used by debugserver (_dyld_process_info_create) has become
much slower in macOS BigSur 11.3 causing a significant performance
regression when attaching. This commit mitigates that by caching the
result when calling the SPI to compute the platform.
Differential revision: https://reviews.llvm.org/D102833
Fix the logic to find the app bundle in a path by correctly accounting
for paths containing multiple occurrences of `.app`. The new logic will
correctly extract `com.app.Foo.app` from `com.app.Foo.app/com.app.Foo`.
rdar://74666208
Differential revision: https://reviews.llvm.org/D97441
Debugging app launch/attach failures can be difficult because of
all of the messages logged to the console on a darwin system;
emitting specific messages around critical API calls can make it
easier to narrow the search for the console messages related to
the failure.
<rdar://problem/67220442>
Differential revision: https://reviews.llvm.org/D94357
Use the newly added spawnattr API, posix_spawnattr_setarchpref_np, to
select a slice preferences per cpu and subcpu types, instead of just cpu
with posix_spawnattr_setarchpref_np.
rdar://16094957
Differential revision: https://reviews.llvm.org/D92712
Adds a command line option that makes debugserver propagate the SIGSEGV
signal to the target process.
Motivation: I'm one of the maintainers of Delve [1] a debugger for Go.
We use debugserver as our backend on macOS and one of the most often
reported bugs is that, on macOS, we don't propagate SIGSEGV back to the
target process [2]. Sometimes some programs will actually cause a
SIGSEGV, by design, and then handle it. Those programs can not be
debugged at all.
Since catching signals isn't very important for a Go debugger I'd much
rather have a command line option in debugserver that causes it to let
SIGSEGV go directly to the target process.
[1] https://github.com/go-delve/delve/
[2] https://github.com/go-delve/delve/issues/852
Differential revision: https://reviews.llvm.org/D89315
Link against CarouselServices on watchos, recognize the
WatchComplicationLaunch launch flag option when that framework
is available.
<rdar://problem/62473967>, <rdar://problem/61230088>
This patch is similar in spirit to https://reviews.llvm.org/D84480,
but does the maccatalyst/macosx disambiguation. I also took the
opportunity to factor out the gdb-remote packet log scanning used by
several testcases into lldbutil functions.
rdar://problem/66059257
Differential Revision: https://reviews.llvm.org/D84576
This patch basically moves the disambiguation code from a place where
it was complicated to implement straight to where the load command is
parsed, which has the neat side affect of actually supporting all call
sites!
rdar://problem/66011909
Differential Revision: https://reviews.llvm.org/D84480
This patch fixes debugserver incorrectly returning the SDK version
instead of the minimum deployment target version.
rdar://problem/65001691
Differential Revision: https://reviews.llvm.org/D83443
This reverts commit 0da0437b2afbd8ebef6b11f114cca33b118e7639 to unbreak
the following tests:
lldb-api.tools/lldb-server.TestAppleSimulatorOSType.py
lldb-api.tools/lldb-server.TestGdbRemoteAttach.py
lldb-api.tools/lldb-server.TestGdbRemoteProcessInfo.py
lldb-api.tools/lldb-server.TestGdbRemoteRegisterState.py
lldb-api.tools/lldb-server.TestGdbRemoteThreadsInStopReply.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py
debugserver and lldb
This patch improves the heuristics for correctly identifying simulator binaries on Darwin and adds support for simulators running on Apple Silicon.
rdar://problem/64046344
Differential Revision: https://reviews.llvm.org/D82616
MachProcess.mm uses a TARGET_OS_ macro without directly including
TargetConditionals.h. This currently works as we get the header
as an indirect dependency, but might not in the future.
I just spent some time investigating an internal regression
caused by a similar issue, so I audited the codebase for such
cases.
Update CallBoardSystemServiceOpenApplication to unconditionally log
the NSError's localizedDescription to Console on app launch failure
(as it was already doing), and also to log the NSError object's
full description to the console, which may contain additional nested
error messages. I'm experimenting to find cases where we will get
more detailed information from app launch failures and will start
by logging both to the console.
<rdar://problem/62709160>
Starting with iOS 13 simulator binaries are identified with an
explicit platform in the new LC_BUILD_VERSION load command.
On older deployment targets using the LC_VERSION_MIN load commands,
this patch detects when an ios process runs on a macOS host and
updates the target triple with the "simulator" environment
accordingly.
(Patch re-applied with bugfix this time).
This is part of https://bugs.swift.org/browse/SR-11971
rdar://problem/58438125
Differential Revision: https://reviews.llvm.org/D75696
Starting with iOS 13 simulator binaries are identified with an
explicit platform in the new LC_BUILD_VERSION load command.
On older deployment targets using the LC_VERSION_MIN load commands,
this patch detects when an ios process runs on a macOS host and
updates the target triple with the "simulator" environment
accordingly.
(Patch re-applied without modifications, the bot failure was unrelated).
This is part of https://bugs.swift.org/browse/SR-11971
rdar://problem/58438125
Differential Revision: https://reviews.llvm.org/D75696
Starting with iOS 13 simulator binaries are identified with an
explicit platform in the new LC_BUILD_VERSION load command.
On older deployment targets using the LC_VERSION_MIN load commands,
this patch detects when an ios process runs on a macOS host and
updates the target triple with the "simulator" environment
accordingly.
This is part of https://bugs.swift.org/browse/SR-11971
rdar://problem/58438125
Differential Revision: https://reviews.llvm.org/D75696
The debugserver profile thread used to suspend itself between samples with
a usleep. When you detach or kill, MachProcess::Clear would delay replying
to the incoming packet until pthread_join of the profile thread returned.
If you are unlucky or the suspend delay is long, it could take longer than
the packet timeout for pthread_join to return. Then you would get an error
about detach not succeeding from lldb - even though in fact the detach was
successful...
I replaced the usleep with PThreadEvents entity. Then we just call a timed
WaitForEventBits, and when debugserver wants to stop the profile thread, it
can set the event bit, and the sleep will exit immediately.
Differential Revision: https://reviews.llvm.org/D75004
