lldb-server had limited support for single-stepping through the lr/sc
atomic sequence. This patch enhances that support for all possible
atomic sequences.
lldb-server built with NativeProcessLinux.cpp and
NativeProcessFreeBSD.cpp can use breakpoints to implement instruction
stepping on cores where there is no native instruction-step primitive.
Currently these set a breakpoint, continue, and if we hit the breakpoint
with the original thread, set the stop reason to be "trace".
I am wrapping up a change to lldb's breakpoint algorithm where I change
its current behavior of
"if a thread stops at a breakpoint site, we set
the thread's stop reason to breakpoint-hit, even if the breakpoint
hasn't been executed" +
"when resuming any thread at a breakpoint site, instruction-step past
the breakpoint before resuming"
to a behavior of
"when a thread executes a breakpoint, set the stop reason to
breakpoint-hit" +
"when a thread has hit a breakpoint, when the thread resumes, we
silently step past the breakpoint and then resume the thread".
For these lldb-server targets doing breakpoint stepping, this means that
if we are sitting on a breakpoint that has not yet executed, and
instruction-step the thread, we will execute the breakpoint instruction
at $pc (instead of $next-pc where it meant to go), and stop again -- at
the same pc value. Then we will rewrite the stop reason to 'trace'. The
higher level logic will see that we haven't hit the breakpoint
instruction again, so it will try to instruction step again, hitting the
breakpoint again forever.
To fix this, I'm checking that the thread matches the one we are
instruction-stepping-by-breakpoint AND that we've stopped at the
breakpoint address we are stepping to. Only in that case will the stop
reason be rewritten to "trace" hiding the implementation detail that the
step was done by breakpoints.
Previously, we were returning an error if we couldn't read the whole
region. This doesn't matter most of the time, because lldb caches memory
reads, and in that process it aligns them to cache line boundaries. As
(LLDB) cache lines are smaller than pages, the reads are unlikely to
cross page boundaries.
Nonetheless, this can cause a problem for large reads (which bypass the
cache), where we're unable to read anything even if just a single byte
of the memory is unreadable. This patch fixes the lldb-server to do
that, and also changes the linux implementation, to reuse any partial
results it got from the process_vm_readv call (to avoid having to
re-read everything again using ptrace, only to find that it stopped at
the same place).
This matches debugserver behavior. It is also consistent with the gdb
remote protocol documentation, but -- notably -- not with actual
gdbserver behavior (which returns errors instead of partial results). We
filed a
[clarification
bug](https://sourceware.org/bugzilla/show_bug.cgi?id=24751) several
years ago. Though we did not really reach a conclusion there, I think
this is the most logical behavior.
The associated test does not currently pass on windows, because the
windows memory read APIs don't support partial reads (I have a WIP patch
to work around that).
This patch removes all of the Set.* methods from Status.
This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.
This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()
Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form
` ResultTy DoFoo(Status& error)
`
to
` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?
The interesting changes are in Status.h and Status.cpp, all other
changes are mostly
` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.
Fixes#85084
Whenever an inferior thread stops, lldb-server sends a SIGSTOP to all
other threads in the process to force them to stop as well. If those
threads stop on their own before they get a signal, this SIGSTOP will
remain pending and be delivered the next time the process resumes.
Normally, this is not a problem, because lldb-server will detect this
stale SIGSTOP and resume the process. However, if we detach from the
process while it has these SIGSTOPs pending, they will get immediately
delivered, and the process will remain stopped (most likely forever).
This patch fixes that by sending a SIGCONT just before detaching from
the process. This signal cancels out any pending SIGSTOPs, and ensures
it is able to run after we detach. It does have one somewhat unfortunate
side-effect that in that the process's SIGCONT handler (if it has one)
will get executed spuriously (from the process's POV).
This could be _sometimes_ avoided by tracking which threads got send a
SIGSTOP, and whether those threads stopped due to it. From what I could
tell by observing its behavior, this is what gdb does. I have not tried
to replicate that behavior here because it adds a nontrivial amount of
complexity and the result is still uncertain -- we still need to send a
SIGCONT (and execute the handler) when any thread stops for some other
reason (and leaves our SIGSTOP hanging). Furthermore, since SIGSTOPs
don't stack, it's also possible that our SIGSTOP/SIGCONT combination
will cancel a genuine SIGSTOP being sent to the debugger application (by
someone else), and there is nothing we can do about that. For this
reason I think it's simplest and most predictible to just always send a
SIGCONT when detaching, but if it turns out this is breaking something,
we can consider implementing something more elaborate.
One alternative I did try is to use PTRACE_INTERRUPT to suspend the
threads instead of a SIGSTOP. PTRACE_INTERUPT requires using
PTRACE_SEIZE to attach to the process, which also made this solution
somewhat complicated, but the main problem with that approach is that
PTRACE_INTERRUPT is not considered to be a signal-delivery-stop, which
means it's not possible to resume it while injecting another signal to
the inferior (which some of our tests expect to be able to do). This
limitation could be worked around by forcing the thread into a signal
delivery stop whenever we need to do this, but this additional
complication is what made me think this approach is also not worthwhile.
This patch should fix (at least some of) the problems with
TestConcurrentVFork, but I've also added a dedicated test for checking
that a process keeps running after we detach. Although the problem I'm
fixing here is linux-specific, the core functinoality of not stopping
after a detach should function the same way everywhere.
This is a follow-up to D116372, which had a rather unfortunate side
effect of making the processing of a single SIGCHLD quadratic in the
number of threads -- which does not matter for simple applications, but
can get really bad for applications with thousands of threads.
This patch fixes the problem by implementing the other possibility
mentioned in the first patch -- doing waitpid(-1) centrally and then
routing the events to the correct process instance. The "uncollected"
threads are held in the process factory class -- which I've renamed to
Manager for this purpose, as it now does more than creating processes.
Differential Revision: https://reviews.llvm.org/D146977
A common reason for LLDB failing to attach to an already-running process on Linux is the Yama security module: https://www.kernel.org/doc/Documentation/security/Yama.txt. This patch adds an explaination and suggested fix when it detects that case happening.
This was previously proposed in D106226, but hasn't been updated in a while. The last request was to put the check in a target-specific location, which is the approach this patch takes. I believe Yama only exists on Linux, so it's put in that package.
This has no end-to-end test because I'm not sure how to set `ptrace_scope` in a test environment -- if there are suggestions on how to do that, I'd be happy to add it. (Also, setting it to `3` is comically irreversible). I tested this locally.
Reviewed By: DavidSpickett, labath
Differential Revision: https://reviews.llvm.org/D144904
Hardware single stepping is not currently supported by the linux kernel.
In order to support single step debugging, add EmulateInstructionLoongArch
to implement the software Single Stepping. This patch only support the
simplest single step execution of non-jump instructions.
Reviewed By: SixWeining, DavidSpickett
Differential Revision: https://reviews.llvm.org/D139158
Hardware single stepping is not currently supported by the linux kernel.
In order to support single step debugging, add EmulateInstructionLoongArch
to implement the software Single Stepping. This patch only support the
simplest single step execution of non-jump instructions.
Reviewed By: SixWeining, DavidSpickett
Differential Revision: https://reviews.llvm.org/D139158
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Add:
- `EmulateInstructionRISCV`, which can be used for riscv32 and riscv64.
- Add unittests for EmulateInstructionRISCV.
Note: Compressed instructions set (RVC) was still not supported in this patch.
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D131759
In this switch case we didn't handle possible errors in `ResumeThread()`, it's hard to get helpful information when it goes wrong.
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D131946
Currently, lldb-server was opening the executable file to determine the
process architecture (to differentiate between 32 and 64 bit
architecture flavours). This isn't a particularly trustworthy source of
information (the file could have been changed since the process was
started) and it is not always available (file could be deleted or
otherwise inaccessible).
Unfortunately, ptrace does not give us a direct API to access the
process architecture, but we can still infer it via some of its
responses -- given that the general purpose register set of 64-bit
applications is larger [citation needed] than the GPR set of 32-bit
ones, we can just ask for the application GPR set and check its size.
This is what this patch does.
Differential Revision: https://reviews.llvm.org/D130985
Implement support for the "t" action that is used to stop a thread.
Normally this action is used only in non-stop mode. However, there's
no technical reason why it couldn't be also used in all-stop mode,
e.g. to express "resume all threads except ..." (`t:...;c`).
While at it, add a more complete test for vCont correctly resuming
a subset of program's threads.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.llvm.org/D126983
- Add collection of context switches per cpu grouped with the per-cpu intel pt traces.
- Move the state handling from the interl pt trace class to the PerfEvent one.
- Add support for stopping and enabling perf event groups.
- Return context switch entries as part of the jLLDBTraceGetState response.
- Move the triggers of whenever the process stopped or resumed. Now the will-resume notification is in a better location, which will ensure that we'll capture the instructions that will be executed.
- Remove IntelPTSingleBufferTraceUP. The unique pointer was useless.
- Add unit tests
Differential Revision: https://reviews.llvm.org/D125897
When tracing on per-core mode, we are tracing all processes, which means
that after hitting a breakpoint, our process will stop running (thus
producing no more tracing data) but other processes will continue
writing to our trace buffers. This causes a big data loss for our trace.
As a way to remediate this, I'm adding some logic to pause and unpause
tracing based on the target's state. The earlier we do it the better,
however, I'm not adding the trigger at the earliest possible point for
simplicity of this diff. Later we can improve that part.
Differential Revision: https://reviews.llvm.org/D124962
This diffs implements per-core tracing on lldb-server. It also includes tests that ensure that tracing can be initiated from the client and that the jLLDBGetState ppacket returns the list of trace buffers per core.
This doesn't include any decoder changes.
Finally, this makes some little changes here and there improving the existing code.
A specific piece of code that can't reliably be tested is when tracing
per core fails due to permissions. In this case we add a
troubleshooting message and this is the manual test:
```
/proc/sys/kernel/perf_event_paranoid set to 1
(lldb) process trace start --per-core-tracing error: perf event syscall failed: Permission denied
You might need that /proc/sys/kernel/perf_event_paranoid has a value of 0 or -1.
``
Differential Revision: https://reviews.llvm.org/D124858
I'm refactoring IntelPTThreadTrace into IntelPTSingleBufferTrace so that it can
both single threads or single cores. In this diff I'm basically renaming the
class, moving it to its own file, and removing all the pieces that are not used
along with some basic cleanup.
Differential Revision: https://reviews.llvm.org/D124648
This patch handles the situation where the main thread exits (through
the SYS_exit syscall). In this case, the process as a whole continues
running until all of the other threads exit, or one of them issues an
exit_group syscall.
The patch consists of two changes:
- a moderate redesign of the handling of thread exit (WIFEXITED) events.
Previously, we were removing (forgetting) a thread once we received
the WIFEXITED (or WIFSIGNALED) event. This was problematic for the
main thread, since the main thread WIFEXITED event (which is better thought
of as a process-wide event) gets reported only after the entire process
exits. This resulted in deadlocks, where we were waiting for the
process to stop (because we still considered the main thread "live").
This patch changes the logic such that the main thread is removed as
soon as its PTRACE_EVENT_EXIT (the pre-exit) event is received. At
this point we can consider the thread gone (for most purposes). As a
corrolary, I needed to add special logic to catch process-wide exit
events in the cases where we don't have the main thread around.
- The second part of the patch is the removal of the assumptions that
the main thread is always available. This generally meant replacing
the uses of GetThreadByID(process_id) with GetCurrentThread() in
various process-wide operations (such as memory reads).
Differential Revision: https://reviews.llvm.org/D122716
- Rename IntelPTManager class and files to IntelPTCollector
- Change GetTimestampCounter API to general trace counter API,
GetCounter
Differential Revision: https://reviews.llvm.org/D121711
Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
infrastructure). This worked because Log.h included Logging.h, even
though it should.
After the recent refactor, it became impossible the two files include
each other in this direction (the opposite inclusion is needed), so this
patch removes the workaround that was put in place and cleans up all
files to include the right thing. It also renames the file to LLDBLog to
better reflect its purpose.
This better describes the intent of the method. Which for AArch64
is removing the top byte which includes the memory tags.
It does not include pointer signatures, for those we need to use
the ABI plugin. The rename makes this a little more clear.
It's a bit awkward that the memory tag manager is removing the whole
top byte not just the memory tags but it's an improvement for now.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D117671
Implement the qXfer:siginfo:read that is used to read the siginfo_t
(extended signal information) for the current thread. This is currently
implemented on FreeBSD and Linux.
Differential Revision: https://reviews.llvm.org/D117113
D116372, while fixing one kind of a race, ended up creating a new one.
The new issue could occur when one inferior thread exits while another
thread initiates termination of the entire process (exit_group(2)).
With some bad luck, we could start processing the exit notification
(PTRACE_EVENT_EXIT) only to have the become unresponsive (ESRCH) in the
middle of the MonitorCallback function. This function would then delete
the thread from our list even though it wasn't completely dead (it stays
zombified until we read the WIFEXITED event). The linux kernel will not
deliver the exited event for the entire process until we process
individual thread exits.
In a pre-D116372 world, this wouldn't be a problem because we would read
this event (even though we would not know what to do with it) with
waitpid(-1). Now, when we issue invididual waitpids, this event will
never be picked up, and we end up hanging.
The fix for this is actually quite simple -- don't delete the thread in
this situation. The thread will be deleted when the WIFEXITED event
comes.
This situation was kind of already tested by
TestCreateDuringInstructionStep (which is how I found this problem), but
it was mostly accidental, so I am also creating a dedicated test which
reproduces this situation.
The lldb-server code is currently set up in a way that each
NativeProcess instance does its own waitpid handling. This works fine
for BSDs, where the code can do a waitpid(process_id), and get
information for all threads in that process.
The situation is trickier on linux, because waitpid(pid) will only
return information for the main thread of the process (one whose tid ==
pid). For this reason the linux code does a waitpid(-1), to get
information for all threads. This was fine while we were supporting just
a single process, but becomes a problem when we have multiple processes
as they end up stealing each others events.
There are two possible solutions to this problem:
- call waitpid(-1) centrally, and then dispatch the events to the
appropriate process
- have each process call waitpid(tid) for all the threads it manages
This patch implements the second approach. Besides fitting better into
the existing design, it also has the added benefit of ensuring
predictable ordering for thread/process creation events (which come in
pairs -- one for the parent and one for the child). The first approach
OTOH, would make this ordering even more complicated since we would
have to keep the half-threads hanging in mid-air until we find the
process we should attach them to.
The downside to this approach is an increased number of syscalls (one
waitpid for each thread), but I think we're pretty far from optimizing
things like this, and so the cleanliness of the design is worth it.
The included test reproduces the circumstances which should demonstrate
the bug (which manifests as a hung test), but I have not been able to
get it to fail. The only place I've seen this failure modes are very
rare hangs in the thread sanitizer tests (tsan forks an addr2line
process to produce its error messages).
Differential Revision: https://reviews.llvm.org/D116372
The MonitorCallback function was assuming that the "exited" argument is
set whenever a thread exits, but the caller was only setting that flag
for the main thread.
This patch deletes the argument altogether, and lets MonitorCallback
compute what it needs itself.
This is almost NFC, since previously we would end up in the
"GetSignalInfo failed for unknown reasons" branch, which was doing the
same thing -- forgetting about the thread.
.. and reduce the scope of others. They don't follow llvm coding
standards (which say they should be used only when the same effect
cannot be achieved with the static keyword), and they set a bad example.
This is implemented using the QMemTags packet, as specified
by GDB in:
https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#General-Query-Packets
(recall that qMemTags was previously added to read tags)
On receipt of a valid packet lldb-server will:
* align the given address and length to granules
(most of the time lldb will have already done this
but the specification doesn't guarantee it)
* Repeat the supplied tags as many times as needed to cover
the range. (if tags > range we just use as many as needed)
* Call ptrace POKEMTETAGS to write the tags.
The ptrace step will loop just like the tag read does,
until all tags are written or we get an error.
Meaning that if ptrace succeeds it could be a partial write.
So we call it again and if we then get an error, return an error to
lldb.
We are not going to attempt to restore tags after a partial
write followed by an error. This matches the behaviour of the
existing memory writes.
The lldb-server tests have been extended to include read and
write in the same test file. With some updated function names
since "qMemTags" vs "QMemTags" isn't very clear when they're
next to each other.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D105180
