This implements coalescing of progress events using a timeout, as
discussed in the RFC on Discourse [1]. This PR consists of two commits
which, depending on the feedback, I may split up into two PRs. For now,
I think it's easier to review this as a whole.
1. The first commit introduces a new generic `Alarm` class. The class
lets you to schedule a function (callback) to be executed after a given
timeout expires. You can cancel and reset a callback before its
corresponding timeout expires. It achieves this with the help of a
worker thread that sleeps until the next timeout expires. The only
guarantee it provides is that your function is called no sooner than the
requested timeout. Because the callback is called directly from the
worker thread, a long running callback could potentially block the
worker thread. I intentionally kept the implementation as simple as
possible while addressing the needs for the `ProgressManager` use case.
If we want to rely on this somewhere else, we can reassess whether we
need to address those limitations.
2. The second commit uses the Alarm class to coalesce progress events.
To recap the Discourse discussion, when multiple progress events with
the same title execute in close succession, they get broadcast as one to
`eBroadcastBitProgressCategory`. The `ProgressManager` keeps track of
the in-flight progress events and when the refcount hits zero, the Alarm
class is used to schedule broadcasting the event. If a new progress
event comes in before the alarm fires, the alarm is reset (and the
process repeats when the new progress event ends). If no new event comes
in before the timeout expires, the progress event is broadcast.
[1]
https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717/
This implements coalescing of progress events using a timeout, as
discussed in the RFC on Discourse [1]. This PR consists of two commits
which, depending on the feedback, I may split up into two PRs. For now,
I think it's easier to review this as a whole.
1. The first commit introduces a new generic `Alarm` class. The class
lets you to schedule a function (callback) to be executed after a given
timeout expires. You can cancel and reset a callback before its
corresponding timeout expires. It achieves this with the help of a
worker thread that sleeps until the next timeout expires. The only
guarantee it provides is that your function is called no sooner than the
requested timeout. Because the callback is called directly from the
worker thread, a long running callback could potentially block the
worker thread. I intentionally kept the implementation as simple as
possible while addressing the needs for the `ProgressManager` use case.
If we want to rely on this somewhere else, we can reassess whether we
need to address those limitations.
2. The second commit uses the Alarm class to coalesce progress events.
To recap the Discourse discussion, when multiple progress events with
the same title execute in close succession, they get broadcast as one to
`eBroadcastBitProgressCategory`. The `ProgressManager` keeps track of
the in-flight progress events and when the refcount hits zero, the Alarm
class is used to schedule broadcasting the event. If a new progress
event comes in before the alarm fires, the alarm is reset (and the
process repeats when the new progress event ends). If no new event comes
in before the timeout expires, the progress event is broadcast.
[1]
https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717/
- Factor our common setup code.
- Split the ProgressManager test into separate tests as they test
separate things.
- Fix usage of EXPECT (which continues on failure) and ASSERT (which
halts on failure). We must use the latter when calling GetEvent as
otherwise we'll try to dereference a null EventSP.
Currently, progress events reported by the ProgressManager and broadcast
to eBroadcastBitProgressCategory always specify they're complete. The
problem is that the ProgressManager reports kNonDeterministicTotal for
both the total and the completed number of (sub)events. Because the
values are the same, the event reports itself as complete.
This patch fixes the issue by reporting 0 as the completed value for the
start event and kNonDeterministicTotal for the end event.
This commit adds the functionality to broadcast events using the
`Debugger::eBroadcastProgressCategory`
bit (https://github.com/llvm/llvm-project/pull/81169) by keeping track
of these reports with the `ProgressManager`
class (https://github.com/llvm/llvm-project/pull/81319). The new bit is
used in such a way that it will only broadcast the initial and final
progress reports for specific progress categories that are managed by
the progress manager.
This commit also adds a new test to the progress report unit test that
checks that only the initial/final reports are broadcasted when using
the new bit.
This file was previously approved and merged from this PR:
https://github.com/llvm/llvm-project/pull/79533 but caused a test
failure on the Linux AArch64 bots due to hitting an assertion that
`Debugger::Initialize` was already called.
To fix this, this commit uses the
changes made here: https://github.com/llvm/llvm-project/pull/80786 to
use a shared call_once flag to initialize the debugger.
Incorporates the changes from
https://github.com/llvm/llvm-project/pull/80786 to use a once_flag from
`TestUtilities` instead of a local flag in order to prevent hitting an
assertion that the debugger was initialized again in another test.
This reverts commit a5a8cbb110384825b99228891576f799082f200e.
The test being added by that commit still fails on the assertion that
Debugger::Initialize has been called more than once.
This reverts commit 209fe1f3d70d1c4a20bb2687e0d0a94b1bbfa0c6.
The original commit failed to due an assertion failure in the unit test
`ProgressReportTest` that the commit added. The Debugger::Initialize()
function was called more than once which triggered the assertion, so
this commit calls that function under a `std::call_once`.
This reverts commit 51e0d1b707a15dd3b96e0c4f1a29ca4dfc79cda0.
That commit breaks a unit test:
```
Failed Tests (1):
lldb-unit :: Core/./LLDBCoreTests/4/8
```
This test is being added as a way to check the behaviour of how progress
events are broadcasted when reports are started and ended with the
current implementation of progress reports. Here we're mainly checking
and ensuring that the current behaviour is that progress events are
broadcasted individually and placed in the event queue in order of their
creation and deletion.
LLVM supports DWARF 5 linetable extension to store source files inline
in DWARF. This is particularly useful for compiler-generated source
code. This implementation tries to materialize them as temporary files
lazily, so SBAPI clients don't need to be aware of them.
rdar://110926168
In 014c41d688f961cdb81cf92e6ba97100a49766a6 I tried to fix these tests,
but it seems that I needed to change TEST for TEST_F to make that work.
It's a pain that these failures don't repro on any of my machines, but I
verified thta the initialization code for the tests is invoked.
This test was broken by 710276a2505514634a7cc805461b1219dcef9337 because
DumpDataExtractor now accesses the Target properties, which someone ends
up relying on the file system.
This is an instance of this error https://lab.llvm.org/buildbot/#/builders/96/builds/45607/steps/6/logs/stdio
I cannot reproduce this locally, but it seems that the error happens
because we are not initializing the FileSystem and the Host as part of
the test setup.
Apple maintains a downstream fork of lldb in order to support swift
debugging. Much of that support is isolated to its own plugins, but some
of it is exposed in more generic code. I would like to take some of
the swift support we have downstream and move it upstream to llvm.org in
an effort to 1) reduce downstream maintenance burden, and 2) work
towards solidifying the process of adding new language support to lldb.
Differential Revision: https://reviews.llvm.org/D158470
We recently saw an uptick in internal reports complaining that LLDB is
slow when sources on network file systems are inaccessible. I looked at
the SourceManger and its cache and I think there’s some room for
improvement in terms of reducing file system accesses:
1. We always resolve the path.
2. We always check the timestamp.
3. We always recheck the file system for negative cache hits.
D153726 fixes (1) but (2) and (3) are necessary because of the cache’s
current design. Source files are cached at the debugger level which
means that the source file cache can span multiple targets and
processes. It wouldn't be correct to not reload a modified or new file
from disk.
We can however significantly reduce the number of file system accesses
by using a two level cache design: one cache at the debugger level and
one at the process level:
- The cache at the debugger level works the way it does today. There is
no negative cache: if we can't find the file on disk, we'll try again
next time the cache is queried. If a cached file's timestamp changes
or if its path remapping changes, the cached file is evicted and we
reload it from disk.
- The cache at the process level is design to avoid accessing the file
system. It doesn't check the file's modification time. It caches
negative results, so if a file didn't exist, it doesn't try to reread
it from disk. Checking if the path remapping changed is cheap
(doesn't involve checking the file system) and is the only way for a
file to get evicted from the process cache.
The result of this patch is that LLDB will not show you new content if a
file is modified or created while a process is running. I would argue
that this is what most people would expect, but it is a change from how
LLDB behaves today.
For an average stop, we query the source cache 4 times. With the current
implementation, that's 4 stats to get the modification time, If the file
doesn't exist on disk, that's an additional 4 stats. Before D153726, if
the path starts with a ~ there are another additional 4 calls to
realpath. When debugging sources on a slow (network) file system, this
quickly adds up.
In addition to the two level caching, this patch also adds a source
logging channel and synchronization to the source file cache. The
logging was helpful during development and hopefully will help us triage
issues in the future. The synchronization isn't a new requirement: as
the cache is shared across targets, there is no guarantees that it can't
be accessed concurrently. The patch also fixes a bug where we would only
set the source remapping ID if the un-remapped file didn't exist, which
led to the file getting evicted from the cache on every access.
rdar://110787562
Differential revision: https://reviews.llvm.org/D153834
Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have to resolve the
path, which can be slow. This patch avoids the call to FileSystem::Resolve
by caching both the resolved and unresolved path. We now only resolve
the path once when we create and cache a new file.
rdar://110787562
Differential revision: https://reviews.llvm.org/D153726
When specifying the C-string format for dumping memory, we treat
unprintable characters as signed. Whether a character is signed or not
is implementation defined, but all printable characters are signed.
Therefore it's fair to assume that unprintable characters are unsigned.
Before this patch, "\xcf\xfa\xed\xfe\f" would be printed as
"\xffffffcf\xfffffffa\xffffffed\xfffffffe\f". Now we correctly print the
original string.
rdar://111126134
Differential revision: https://reviews.llvm.org/D153644
This teaches DumpRegisterInfo to generate a table from the register
flags type. It just calls a method on RegisterFlags.
As such, the extra tests are minimal and only show that the intergration
works. Exhaustive formatting tests are done with RegisterFlags itself.
Example:
```
(lldb) register info cpsr
Name: cpsr
Size: 4 bytes (32 bits)
In sets: general (index 0)
| 31 | 30 | 29 | 28 | 27-26 | 25 | 24 | 23 | 22 | 21 | 20 | 19-13 | 12 | 11-10 | 9 | 8 | 7 | 6 | 5 | 4 | 3-2 | 1 | 0 |
|----|----|----|----|-------|-----|-----|-----|-----|----|----|-------|------|-------|---|---|---|---|---|-----|-----|---|----|
| N | Z | C | V | | TCO | DIT | UAO | PAN | SS | IL | | SSBS | | D | A | I | F | | nRW | EL | | SP |
```
LLDB limits the max terminal width to 80 chars by default.
So to get that full width output you will need to change the "term-width"
setting to something higher.
Reviewed By: jasonmolenda
Differential Revision: https://reviews.llvm.org/D152918
This adds a new command that will show all the information lldb
knows about a register.
```
(lldb) register info s0
Name: s0
Size: 4 bytes (32 bits)
Invalidates: v0, d0
Read from: v0
In sets: Floating Point Registers (index 1)
```
Currently it only allows a single register, and we get the
information from the RegisterInfo structure.
For those of us who know the architecture well, this information
is all pretty obvious. For those who don't, it's nice to have it
at a glance without leaving the debugger.
I hope to have more in depth information to show here in the future,
which will be of wider use.
Reviewed By: jasonmolenda
Differential Revision: https://reviews.llvm.org/D152916
This patch is preparatory work for Scripted Platform support and does
multiple things:
First, it introduces new options for the `platform select` command and
`SBPlatform::Create` API, to hold a reference to the debugger object,
the name of the python script managing the Scripted Platform and a
structured data dictionary that the user can use to pass arbitrary data.
Then, it updates the various `Create` and `GetOrCreate` methods for
the `Platform` and `PlatformList` classes to pass down the new parameter
to the `Platform::CreateInstance` callbacks.
Finally, it updates every callback to reflect these changes.
Differential Revision: https://reviews.llvm.org/D139249
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Split the read thread support from Communication into a dedicated
ThreadedCommunication subclass. The read thread support is used only
by a subset of Communication consumers, and it adds a lot of complexity
to the base class. Furthermore, having a dedicated subclass makes it
clear whether a particular consumer needs to account for the possibility
of read thread being running or not.
The modules currently calling `StartReadThread()` are updated to use
`ThreadedCommunication`. The remaining modules use the simplified
`Communication` class.
`SBCommunication` is changed to use `ThreadedCommunication` in order
to avoid changing the public API.
`CommunicationKDP` is updated in order to (hopefully) compile with
the new code. However, I do not have a Darwin box to test it, so I've
limited the changes to the bare minimum.
`GDBRemoteCommunication` is updated to become a `Broadcaster` directly.
Since it does not inherit from `ThreadedCommunication`, its event
support no longer collides with the one used for read thread and can
be implemented cleanly. The support for
`eBroadcastBitReadThreadDidExit` is removed from the code -- since
the read thread was not used, this event was never reported.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.llvm.org/D133251
Ensure that the ConnectionStatus and Status from
Communication::ReadThread() is correctly passed to ::Read() in order
to fix further discrepancies between ::Read() calls in non-threaded
and threaded modes.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.llvm.org/D132577
Our (TCP) socket support is in a much better state than pipes. Use that
for testing the Communication class.
Move the CreateTCPConnectedSockets function
(SocketTestUtilities.{h,cpp}) to a place where it can be used from
Communication tests.
Harmonize the status and error values of Communication::Read() when
running with and without read thread. Prior to this change, Read()
would return eConnectionStatusSuccess if read thread was enabled
and the read timed out or reached end-of-file, rather than
the respective states that are returned if read thread was disabled.
Now, it correctly returns eConnectionStatusTimedOut
and eConnectionStatusEndOfFile, and sets the error respectively.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.llvm.org/D132217
Currently a FileSpecList::FindFileIndex(...) will only match the specified FileSpec if:
- it has filename and directory and both match exactly
- if has a filename only and any filename in the list matches
Because of this, we modify our breakpoint resolving so it can handle relative paths by doing some extra code that removes the directory from the FileSpec when searching if the path is relative.
This patch is intended to fix breakpoints so they work as users expect them to by adding the following features:
- allow matches to relative paths in the file list to match as long as the relative path is at the end of the specified path at valid directory delimiters
- allow matches to paths to match if the specified path is relative and shorter than the file paths in the list
This allows us to remove the extra logic from BreakpointResolverFileLine.cpp that added support for setting breakpoints with relative paths.
This means we can still set breakpoints with relative paths when the debug info contains full paths. We add the ability to set breakpoints with full paths when the debug info contains relative paths.
Debug info contains "./a/b/c/main.cpp", the following will set breakpoints successfully:
- /build/a/b/c/main.cpp
- a/b/c/main.cpp
- b/c/main.cpp
- c/main.cpp
- main.cpp
- ./c/main.cpp
- ./a/b/c/main.cpp
- ./b/c/main.cpp
- ./main.cpp
This also ensures that we won't match partial directory names, if a relative path is in the list or is used for the match, things must match at the directory level.
The breakpoint resolving code will now use the new FileSpecList::FindCompatibleIndex(...) function to allow this fuzzy matching to work for breakpoints.
Differential Revision: https://reviews.llvm.org/D130401
The DumpDataExtractor function had two branches for printing floating
point values. One branch (APFloat) was used if we had a Target object
around and could query it for the appropriate semantics. If we didn't
have a Target, we used host operations to read and format the value.
This patch changes second path to use APFloat as well. To make it work,
I pick reasonable defaults for different byte size. Notably, I did not
include x87 long double in that list (as it is ambibuous and
architecture-specific). This exposed a bug where we were printing
register values using the target-less branch, even though the registers
definitely belong to a target, and we had it available. Fixing this
prompted the update of several tests for register values due to slightly
different floating point outputs.
The most dubious aspect of this patch is the change in
TypeSystemClang::GetFloatTypeSemantics to recognize `10` as a valid size
for x87 long double. This was necessary because because sizeof(long
double) on x86_64 is 16 even though it only holds 10 bytes of useful
data. This generalizes the hackaround present in the target-free branch
of the dumping function.
Differential Revision: https://reviews.llvm.org/D129750
This patch introduces the concept of a log handlers. Log handlers allow
customizing the way log output is emitted. The StreamCallback class
tried to do something conceptually similar. The benefit of the log
handler interface is that you don't need to conform to llvm's
raw_ostream interface.
Differential revision: https://reviews.llvm.org/D127922
Report warnings and errors through events instead of printing directly
the to the debugger's error stream. By using events, IDEs such as Xcode
can report these issues in the UI instead of having them show up in the
debugger console.
The new diagnostic events are handled by the default event loop. If a
diagnostic is reported while nobody is listening for the new event
types, it is printed directly to the debugger's error stream.
Differential revision: https://reviews.llvm.org/D121511
Have the different ::Parse.* methods return the demangled string
directly instead of having to go through ::GetBufferRef.
Differential revision: https://reviews.llvm.org/D118953
Currently, running the test suite with LLVM_ENABLE_EXPENSIVE_CHECKS=On
causes a couple of tests to fail. This happens because they expect a
certain order of variables (all of them happen to use the "target
variable" command, but other lookup functions should suffer from the
same issues), all of which have the same name. Sort algorithms often
preserve the order of equivalent elements (in this case the entries in
the NameToDIE map), but that not guaranteed, and
LLVM_ENABLE_EXPENSIVE_CHECKS stresses that by pre-shuffling all inputs
before sorting.
While this could easily be fixed by relaxing the test expectations,
having a deterministic output seems like a worthwhile goal,
particularly, as this could have bigger consequences than just a
different print order -- in some cases we just pick the first entry that
we find, whatever that is. Therefore this patch makes the sort
deterministic by introducing another sort key -- UniqueCString::Sort
gets a value comparator functor, which can be used to sort elements with
the same name -- in the DWARF case we use DIERef::operator<, which
roughly equals the order in which the entries appear in the debug info,
and matches the current "accidental" order.
Using a extra functor seemed preferable to using stable_sort, as the
latter allocates extra O(n) of temporary memory.
I observed no difference in debug info parsing speed with this patch
applied.
Differential Revision: https://reviews.llvm.org/D118251
Remove the Mangled::operator! and Mangled::operator void* where the
comments in header and implementation files disagree and replace them
with operator bool.
This fix PR52702 as https://reviews.llvm.org/D106837 used the buggy
Mangled::operator! in Symbol::SynthesizeNameIfNeeded. For example,
consider the symbol "puts" in a hello world C program:
// Inside Symbol::SynthesizeNameIfNeeded
(lldb) p m_mangled
(lldb_private::Mangled) $0 = (m_mangled = None, m_demangled = "puts")
(lldb) p !m_mangled
(bool) $1 = true # should be false!!
This leads to Symbol::SynthesizeNameIfNeeded overwriting m_demangled
part of Mangled (in this case "puts").
In conclusion, this patch turns
callq 0x401030 ; symbol stub for: ___lldb_unnamed_symbol36
back into
callq 0x401030 ; symbol stub for: puts .
Differential Revision: https://reviews.llvm.org/D116217
This is part of https://github.com/dlang/projects/issues/81 .
This patch enables support for D programming language demangler by using a
pretty printed stacktrace with demangled D symbols, when present.
Signed-off-by: Luís Ferreira <contact@lsferreira.net>
Reviewed By: JDevlieghere, teemperor
Differential Revision: https://reviews.llvm.org/D110578
Add a Communication::WriteAll() that resumes Write() if the initial call
did not write all data. Use it in GDBRemoteCommunication when sending
packets in order to fix handling partial writes (i.e. just resume/retry
them rather than erring out). This fixes LLDB failures when writing
large packets to a pty.
Differential Revision: https://reviews.llvm.org/D112169
Rust's v0 name mangling scheme [1] is easy to disambiguate from other
name mangling schemes because symbols always start with `_R`. The llvm
Demangle library supports demangling the Rust v0 scheme. Use it to
demangle Rust symbols.
Added unit tests that check simple symbols. Ran LLDB built with this
patch to debug some Rust programs compiled with the v0 name mangling
scheme. Confirmed symbol names were demangled as expected.
Note: enabling the new name mangling scheme requires a nightly
toolchain:
```
$ cat main.rs
fn main() {
println!("Hello world!");
}
$ $(rustup which --toolchain nightly rustc) -Z symbol-mangling-version=v0 main.rs -g
$ /home/asm/hacking/llvm/build/bin/lldb ./main --one-line 'b main.rs:2'
(lldb) target create "./main"
Current executable set to '/home/asm/hacking/llvm/rust/main' (x86_64).
(lldb) b main.rs:2
Breakpoint 1: where = main`main::main + 4 at main.rs:2:5, address = 0x00000000000076a4
(lldb) r
Process 948449 launched: '/home/asm/hacking/llvm/rust/main' (x86_64)
warning: (x86_64) /lib64/libgcc_s.so.1 No LZMA support found for reading .gnu_debugdata section
Process 948449 stopped
* thread #1, name = 'main', stop reason = breakpoint 1.1
frame #0: 0x000055555555b6a4 main`main::main at main.rs:2:5
1 fn main() {
-> 2 println!("Hello world!");
3 }
(lldb) bt
error: need to add support for DW_TAG_base_type '()' encoded with DW_ATE = 0x7, bit_size = 0
* thread #1, name = 'main', stop reason = breakpoint 1.1
* frame #0: 0x000055555555b6a4 main`main::main at main.rs:2:5
frame #1: 0x000055555555b78b main`<fn() as core::ops::function::FnOnce<()>>::call_once((null)=(main`main::main at main.rs:1), (null)=<unavailable>) at function.rs:227:5
frame #2: 0x000055555555b66e main`std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>(f=(main`main::main at main.rs:1)) at backtrace.rs:125:18
frame #3: 0x000055555555b851 main`std::rt::lang_start::<()>::{closure#0} at rt.rs:49:18
frame #4: 0x000055555556c9f9 main`std::rt::lang_start_internal::hc51399759a90501a [inlined] core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_once::h04259e4a34d07c2f at function.rs:259:13
frame #5: 0x000055555556c9f2 main`std::rt::lang_start_internal::hc51399759a90501a [inlined] std::panicking::try::do_call::hb8da45704d5cfbbf at panicking.rs:401:40
frame #6: 0x000055555556c9f2 main`std::rt::lang_start_internal::hc51399759a90501a [inlined] std::panicking::try::h4beadc19a78fec52 at panicking.rs:365:19
frame #7: 0x000055555556c9f2 main`std::rt::lang_start_internal::hc51399759a90501a [inlined] std::panic::catch_unwind::hc58016cd36ba81a4 at panic.rs:433:14
frame #8: 0x000055555556c9f2 main`std::rt::lang_start_internal::hc51399759a90501a at rt.rs:34:21
frame #9: 0x000055555555b830 main`std::rt::lang_start::<()>(main=(main`main::main at main.rs:1), argc=1, argv=0x00007fffffffcb18) at rt.rs:48:5
frame #10: 0x000055555555b6fc main`main + 28
frame #11: 0x00007ffff73f2493 libc.so.6`__libc_start_main + 243
frame #12: 0x000055555555b59e main`_start + 46
(lldb)
```
[1]: https://github.com/rust-lang/rust/issues/60705
Reviewed By: clayborg, teemperor
Differential Revision: https://reviews.llvm.org/D104054
