This patch is a reworking of Pete Lawrence's (@PortalPete) proposal
for better expression evaluator error messages:
https://github.com/llvm/llvm-project/pull/80938
Before:
```
$ lldb -o "expr a+b"
(lldb) expr a+b
error: <user expression 0>:1:1: use of undeclared identifier 'a'
a+b
^
error: <user expression 0>:1:3: use of undeclared identifier 'b'
a+b
^
```
After:
```
(lldb) expr a+b
^ ^
│ ╰─ error: use of undeclared identifier 'b'
╰─ error: use of undeclared identifier 'a'
```
This eliminates the confusing `<user expression 0>:1:3` source
location and avoids echoing the expression to the console again, which
results in a cleaner presentation that makes it easier to grasp what's
going on. You can't see it here, bug the word "error" is now also in
color, if so desired.
Depends on https://github.com/llvm/llvm-project/pull/106442.
This patch removes all of the Set.* methods from Status.
This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.
This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()
Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form
` ResultTy DoFoo(Status& error)
`
to
` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?
The interesting changes are in Status.h and Status.cpp, all other
changes are mostly
` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.
Compilers and language runtimes often use helper functions that are
fundamentally uninteresting when debugging anything but the
compiler/runtime itself. This patch introduces a user-extensible
mechanism that allows for these frames to be hidden from backtraces and
automatically skipped over when navigating the stack with `up` and
`down`.
This does not affect the numbering of frames, so `f <N>` will still
provide access to the hidden frames. The `bt` output will also print a
hint that frames have been hidden.
My primary motivation for this feature is to hide thunks in the Swift
programming language, but I'm including an example recognizer for
`std::function::operator()` that I wished for myself many times while
debugging LLDB.
rdar://126629381
Example output. (Yes, my proof-of-concept recognizer could hide even
more frames if we had a method that returned the function name without
the return type or I used something that isn't based off regex, but it's
really only meant as an example).
before:
```
(lldb) thread backtrace --filtered=false
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
* frame #0: 0x0000000100001f04 a.out`foo(x=1, y=1) at main.cpp:4:10
frame #1: 0x0000000100003a00 a.out`decltype(std::declval<int (*&)(int, int)>()(std::declval<int>(), std::declval<int>())) std::__1::__invoke[abi:se200000]<int (*&)(int, int), int, int>(__f=0x000000016fdff280, __args=0x000000016fdff224, __args=0x000000016fdff220) at invoke.h:149:25
frame #2: 0x000000010000399c a.out`int std::__1::__invoke_void_return_wrapper<int, false>::__call[abi:se200000]<int (*&)(int, int), int, int>(__args=0x000000016fdff280, __args=0x000000016fdff224, __args=0x000000016fdff220) at invoke.h:216:12
frame #3: 0x0000000100003968 a.out`std::__1::__function::__alloc_func<int (*)(int, int), std::__1::allocator<int (*)(int, int)>, int (int, int)>::operator()[abi:se200000](this=0x000000016fdff280, __arg=0x000000016fdff224, __arg=0x000000016fdff220) at function.h:171:12
frame #4: 0x00000001000026bc a.out`std::__1::__function::__func<int (*)(int, int), std::__1::allocator<int (*)(int, int)>, int (int, int)>::operator()(this=0x000000016fdff278, __arg=0x000000016fdff224, __arg=0x000000016fdff220) at function.h:313:10
frame #5: 0x0000000100003c38 a.out`std::__1::__function::__value_func<int (int, int)>::operator()[abi:se200000](this=0x000000016fdff278, __args=0x000000016fdff224, __args=0x000000016fdff220) const at function.h:430:12
frame #6: 0x0000000100002038 a.out`std::__1::function<int (int, int)>::operator()(this= Function = foo(int, int) , __arg=1, __arg=1) const at function.h:989:10
frame #7: 0x0000000100001f64 a.out`main(argc=1, argv=0x000000016fdff4f8) at main.cpp:9:10
frame #8: 0x0000000183cdf154 dyld`start + 2476
(lldb)
```
after
```
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
* frame #0: 0x0000000100001f04 a.out`foo(x=1, y=1) at main.cpp:4:10
frame #1: 0x0000000100003a00 a.out`decltype(std::declval<int (*&)(int, int)>()(std::declval<int>(), std::declval<int>())) std::__1::__invoke[abi:se200000]<int (*&)(int, int), int, int>(__f=0x000000016fdff280, __args=0x000000016fdff224, __args=0x000000016fdff220) at invoke.h:149:25
frame #2: 0x000000010000399c a.out`int std::__1::__invoke_void_return_wrapper<int, false>::__call[abi:se200000]<int (*&)(int, int), int, int>(__args=0x000000016fdff280, __args=0x000000016fdff224, __args=0x000000016fdff220) at invoke.h:216:12
frame #6: 0x0000000100002038 a.out`std::__1::function<int (int, int)>::operator()(this= Function = foo(int, int) , __arg=1, __arg=1) const at function.h:989:10
frame #7: 0x0000000100001f64 a.out`main(argc=1, argv=0x000000016fdff4f8) at main.cpp:9:10
frame #8: 0x0000000183cdf154 dyld`start + 2476
Note: Some frames were hidden by frame recognizers
```
This patch is a follow-up to #97263 that fix ambigous abbreviated
command resolution.
When multiple commands are resolved, instead of failing to pick a
command to
run, this patch changes to resolution logic to check if there is a
single
alias match and if so, it will run the alias instead of the other
matches.
This has as a side-effect that we don't need to make aliases for every
substring of aliases to support abbrivated alias resolution.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This PR introduces a new `ThreadPlanSingleThreadTimeout` that will be
used to address potential deadlock during single-thread stepping.
While debugging a target with a non-trivial number of threads (around
5000 threads in one example target), we noticed that a simple step over
can take as long as 10 seconds. Enabling single-thread stepping mode
significantly reduces the stepping time to around 3 seconds. However,
this can introduce deadlock if we try to step over a method that depends
on other threads to release a lock.
To address this issue, we introduce a new
`ThreadPlanSingleThreadTimeout` that can be controlled by the
`target.process.thread.single-thread-plan-timeout` setting during
single-thread stepping mode. The concept involves counting the elapsed
time since the last internal stop to detect overall stepping progress.
Once a timeout occurs, we assume the target is not making progress due
to a potential deadlock, as mentioned above. We then send a new async
interrupt, resume all threads, and `ThreadPlanSingleThreadTimeout`
completes its task.
To support this design, the major changes made in this PR are:
1. `ThreadPlanSingleThreadTimeout` is popped during every internal stop
and reset (re-pushed) to the top of the stack (as a leaf node) during
resume. This is achieved by always returning `true` from
`ThreadPlanSingleThreadTimeout::DoPlanExplainsStop()` and
`ThreadPlanSingleThreadTimeout::MischiefManaged()`.
2. A new thread-specific async interrupt stop is introduced, which can
be detected/consumed by `ThreadPlanSingleThreadTimeout`.
3. The clearing of branch breakpoints in the range thread plan has been
moved from `DoPlanExplainsStop()` to `ShouldStop()`, as it is not
guaranteed that it will be called.
The detailed design is discussed in the RFC below:
[https://discourse.llvm.org/t/improve-single-thread-stepping/74599](https://discourse.llvm.org/t/improve-single-thread-stepping/74599)
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
This patch introduces a new top-level `scripting` command with an `run`
sub-command, that basically replaces the `script` raw command.
To avoid breaking the `script` command usages, this patch also adds an
`script` alias to the `scripting run` sub-command.
The reason behind this change is to have a top-level command that will
cover scripting related subcommands.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This is useful if you have a transcript of a user session and want to
rerun those commands with RunCommandInterpreter. The same functionality
is also useful in testing.
I'm adding it primarily for the second reason. In a subsequent patch,
I'm adding the ability to Python based commands to provide their
"auto-repeat" command. Among other things, that will allow potentially
state destroying user commands to prevent auto-repeat. Testing this with
Shell or pexpect tests is not nearly as accurate or convenient as using
RunCommandInterpreter, but to use that I need to allow auto-repeat.
I think for consistency's sake, having interactive sessions always do
auto-repeats is the right choice, though that's a lightly held
opinion...
# Changes
1. Changes to the structured transcript.
1. Add fields `commandName` and `commandArguments`. They will hold the
name and the arguments string of the expanded/executed command (e.g.
`breakpoint set` and `-f main.cpp -l 4`). This is not to be confused
with the `command` field, which holds the user input (e.g. `br s -f
main.cpp -l 4`).
2. Add field `timestampInEpochSeconds`. It will hold the timestamp when
the command is executed.
3. Rename field `seconds` to `durationInSeconds`, to improve
readability, especially since `timestampInEpochSeconds` is added.
2. When transcript is available and the newly added option
`--transcript` is present, add the transcript to the output of
`statistics dump`, as a JSON array under a new field `transcript`.
3. A few test name and comment changes.
1. Use dashes (-) instead of colons (:) as time separator in a session log
file name since Windows doesn't support saving files with names containing
colons.
2. Temporary file creation code is changed in the test:
On Windows, the temporary file should be closed before 'session save'
writes session log to it. NamedTemporaryFile() can preserve the file
after closing it with delete_on_close=False option.
However, this option is only available since Python 3.12. Thus
mkstemp() is used for temporary file creation as the more compatible
option.
We noticed that `apropos backtrace` did not return the `bt` alias. This change adds the
word "backtrace" to the help for `bt`. It also updates `thread backtrace` to keep the
language used roughly in sync.
# Motivation
Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::SaveTranscript`). However, the
only way to obtain the transcript data as a user is to first destroy the
debugger, then read the save directory. Note that destroy-callbacks
cannot be used, because 1\ transcript data is private to the command
interpreter (see `CommandInterpreter.h`), and 2\ the writing of the
transcript is *after* the invocation of destory-callbacks (see
`Debugger::Destroy`).
So basically, there is no way to obtain the transcript:
* during the lifetime of a debugger (including the destroy-callbacks,
which often performs logging tasks, where the transcript can be useful)
* without relying on external storage
In theory, there are other ways for user to obtain transcript data
during a debugger's life cycle:
* Use Python API and intercept commands and results.
* Use CLI and record console input/output.
However, such ways rely on the client's setup and are not supported
natively by LLDB.
# Proposal
Add a new Python API `SBCommandInterpreter::GetTranscript()`.
Goals:
* It can be called at any time during the debugger's life cycle,
including in destroy-callbacks.
* It returns data in-memory.
Structured data:
* To make data processing easier, the return type is `SBStructuredData`.
See comments in code for how the data is organized.
* In the future, `SaveTranscript` can be updated to write different
formats using such data (e.g. JSON). This is probably accompanied by a
new setting (e.g. `interpreter.save-session-format`).
# Alternatives
The return type can also be `std::vector<std::pair<std::string,
SBCommandReturnObject>>`. This will make implementation easier, without
having to translate it to `SBStructuredData`. On the other hand,
`SBStructuredData` can convert to JSON easily, so it's more convenient
for user to process.
# Privacy
Both user commands and output/error in the transcript can contain
privacy data. However, as mentioned, the transcript is already available
to the user. The addition of the new API doesn't increase the level of
risk. In fact, it _lowers_ the risk of privacy data being leaked later
on, by avoiding writing such data to external storage.
Once the user (or their code) gets the transcript, it will be their
responsibility to make sure that any required privacy policies are
guaranteed.
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/interpreter/TestCommandInterpreterAPI.py
```
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/session/save/TestSessionSave.py
```
---------
Co-authored-by: Roy Shi <royshi@meta.com>
Co-authored-by: Med Ismail Bennani <ismail@bennani.ma>
These are hardcoded strings that are already present in the data section
of the binary, no need to immediately place them in the ConstString
StringPools. Lots of code still calls `GetBroadcasterClass` and places
the return value into a ConstString. Changing that would be a good
follow-up.
Additionally, calls to these functions are still wrapped in ConstStrings
at the SBAPI layer. This is because we must guarantee the lifetime of
all strings handed out publicly.
Adding command interpreter statistics into "statistics dump" command so
that we can track the command usage frequency for telemetry purpose.
This is useful to answer questions like what is the most frequently used
lldb commands across all our users.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
If adding a user commands fails because a command with the same name
already exists, we only say that "force replace is not set" without
telling the user _how_ to set it. There are two ways to do so; this
commit changes the error message to mention both.
The problem is that the when the "attach" command is initiated, the
ExecutionContext for the command has a process - it's the exited one
from the previour run. But the `attach wait` creates a new process for
the attach, and then errors out instead of interrupting when it finds
that its process and the one in the command's ExecutionContext don't
match.
This change checks that if we're returning a target from
GetExecutionContext, we fill the context with it's current process, not
some historical one.
Previously we would check all built-ins first for suggestions,
then check built-ins and aliases. This meant that if you had
an alias brkpt -> breakpoint, "br" would complete to "breakpoint".
Instead of giving you the choice of "brkpt" or "breakpoint".
ConstString can be implicitly converted into a llvm::StringRef. This is
very useful in many places, but it also hides places where we are
creating a ConstString only to use it as a StringRef for the entire
lifespan of the ConstString object.
I locally removed the implicit conversion and found some of the places we
were doing this.
Differential Revision: https://reviews.llvm.org/D159237
StreamFile subclasses Stream (from lldbUtility) and is backed by a File
(from lldbHost). It does not depend on anything from lldbCore or any of its
sibling libraries, so I think it makes sense for this to live in
lldbHost instead.
Differential Revision: https://reviews.llvm.org/D157460
Also, make it possible for new Targets which haven't been added to
the TargetList yet to check for interruption, and add a few more
places in building modules where we can check for interruption.
Differential Revision: https://reviews.llvm.org/D154542
This patch should allow the user to set specific auto-completion type
for their custom commands.
To do so, we had to hoist the `CompletionType` enum so the user can
access it and add a new completion type flag to the CommandScriptAdd
Command Object.
So now, the user can specify which completion type will be used with
their custom command, when they register it.
This also makes the `crashlog` custom commands use disk-file completion
type, to browse through the user file system and load the report.
Differential Revision: https://reviews.llvm.org/D152011
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Use templates to simplify {Get,Set}PropertyAtIndex. It has always
bothered me how cumbersome those calls are when adding new properties.
After this patch, SetPropertyAtIndex infers the type from its arguments
and GetPropertyAtIndex required a single template argument for the
return value. As an added benefit, this enables us to remove a bunch of
wrappers from UserSettingsController and OptionValueProperties.
Differential revision: https://reviews.llvm.org/D149774
The majority of call sites are nullptr as the execution context.
Refactor OptionValueProperties to make the argument optional and
simplify all the callers.
Similar to fdbe7c7faa54, refactor OptionValueProperties to return a
std::optional instead of taking a fail value. This allows the caller to
handle situations where there's no value, instead of being unable to
distinguish between the absence of a value and the value happening the
match the fail value. When a fail value is required,
std::optional::value_or() provides the same functionality.
Add a new setting (debugger.external-editor) to specify an external
editor. The setting takes precedence over the existing
LLDB_EXTERNAL_EDITOR environment variable.
Differential revision: https://reviews.llvm.org/D149565
This patch refactors the macOS implementation of
OpenFileInExternalEditor. It fixes an AppleEvent memory leak, the
caching of LLDB_EXTERNAL_EDITOR and speculatively fixes a crash when
CFURL is NULL (rdar://108633464). The new code also improves error
handling, readability and documents calls to the CoreFoundation Launch
Services APIs.
A bunch of the Launch Services APIs have been deprecated
(LSFindApplicationForInfo, LSOpenURLsWithRole). The preferred API is
LSOpenCFURLRef but it doesn't specifying the "location" Apple Event
which is used to highlight the current line and switching over would
regress the existing behavior.
rdar://108633464
Differential revision: https://reviews.llvm.org/D149482
Some LLDB set ups need to hide certain commands for security reasons, so I'm adding a flag that allows removing non-user commands.
Differential Revision: https://reviews.llvm.org/D149312
Fix logic for repeat commands, so that regex commands (specificially `bt`) are
given the opportunity to provide a repeat command.
rdar://104562616
Differential Revision: https://reviews.llvm.org/D143695
Redefine the `p` alias to the `dwim-print` command instead of `expression`.
See https://reviews.llvm.org/D138315 for the introduction of `dwim-print`.
To summarize, `dwim-print` is, as the name suggests, a command for printing. How a value
gets printed, is decided by `dwim-print`. In some cases, `dwim-print` will print values
using the same means as `frame variable` (because it's generally more reliable and
faster that `expression` evaluation), and in other cases `dwim-print` uses the same code
path as `expression`.
This change has been tested in two different ways:
1. Re-aliasing `p` to `dwim-print`, as in this patch
2. Redefinining the `expression` command to `CommandObjectDWIMPrint`
Previously, many of the lldb's tests used `p`, and which meant a test run with `p`
aliases to `dwim-print` was a good way to test `dwim-print`. However most of those tests
were updated to use `expression` explicitly (in anticipation of this change). Now, the
best way to test `dwim-print` is the second approach:
```
diff --git a/lldb/source/Interpreter/CommandInterpreter.cpp b/lldb/source/Interpreter/CommandInterpreter.cpp
index 373c894f34f5..9c943cd30c7c 100644
--- a/lldb/source/Interpreter/CommandInterpreter.cpp
+++ b/lldb/source/Interpreter/CommandInterpreter.cpp
@@ -539,7 +539,7 @@ void CommandInterpreter::LoadCommandDictionary() {
REGISTER_COMMAND_OBJECT("diagnostics", CommandObjectDiagnostics);
REGISTER_COMMAND_OBJECT("disassemble", CommandObjectDisassemble);
REGISTER_COMMAND_OBJECT("dwim-print", CommandObjectDWIMPrint);
- REGISTER_COMMAND_OBJECT("expression", CommandObjectExpression);
+ REGISTER_COMMAND_OBJECT("expression", CommandObjectDWIMPrint);
REGISTER_COMMAND_OBJECT("frame", CommandObjectMultiwordFrame);
REGISTER_COMMAND_OBJECT("gui", CommandObjectGUI);
REGISTER_COMMAND_OBJECT("help", CommandObjectHelp);
```
When the test suite is run with this change, there are two main categories of test
failures for specific to features that `dwim-print` intentionally doesn't support:
1. Top level expressions (`--top-level`/`-p`)
2. Multiline expressions
In cases where the behavior of `expression` is needed, users can use `expression` at
those times.
Differential Revision: https://reviews.llvm.org/D145189
Redefine the `p` alias to the `dwim-print` command instead of `expression`.
See https://reviews.llvm.org/D138315 for the introduction of `dwim-print`.
To summarize, `dwim-print` is, as the name suggests, a command for printing. How a value
gets printed, is decided by `dwim-print`. In some cases, `dwim-print` will print values
using the same means as `frame variable` (because it's generally more reliable and
faster that `expression` evaluation), and in other cases `dwim-print` uses the same code
path as `expression`.
This change has been tested in two different ways:
1. Re-aliasing `p` to `dwim-print`, as in this patch
2. Redefinining the `expression` command to `CommandObjectDWIMPrint`
Previously, many of the lldb's tests used `p`, and which meant a test run with `p`
aliases to `dwim-print` was a good way to test `dwim-print`. However most of those tests
were updated to use `expression` explicitly (in anticipation of this change). Now, the
best way to test `dwim-print` is the second approach:
```
diff --git a/lldb/source/Interpreter/CommandInterpreter.cpp b/lldb/source/Interpreter/CommandInterpreter.cpp
index 373c894f34f5..9c943cd30c7c 100644
--- a/lldb/source/Interpreter/CommandInterpreter.cpp
+++ b/lldb/source/Interpreter/CommandInterpreter.cpp
@@ -539,7 +539,7 @@ void CommandInterpreter::LoadCommandDictionary() {
REGISTER_COMMAND_OBJECT("diagnostics", CommandObjectDiagnostics);
REGISTER_COMMAND_OBJECT("disassemble", CommandObjectDisassemble);
REGISTER_COMMAND_OBJECT("dwim-print", CommandObjectDWIMPrint);
- REGISTER_COMMAND_OBJECT("expression", CommandObjectExpression);
+ REGISTER_COMMAND_OBJECT("expression", CommandObjectDWIMPrint);
REGISTER_COMMAND_OBJECT("frame", CommandObjectMultiwordFrame);
REGISTER_COMMAND_OBJECT("gui", CommandObjectGUI);
REGISTER_COMMAND_OBJECT("help", CommandObjectHelp);
```
When the test suite is run with this change, there are two main categories of test
failures for specific to features that `dwim-print` intentionally doesn't support:
1. Top level expressions (`--top-level`/`-p`)
2. Multiline expressions
In cases where the behavior of `expression` is needed, users can use `expression` at
those times.
Differential Revision: https://reviews.llvm.org/D145189
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Implements `dwim-print`, a printing command that chooses the most direct,
efficient, and resilient means of printing a given expression.
DWIM is an acronym for Do What I Mean. From Wikipedia, DWIM is described as:
> attempt to anticipate what users intend to do, correcting trivial errors
> automatically rather than blindly executing users' explicit but
> potentially incorrect input
The `dwim-print` command serves as a single print command for users who don't
yet know, or prefer not to know, the various lldb commands that can be used to
print, and when to use them.
This initial implementation is the base foundation for `dwim-print`. It accepts
no flags, only an expression. If the expression is the name of a variable in
the frame, then effectively `frame variable` is used to get, and print, its
value. Otherwise, printing falls back to using `expression` evaluation. In this
initial version, frame variable paths will be handled with `expression`.
Following this, there are a number of improvements that can be made. Some
improvements include supporting `frame variable` expressions or registers.
To provide transparency, especially as the `dwim-print` command evolves, a new
setting is also introduced: `dwim-print-verbosity`. This setting instructs
`dwim-print` to optionally print a message showing the effective command being
run. For example `dwim-print var.meth()` can print a message such as: "note:
ran `expression var.meth()`".
See https://discourse.llvm.org/t/dwim-print-command/66078 for the proposal and
discussion.
Differential Revision: https://reviews.llvm.org/D138315
