This changes the example command added in
https://github.com/llvm/llvm-project/pull/145793 so that the fdis
program does not have to be a single program name.
Doing so also means we can run the test on Windows where the program
needs to be "python.exe script_name".
I've changed "fdis set" to treat the rest of the command as the program.
Then store that as a list to be passed to subprocess. If we just use a
string, Python will think that "python.exe foo" is the name of an actual
program instead of a program and an argument to it.
This will still break if the paths have spaces in, but I'm trying to do
just enough to fix the test here without rewriting all the option
handling.
LLDB uses the LLVM disassembler to determine the size of instructions and
to do the actual disassembly. Currently, if the LLVM disassembler can't
disassemble an instruction, LLDB will ignore the instruction size, assume
the instruction size is the minimum size for that device, print no useful
opcode, and print nothing for the instruction.
This patch changes this behavior to separate the instruction size and
"can't disassemble". If the LLVM disassembler knows the size, but can't
dissasemble the instruction, LLDB will use that size. It will print out
the opcode, and will print "<unknown>" for the instruction. This is much
more useful to both a user and a script.
The impetus behind this change is to clean up RISC-V disassembly when
the LLVM disassembler doesn't understand all of the instructions.
RISC-V supports proprietary extensions, where the TD files don't know
about certain instructions, and the disassembler can't disassemble them.
Internal users want to be able to disassemble these instructions.
With llvm-objdump, the solution is to pipe the output of the disassembly
through a filter program. This patch modifies LLDB's disassembly to look
more like llvm-objdump's, and includes an example python script that adds
a command "fdis" that will disassemble, then pipe the output through a
specified filter program. This has been tested with crustfilt, a sample
filter located at https://github.com/quic/crustfilt .
Changes in this PR:
- Decouple "can't disassemble" with "instruction size".
DisassemblerLLVMC::MCDisasmInstance::GetMCInst now returns a bool for
valid disassembly, and has the size as an out paramter.
Use the size even if the disassembly is invalid.
Disassemble if disassemby is valid.
- Always print out the opcode when -b is specified.
Previously it wouldn't print out the opcode if it couldn't disassemble.
- Print out RISC-V opcodes the way llvm-objdump does.
Code for the new Opcode Type eType16_32Tuples by Jason Molenda.
- Print <unknown> for instructions that can't be disassembled, matching
llvm-objdump, instead of printing nothing.
- Update max riscv32 and riscv64 instruction size to 8.
- Add example "fdis" command script.
- Added disassembly byte test for x86 with known and unknown instructions.
- Added disassembly byte test for riscv32 with known and unknown instructions,
with and without filtering.
- Added test from Jason Molenda to RISC-V disassembly unit tests.
This patch addresses 2 issues:
1. It makes registers available on non-crashed threads all the time
2. It fixes arm64 registers parsing for registers that don't use the `x`
prefix (`fp` -> `x29` / `lr` -> `x30`)
---------
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch adds support to the haswell sub-architecture (x86_64h) to
scripted processes.
rdar://147208252
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch makes interactive mode as the default when using the crashlog
command. It replaces the existing `-i|--interactive` flag with a new
`-m|--mode` option, that can either be `interactive` or `batch`.
By default, when the option is not explicitely set by the user, the
interactive mode is selected, however, lldb will fallback to batch mode
if the command interpreter is not interactive or if stdout is not a tty.
This also adds some railguards to prevent users from using interactive
only options with the batch mode and updates the tests accordingly.
rdar://97801509
Differential Revision: https://reviews.llvm.org/D141658
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
lldb's history log file is written to at the end of a debugging session.
As a result, the log does not contain commands run during the current
session.
This extends the `fzf_history` to include the output of `session
history`.
Adds a `fzf_history` to the examples directory.
This python command invokes [fzf](https://github.com/junegunn/fzf) to
select from lldb's command history.
Tighter integration is available on macOS, via commands for copy and
paste. The user's chosen history entry back is pasted into the lldb
console (via AppleScript). By pasting it, users have the opportunity to
edit it before running it. This matches how fzf's history search works.
Without copy and paste, the user's chosen history entry is printed to
screen and then run automatically.
This fixes a typo when creating a target from the crashlog script and
that we were not able to find a valid architecture from the crash
report.
rdar://137344016
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This PR adds a proof-of-concept for a bytecode designed to ship and
run LLDB data formatters. More motivation and context can be found in
the formatter-bytecode.rst file and on discourse.
https://discourse.llvm.org/t/a-bytecode-for-lldb-data-formatters/82696
Relanding with a fix for a case-sensitive path.
This PR adds a proof-of-concept for a bytecode designed to ship and
run LLDB data formatters. More motivation and context can be found in
the formatter-bytecode.rst file and on discourse.
https://discourse.llvm.org/t/a-bytecode-for-lldb-data-formatters/82696
Relanding with a fix for a case-sensitive path.
The newly added
test/API/functionalities/scripted_process_empty_memory_region/dummy_scripted_process.py
imports
examples/python/templates/scripted_process.py
which only has register definitions for x86_64 and arm64.
Only run this test on those two architectures for now.
If your arguments or option values are of a type that naturally uses one
of our common completion mechanisms, you will get completion for free.
But if you have your own custom values or if you want to do fancy things
like have `break set -s foo.dylib -n ba<TAB>` only complete on symbols
in foo.dylib, you can use this new mechanism to achieve that.
The API is present, and we even have a test for it, but it isn't
documented so no one probably knows you can set requirements for your
scripted commands. This just adds docs and uses it appropriately in the
`framestats` example command.
Among other things, returning an empty string as the repeat command
disables auto-repeat, which can be useful for state-changing commands.
There's one remaining refinement to this setup, which is that for parsed
script commands, it should be possible to change an option value, or add
a new option value that wasn't originally specified, then ask lldb "make
this back into a command string". That would make doing fancy things
with repeat commands easier.
That capability isn't present in the lldb_private side either, however.
So that's for a next iteration.
I haven't added this to the docs on adding commands yet. I wanted to
make sure this was an acceptable approach before I spend the time to do
that.
Following a feedback request in #97262, I took out the scripted thread
plan python base class from it and make a separate PR for it.
This patch adds the scripted thread plan base python class to the lldb
python module as well as the lldb documentation website.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch adds the documentation for a subset of scripting extensions
such as scripted process, scripted thread, operating system threads &
scritped thread plans to the lldb website.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
This patch makes interactive mode as the default when using the crashlog
command. It replaces the existing `-i|--interactive` flag with a new
`-m|--mode` option, that can either be `interactive` or `batch`.
By default, when the option is not explicitely set by the user, the
interactive mode is selected, however, lldb will fallback to batch mode
if the command interpreter is not interactive or if stdout is not a tty.
This also adds some railguards to prevent users from using interactive
only options with the batch mode and updates the tests accordingly.
rdar://97801509
Differential Revision: https://reviews.llvm.org/D141658
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This PR removes the `target-aarch64` requirement on the crashlog tests
to exercice them on Intel bots and make image loading single-threaded
temporarily while implementing a fix for a deadlock issue when loading
the images in parallel.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch changes the crashlog image loading default behaviour to not
only load images from the crashed thread but also for the application
specific backtrace thread.
This patch also move the Application Specific Backtrace / Last Exception
Backtrace tag from the thread queue field to the thread name.
rdar://128276576
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch adds the `--no-parallel-image-loading` to the crashlog
command. By default, image loading will happen in parallel in the
crashlog script however, sometimes, when running tests or debugging the
crashlog script itself, it's better to load the images sequentially.
As its name suggests, this flag will disable the default image loading
behaviour to load all the images sequencially in the main thread.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In lldb, users can change the `dsymForUUID` binary using the
`LLDB_APPLE_DSYMFORUUID_EXECUTABLE` environment variable.
This patch changes the crashlog to support the same behaviour as lldb
and uses this environment variable to disable `dsymForUUID` lookups in
crashlog test by having it be empty. Since CI bots shoudn't have access
to images on build records, it doesn't make sense to make use of
`dsymForUUID` in tests.
rdar://128953725
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This should fix the various crashlog test failures on the bots:
```
lldb-shell :: ScriptInterpreter/Python/Crashlog/app_specific_backtrace_crashlog.test
lldb-shell :: ScriptInterpreter/Python/Crashlog/interactive_crashlog_json.test
lldb-shell :: ScriptInterpreter/Python/Crashlog/interactive_crashlog_legacy.test
lldb-shell :: ScriptInterpreter/Python/Crashlog/last_exception_backtrace_crashlog.test
lldb-shell :: ScriptInterpreter/Python/Crashlog/skipped_status_interactive_crashlog.test
```
When we create a target by using the command option, we don't set it in
the crashlog object which later on cause us to fail loading the images.
rdar://127832961
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch fixes a bug in when resolving and loading modules from the
binary image list.
When loading a module, we would first use the UUID from the binary image
list with `dsymForUUID` to fetch the dSYM bundle from our remote build
records and copy the executable locally.
If we failed to find a matching dSYM bundle for that UUID on the build
record, let's say if that module was built locally, we use Spotlight
(`mdfind`) to find the dSYM bundle once again using the UUID.
Prior to this patch, we would set the image path to be the same as the
symbol file. This resulted in trying to load the dSYM as a module in
lldb, which isn't allowed.
This patch address that by looking for a binary matching the image
identifier, next to the dSYM bundle and try to load that instead.
rdar://127433616
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch rephrases the crashlog `--no-crashed-only` option help
message. This option is mainly used in batch mode to symbolicate and
dump all the threads backtraces, instead of only doing it for the
crashed thread which is the default behavior.
rdar://127391524
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In `27f27d1`, we changed the image loading logic to conform to the
various options (`-a|--load-all` & `-c|--crashed-only`) and loaded them
concurrently.
However, instead of the subset of images that matched the user option,
the thread pool would always run on all the crashlog images, causing
them to be all loaded in the target everytime.
This matches the `-a|--load-all` option behaviour but depending on the
report, it can cause lldb to load thousands of images, which can take a
very long time if the images are downloaded over the network.
This patch fixes that issue by keeping a list of `images_to_load` based
of
the user-provided option. This list will be used with our executor
thread pool to load the images according to the user selection, and
reinstates
the expected default behaviour, by only loading the crashed thread
images and
skipping all the others.
This patch also unifies the way we load images into a single method
that's shared by both the batch mode & the interactive scripted process.
rdar://123694062
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch should address some register parsing issue in the legacy
report format.
rdar://107210149
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Python3.9 does not allow you to put a reference to a class staticmethod
in a table and call it from there. Python3.10 and following do allow
this, but we still support 3.9. staticmethod was slightly cleaner,
but this will do.
This allows you to specify options and arguments and their definitions
and then have lldb handle the completions, help, etc. in the same way
that lldb does for its parsed commands internally.
This feature has some design considerations as well as the code, so I've
also set up an RFC, but I did this one first and will put the RFC
address in here once I've pushed it...
Note, the lldb "ParsedCommand interface" doesn't actually do all the
work that it should. For instance, saying the type of an option that has
a completer doesn't automatically hook up the completer, and ditto for
argument values. We also do almost no work to verify that the arguments
match their definition, or do auto-completion for them. This patch
allows you to make a command that's bug-for-bug compatible with built-in
ones, but I didn't want to stall it on getting the auto-command checking
to work all the way correctly.
As an overall design note, my primary goal here was to make an interface
that worked well in the script language. For that I needed, for
instance, to have a property-based way to get all the option values that
were specified. It was much more convenient to do that by making a
fairly bare-bones C interface to define the options and arguments of a
command, and set their values, and then wrap that in a Python class
(installed along with the other bits of the lldb python module) which
you can then derive from to make your new command. This approach will
also make it easier to experiment.
See the file test_commands.py in the test case for examples of how this
works.
On macOS, we usually use the DebugSymbols framework to find dSYMs, but
we have a few places (including crashlog.py) that calls out directly to
dsymForUUID. Currently, this invocation is missing two important
options:
* `--ignoreNegativeCache`: Poor network connectivity or lack of VPN can
lead to a negative cache hit. Avoiding those issues is worth the penalty
of skipping these caches.
* `--copyExecutable`: Ensure we copy the executable as it might not be
available at its original location.
rdar://118480731
This patch fixes the various crashlog test failures following
ec456ba9ca0a, which renamed the process member variable in the Scripted
Thread python base class.
This patch updates the crashlog scripted process implementation to
reflect that change.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch introduces an `OperatingSystem` base implementation in the
`lldb` python module to make it easier for lldb users to write their own
implementation.
The `OperatingSystem` base implementation is derived itself from the
`ScriptedThread` base implementation since they share some common grounds.
To achieve that, this patch makes changes to the `ScriptedThread`
initializer since it gets called by the `OperatingSystem` initializer.
I also took the opportunity to document the `OperatingSystem` base
class and methods.
Differential Revision: https://reviews.llvm.org/D159315
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch moves the template files for the various scripting
affordances to a separate directory.
This is a preparatory work for upcoming improvements and consolidations
to other scripting affordances.
Differential Revision: https://reviews.llvm.org/D159310
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In 21a597c, we fixed a module loading issue by using the new
`argparse.BooleanOptionalAction`. However, this is only available
starting python 3.9 and causes test failures on bots that don't fulfill
this requirement.
To address that, this patch replaces the use of `BooleanOptionalAction`
by a pair of 2 opposite `store` actions pointing to the same destination
variable.
Differential Revision: https://reviews.llvm.org/D158452
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch fixes TestSaveCrashlog.py failure introduces by eef5eadbe617,
which restricts the number of positional argument for the output file to 1.
I expected to get the output file but `argparse` puts the object in a
list (even by constrained to a singled positional argument).
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
When using the `crashlog` command in batch mode, most users only care
about the crashed thread and end up having to scroll past all the
non-crashed threads, which is not a good user experience.
Now that `-c|--crashed-only` is set by default, we should also apply
that behavior for batch mode: Only the crashed thread and "Application
Specific Backtrace" threads will be shown to the user in batch mode.
The user will still have the ability to show all the threads if they use
`--no-crashed-only` which will parse the symbols from the report, or
with `-a|--load-all-images` which will fetch binaries and debug info
from the build record and symbolicate every thread.
rdar://106329893
Differential Revision: https://reviews.llvm.org/D157852
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch adds support to the "Last Exception Backtrace" to the
`crashlog` command.
This metadata is homologous to the "Application Specific Backtrace",
however the format is closer to a regular stack frame.
Since the thread that "contains" the "Last Exception Backtrace" doesn't
really exist, this information is displayed when requesting an extended
backtrace of the crashed thread, similarly to the "Application Specific
Backtrace".
To achieve that, this patch includes some refactors and fixes to the
existing "Application Specific Backtrace" handling.
rdar://113046509
Differential Revision: https://reviews.llvm.org/D157851
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Before 27f27d15f, the `crashlog` command would always load images even
if `-a` or `-c` was not set by the user.
Since that change, images are loaded only when one of these 2 flags are
set, otherwise, we fallback to parsing the symbols from the report and
load them into a `SymbolFileJSON`.
Although that makes it way faster than pulling binaries and debug
symbols from build records, that cause a degraded experience since none
of our users are used to set these 2 flags. For instance, that would
symbolicate the backtraces, however the users wouldn't see sources.
To address that change of behavior, this patch changes the default value
for the `-c|--crash-only` flag to `true`. On the other hand, thanks to
the move to `argparse`, we introduced a new `--no-only-crashed` flag
that will let the user force skipping loading any images, relying only
on the `SymbolFileJSON`.
This gives the users a good compromise since they would be able to see
sources for the crashed thread if they're available, otherwise, they'll
only get a symbolicated backtrace.
Differential Revision: https://reviews.llvm.org/D157850
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch replace the deprecated `optparse` module used for the
`crashlog`& `save_crashlog` commands with the new `argparse` from the
python standard library. This provides many benefits such as showing the
default values for each option in the help description, but also greatly
improve the handling of position arguments.
Differential Revision: https://reviews.llvm.org/D157849
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch skips images with an empty path or a 0 UUID from loading as a
SymbolFileJSON.
rdar://112107986
Differential Revision: https://reviews.llvm.org/D157137
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Prior to this patch, when a user loaded multiple crash report in lldb,
they could get in a situation where all the targets would keep the same
architecture and executable path as the first one that we've created.
The reason behind this was that even if we created a new CrashLog
object, which is derived from a Symbolicator class that has a newly
constructoted image list as a default argument, because that default
argument is only created once when the function is defined, every CrashLog
object would share the same list.
That will cause use to append newly parsed images to the same
Symbolicator image list accross multiple CrashLog objects.
To address this, this patch changes the default argument value for the
image parameter to `None` and only initialize it as an empty list when
no argument was passed.
This also removes the image list stored in each CrashLog parsers since
they shouldn't have any state and should be re-usable. So now, the only
source of truth is stored in the CrashLog object.
rdar://84984949
Differential Revision: https://reviews.llvm.org/D157044
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
