Clang's v2 output appears to be missing a key DW_AT_type attribute,
and this causes the "max" of the unsigned enum to appear as -1 instead
of "max" aka 3.
```
(BitfieldStruct) $0 = {
signed_min = min
signed_other = -1
signed_max = max
unsigned_min = min
unsigned_other = 1
unsigned_max = -1
}
```
Test added by #96202.
Among other things, returning an empty string as the repeat command
disables auto-repeat, which can be useful for state-changing commands.
There's one remaining refinement to this setup, which is that for parsed
script commands, it should be possible to change an option value, or add
a new option value that wasn't originally specified, then ask lldb "make
this back into a command string". That would make doing fancy things
with repeat commands easier.
That capability isn't present in the lldb_private side either, however.
So that's for a next iteration.
I haven't added this to the docs on adding commands yet. I wanted to
make sure this was an acceptable approach before I spend the time to do
that.
While testing register fields I found that if you put the max value into
a bitfield with an underlying type that is an unsigned enum, lldb would
not print the enum name.
This is because the code to match values to names wasn't checking
whether the enum's type was signed, it just assumed it was.
So for example a 2 bit field with value 3 got signed extended to -1,
which didn't match the enumerator value of 3. So lldb just printed the
number instead of the name.
For a value of 1, the top bit was 0 so the sign extend became a zero
extend, and lldb did print the name of the enumerator.
I added a new test because I needed to use C++ to get typed enums. It
checks min, max and an in between value for signed and unsigned enums
applied to a bitfield.
This enables XML output for enums and adds enums for 2 fields on AArch64
Linux:
* mte_ctrl.tcf, which controls how tag faults are delivered.
* fpcr.rmode, which sets the rounding mode for floating point
operations.
The other one we could do is cpsr.btype, but it is not clear what would
be useful here so I'm not including it in this change.
This extends the existing register fields support from AArch64 Linux
to AArch64 FreeBSD. So you will now see output like this:
```
(lldb) register read cpsr
cpsr = 0x60000200
= (N = 0, Z = 1, C = 1, V = 0, DIT = 0, SS = 0, IL = 0, SSBS = 0, D = 1, A = 0, I = 0, F = 0, nRW = 0, EL = 0, SP = 0)
```
Linux and FreeBSD both have HWCAP/HWCAP2 so the detection mechanism
is the same and I've renamed the detector class to reflect that.
I have confirmed that FreeBSD's treatment of CPSR (spsr as the kernel
calls it) is similair enough that we can use the same field information.
(see `sys/arm64/include/armreg.h` and `PSR_SETTABLE_64`)
For testing I've enabled the same live process test as Linux
and added a shell test using an existing FreeBSD core file.
Note that the latter does not need XML support because when reading
a core file we are not sending the information via target.xml,
it's just internal to LLDB.
In one of my recent PRs I mistakenly had two test-cases with the same
name, preventing one of them to run. Since it's an easy mistake to make
(e.g., copy pasting existing test-cases), I ran following sanity-check
script over `lldb/test/API`, which found couple of tests which were
losing coverage because of this (or in some cases simply had duplicate
tests):
```
import ast
import sys
filename = sys.argv[1]
print(f'Checking {filename}...')
tree = ast.parse(open(filename, 'r').read())
for node in ast.walk(tree):
if not isinstance(node, ast.ClassDef):
continue
func_names = []
for child in ast.iter_child_nodes(node):
if isinstance(child, ast.FunctionDef):
func_names.append(child.name)
seen_func_names = set()
duplicate_func_names = []
for name in func_names:
if name in seen_func_names:
duplicate_func_names.append(name)
else:
seen_func_names.add(name)
if len(duplicate_func_names) != 0:
print(f'Multiple func names found:\n\t{duplicate_func_names}\n\tclass {node.name}\n\tfile: {filename}')
```
This patch fixes these cases.
These handful of tests had a BOM (Byte order mark) at the beginning of
the file. This marker is unnecessary in our test files. The main
motivation for this is that the `ast` python module breaks when passing
a file to it with a BOM marker (and might break other tooling which
doesn't expect it). E.g.,:
```
"""Test that lldb command 'process signal SIGUSR1' to send a signal to the inferior works."""
^
SyntaxError: invalid non-printable character U+FEFF
```
If anyone is aware of a good reason to keep it, happy to drop this.
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
This change by itself has no measurable effect on the LLDB
testsuite. I'm making it in preparation for threading through more
errors in the Swift language plugin.
# Added/changed options
The following options are **added** to the `statistics dump` command:
* `--targets=bool`: Boolean. Dumps the `targets` section.
* `--modules=bool`: Boolean. Dumps the `modules` section.
When both options are given, the field `moduleIdentifiers` will be
dumped for each target in the `targets` section.
The following options are **changed**:
* `--transcript=bool`: Changed to a boolean. Dumps the `transcript`
section.
# Behavior of `statistics dump` with various options
The behavior is **backward compatible**:
- When no options are provided, `statistics dump` dumps all sections.
- When `--summary` is provided, only dumps the summary info.
**New** behavior:
- `--targets=bool`, `--modules=bool`, `--transcript=bool` overrides the
above "default".
For **example**:
- `statistics dump --modules=false` dumps summary + targets +
transcript. No modules.
- `statistics dump --summary --targets=true --transcript=true` dumps
summary + targets (in summary mode) + transcript.
# Added options into public API
In `SBStatisticsOptions`, add:
* `Set/GetIncludeTargets`
* `Set/GetIncludeModules`
* `Set/GetIncludeTranscript`
**Alternative considered**: Thought about adding
`Set/GetIncludeSections(string sections_spec)`, which receives a
comma-separated list of section names to be included ("targets",
"modules", "transcript"). The **benefit** of this approach is that the
API is more future-proof when it comes to possible adding/changing of
section names. **However**, I feel the section names are likely to
remain unchanged for a while - it's not like we plan to make big changes
to the output of `statistics dump` any time soon. The **downsides** of
this approach are: 1\ the readability of the API is worse (requires
reading doc to understand what string can be accepted), 2\ string input
are more prone to human error (e.g. typo "target" instead of expected
"targets").
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/statistics/basic/TestStats.py
```
```
./tools/lldb/unittests/Interpreter/InterpreterTests
```
New test cases have been added to verify:
* Different sections are dumped/not dumped when different
`StatisticsOptions` are given through command line (CLI or
`HandleCommand`; see `test_sections_existence_through_command`) or API
(see `test_sections_existence_through_api`).
* The order in which the options are given in command line does not
matter (see `test_order_of_options_do_not_matter`).
---------
Co-authored-by: Roy Shi <royshi@meta.com>
When PIE is enabled on a platform by default, these tests fail since the
`target variable` command can't read a global string variable value
before running an inferior process.
It fixes the following tests when built with clang on Ubuntu aarch64:
```
commands/target/basic/TestTargetCommand.py
lang/c/global_variables/TestGlobalVariables.py
lang/cpp/char8_t/TestCxxChar8_t.py
```
# Changes
1. Changes to the structured transcript.
1. Add fields `commandName` and `commandArguments`. They will hold the
name and the arguments string of the expanded/executed command (e.g.
`breakpoint set` and `-f main.cpp -l 4`). This is not to be confused
with the `command` field, which holds the user input (e.g. `br s -f
main.cpp -l 4`).
2. Add field `timestampInEpochSeconds`. It will hold the timestamp when
the command is executed.
3. Rename field `seconds` to `durationInSeconds`, to improve
readability, especially since `timestampInEpochSeconds` is added.
2. When transcript is available and the newly added option
`--transcript` is present, add the transcript to the output of
`statistics dump`, as a JSON array under a new field `transcript`.
3. A few test name and comment changes.
1. Use dashes (-) instead of colons (:) as time separator in a session log
file name since Windows doesn't support saving files with names containing
colons.
2. Temporary file creation code is changed in the test:
On Windows, the temporary file should be closed before 'session save'
writes session log to it. NamedTemporaryFile() can preserve the file
after closing it with delete_on_close=False option.
However, this option is only available since Python 3.12. Thus
mkstemp() is used for temporary file creation as the more compatible
option.
PR #92245 fixed these tests on Linux. They likely work on FreeBSD too
but leaving the xfail for that so it can be confirmed later.
Also updated a bugzilla link to one that redirects to Github issues.
Relates to issues #43398 and #48751.
# Motivation
Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::SaveTranscript`). However, the
only way to obtain the transcript data as a user is to first destroy the
debugger, then read the save directory. Note that destroy-callbacks
cannot be used, because 1\ transcript data is private to the command
interpreter (see `CommandInterpreter.h`), and 2\ the writing of the
transcript is *after* the invocation of destory-callbacks (see
`Debugger::Destroy`).
So basically, there is no way to obtain the transcript:
* during the lifetime of a debugger (including the destroy-callbacks,
which often performs logging tasks, where the transcript can be useful)
* without relying on external storage
In theory, there are other ways for user to obtain transcript data
during a debugger's life cycle:
* Use Python API and intercept commands and results.
* Use CLI and record console input/output.
However, such ways rely on the client's setup and are not supported
natively by LLDB.
# Proposal
Add a new Python API `SBCommandInterpreter::GetTranscript()`.
Goals:
* It can be called at any time during the debugger's life cycle,
including in destroy-callbacks.
* It returns data in-memory.
Structured data:
* To make data processing easier, the return type is `SBStructuredData`.
See comments in code for how the data is organized.
* In the future, `SaveTranscript` can be updated to write different
formats using such data (e.g. JSON). This is probably accompanied by a
new setting (e.g. `interpreter.save-session-format`).
# Alternatives
The return type can also be `std::vector<std::pair<std::string,
SBCommandReturnObject>>`. This will make implementation easier, without
having to translate it to `SBStructuredData`. On the other hand,
`SBStructuredData` can convert to JSON easily, so it's more convenient
for user to process.
# Privacy
Both user commands and output/error in the transcript can contain
privacy data. However, as mentioned, the transcript is already available
to the user. The addition of the new API doesn't increase the level of
risk. In fact, it _lowers_ the risk of privacy data being leaked later
on, by avoiding writing such data to external storage.
Once the user (or their code) gets the transcript, it will be their
responsibility to make sure that any required privacy policies are
guaranteed.
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/interpreter/TestCommandInterpreterAPI.py
```
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/session/save/TestSessionSave.py
```
---------
Co-authored-by: Roy Shi <royshi@meta.com>
Co-authored-by: Med Ismail Bennani <ismail@bennani.ma>
os.path.join() uses the path separator of the host OS by default.
outfile_arg will be incorrect in case of Windows host and Linux target.
Use lldbutil.append_to_process_working_directory() instead.
The setting `platform.module-cache-directory` is a local path on the
host. It cannot be set to a working directory from the remote target.
This test failed in case of Windows host and Linux target because of the
incompatible path. Use the local build dir instead.
Fixes#85084
Whenever an inferior thread stops, lldb-server sends a SIGSTOP to all
other threads in the process to force them to stop as well. If those
threads stop on their own before they get a signal, this SIGSTOP will
remain pending and be delivered the next time the process resumes.
Normally, this is not a problem, because lldb-server will detect this
stale SIGSTOP and resume the process. However, if we detach from the
process while it has these SIGSTOPs pending, they will get immediately
delivered, and the process will remain stopped (most likely forever).
This patch fixes that by sending a SIGCONT just before detaching from
the process. This signal cancels out any pending SIGSTOPs, and ensures
it is able to run after we detach. It does have one somewhat unfortunate
side-effect that in that the process's SIGCONT handler (if it has one)
will get executed spuriously (from the process's POV).
This could be _sometimes_ avoided by tracking which threads got send a
SIGSTOP, and whether those threads stopped due to it. From what I could
tell by observing its behavior, this is what gdb does. I have not tried
to replicate that behavior here because it adds a nontrivial amount of
complexity and the result is still uncertain -- we still need to send a
SIGCONT (and execute the handler) when any thread stops for some other
reason (and leaves our SIGSTOP hanging). Furthermore, since SIGSTOPs
don't stack, it's also possible that our SIGSTOP/SIGCONT combination
will cancel a genuine SIGSTOP being sent to the debugger application (by
someone else), and there is nothing we can do about that. For this
reason I think it's simplest and most predictible to just always send a
SIGCONT when detaching, but if it turns out this is breaking something,
we can consider implementing something more elaborate.
One alternative I did try is to use PTRACE_INTERRUPT to suspend the
threads instead of a SIGSTOP. PTRACE_INTERUPT requires using
PTRACE_SEIZE to attach to the process, which also made this solution
somewhat complicated, but the main problem with that approach is that
PTRACE_INTERRUPT is not considered to be a signal-delivery-stop, which
means it's not possible to resume it while injecting another signal to
the inferior (which some of our tests expect to be able to do). This
limitation could be worked around by forcing the thread into a signal
delivery stop whenever we need to do this, but this additional
complication is what made me think this approach is also not worthwhile.
This patch should fix (at least some of) the problems with
TestConcurrentVFork, but I've also added a dedicated test for checking
that a process keeps running after we detach. Although the problem I'm
fixing here is linux-specific, the core functinoality of not stopping
after a detach should function the same way everywhere.
This patch attempts to decouple C++ expression evaluation from
Objective-C support. We've previously enabled it by default (if a
runtime existed), but that meant we're opting into extra work we only
need to do for Objective-C, which complicates/slows down C++ expression
evaluation. Of course there's a valid use-case for this, which is
calling Objective-C APIs when stopped in C++ frames (which Objective-C++
developers might want to do). In those cases we should really prompt the
user to add the `expr --language objc++` flag. To accomodate a likely
frequent use-case where a user breaks in a system C++ library (without
debug-symbols) but their application is actually an Objective-C app, we
allow Objective-C support in C++ expressions if the current frame
doesn't have debug-info.
This fixes https://github.com/llvm/llvm-project/issues/75443 and allows
us to add more `LangOpts.ObjC` guards around the expression evaluator in
the future (e.g., we could avoid looking into the Objective-C runtime
during C++ expression evaluation, which we currently do
unconditionally).
Depends on https://github.com/llvm/llvm-project/pull/87657
The idea behind the address-expression is that it handles all the common
expressions that produce addresses. It handles actual valid expressions
that return a scalar, and it handles useful cases that the various
source languages don't support. At present, the fallback handles:
<symbol_name>{+-}<offset>
which isn't valid C but is very handy.
This patch adds handling of:
$<reg_name>
and
$<reg_name>{+-}<offset>
That's kind of pointless in C because the C expression parser handles
that expression already. But some languages don't have a straightforward
way to represent register values like this (swift) so having this
fallback is quite a quality of life improvement.
I added a test which tests that I didn't mess up either of these
fallbacks, though it doesn't test the actually handling of registers
that I added, since the expression parser for C succeeds in that case
and returns before this code gets run.
I will add a test on the swift fork for that checks that this works the
same way for a swift frame after this check.
`EvaluateExpression` does not always create a new persistent result. If the expression
is a bare persistent variable, then a new persistent result is not created. This means
the caller can't assume a new persistent result is created for each evaluation.
However, `dwim-print` was doing exactly that: assuming a new persistent result for each
evaluation. This resulted in a bug:
```
(lldb) p int $j = 23
(lldb) p $j
(lldb) p $j
```
The first `p $j` would not create a persistent result, and so `dwim-print` would
inadvertently delete `$j`. The second `p $j` would fail.
The fix is to try `expr` as a persistent variable, after trying `expr` as a frame
variable. For persistent variables, this avoids calling `EvaluateExpression`.
Resolves https://github.com/llvm/llvm-project/issues/84806
rdar://124688427
There was a think-o in a previous commit that made us only able to
define 1 line commands when using command script add interactively.
There was also no test for this feature, so I fixed the think-o and
added a test.
I noticed that the term-width setting would always report its default
value (80) despite the driver correctly setting the value with
SBDebugger::SetTerminalWidth.
```
(lldb) settings show term-width
term-width (int) = 80
```
The issue is that the setting was defined as a SInt64 instead of a
UInt64 while the getter returned an unsigned value. There's no reason
the terminal width should be a signed value. My best guess it that it
was using SInt64 because UInt64 didn't support min and max values. I
fixed that and correct the type and now lldb reports the correct
terminal width:
```
(lldb) settings show term-width
term-width (unsigned) = 189
```
rdar://123488999
Any time we see the pattern `assertEqual(value, bool)`, we can replace
that with `assert<bool>(value)`. Likewise for `assertNotEqual`.
Technically this relaxes the test a bit, as we may want to make sure
`value` is either `True` or `False`, and not something that implicitly
converts to a bool. For example, `assertEqual("foo", True)` will fail,
but `assertTrue("foo")` will not. In most cases, this distinction is not
important.
There are two such places that this patch does **not** transform, since
it seems intentional that we want the result to be a bool:
*
5daf2001a1/lldb/test/API/python_api/sbstructureddata/TestStructuredDataAPI.py (L90)
*
5daf2001a1/lldb/test/API/commands/settings/TestSettings.py (L940)
Followup to 9c2468821ec51defd09c246fea4a47886fff8c01. I patched `teyit`
with a `visit_assertEqual` node handler to generate this.
This uses [teyit](https://pypi.org/project/teyit/) to modernize asserts,
as recommended by the [unittest release
notes](https://docs.python.org/3.12/whatsnew/3.12.html#id3).
For example, `assertTrue(a == b)` is replaced with `assertEqual(a, b)`.
This produces better error messages, e.g. `error: unexpectedly found 1
and 2 to be different` instead of `error: False`.
This is a follow-on to:
https://github.com/llvm/llvm-project/pull/82085
The completer for register names was missing from the argument table. I
somehow missed that the only register completer test was x86_64, so that
test broke.
I added the completer in to the right slot in the argument table, and
added a small completions test that just uses the alias register names.
If we end up having a platform that doesn't define register names, we'll
have to skip this test there, but it should add a sniff test for
register completion that will run most everywhere.
This reverts commit 21631494b068d9364b8dc8f18e59adee9131a0a5.
Reverted because of greendragon failure:
******************** TEST 'lldb-api :: functionalities/completion/TestCompletion.py' FAILED ********************
Script:
Most commands were adding argument completion handling by themselves,
resulting in a lot of unnecessary boilerplate. In many cases, this could
be done generically given the argument definition and the entries in the
g_argument_table.
I'm going to address this in a couple passes. In this first pass, I
added handling of commands that have only one argument list, with one
argument type, either single or repeated, and changed all the commands
that are of this sort (and don't have other bits of business in their
completers.)
I also added some missing connections between arg types and completions
to the table, and added a RemoteFilename and RemotePath to use in places
where we were using the Remote completers. Those arguments used to say
they were "files" but they were in fact remote files.
I also added a module arg type to use where we were using the module
completer. In that case, we should call the argument module.
Updates:
- The previous patch changed the default behavior to not load dwos in
`DWARFUnit`
~~`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info =
false);`~~
`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info = true);`
- This broke some lldb-shell tests (see
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273/)
- TestDebugInfoSize.py
- with symbol on-demand, by default statistics dump only reports
skeleton debug info size
- `statistics dump -f` will load all dwos. debug info = skeleton debug
info + all dwo debug info
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
Switching to modern `unittest` in
5b386158aacac4b41126983a5379d36ed413d0ea needs xfail annotations to be
known prior to test running. In contrast, skipping can happen at any
time, even during test execution.
Thus, `expectedFailureIfFn` inherently doesn't work. Either we eagerly
evaluate the function and use `expectedFailureIf` instead, or we use a
skip annotation to lazily evaluate the function and potentially skip the
test right before it starts.
- For `expectedFailureAndroid`, the intent seems to be that certain
tests _should_ work on android, but don't. Thus, xfail is appropriate,
to ensure the test is re-enabled once those bugs are ever fixed.
- For the other uses in individual tests, those generally seem to be
cases where the test environment doesn't support the setup required by
the test, and so it isn't meaningful to run the test at all. For those,
a drop-in replacement to `skipTestIfFn` works.
Python3.9 does not allow you to put a reference to a class staticmethod
in a table and call it from there. Python3.10 and following do allow
this, but we still support 3.9. staticmethod was slightly cleaner,
but this will do.
When the parsed command python code is run on 3.9, I get:
File ".../lib/python3.9/site-packages/lldb/plugins/parsed_cmd.py", line 124, in translate_value
return cls.translators[value_type](value)
TypeError: 'staticmethod' object is not callable
But this works correctly in Python 3.10 on macOS and Linux. I'm guessing something
changed between those versions, and I'll have to do something to work around the difference.
But I'm going to skip the test on 3.9 while I figure that out.
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
