Originally commited in 362b9d78b4ee9107da2b5e90b3764b0f0fa610fe and then
reverted in cb63b75e32a415c9bfc298ed7fdcd67e8d9de54c.
This re-lands a subset of the changes to
dap_server.py/DebugCommunication and addresses the python3.10
compatibility issue.
This includes less type annotations since those were the reason for the
failures on that specific version of python.
I've done additional testing on python3.8, python3.10 and python3.13 to
further validate these changes.
Resolves#141955
- Adds data to breakpoints `Source` object, in order for assembly
breakpoints, which rely on a temporary `sourceReference` value, to be
able to resolve in future sessions like normal path+line breakpoints
- Adds optional `instructions_offset` parameter to `BreakpointResolver`
You cannot use a NamedTempFile with an external process because it may
not be flushed to disk. The safest and most portable approach is to
close the file, call the other process and then unlink the file
manually.
Presumably this works fine on Linux, but it fails on Darwin when
targeting remote-linux.
See https://bugs.python.org/issue29573
Fix cross-compilation of test inferiors on Darwin, targeting remote
Linux. This requires specifying the target triple and using LLD for
linking.
Fixes#150806
When debug info categories were set for a test method with the
`@add_test_categories` decorator, they were all added to its
"categories" attribute. If some of these categories were not supported,
`LLDBTestResult.startTest()` skipped all variants of the test method.
For example, the tests in `TestWithLimitDebugInfo.py` use the categories
`dwarf` and `dwo`. However, since `dwo` is not supported on Windows, all
the tests in this file were skipped, even though the tests for `dwarf`
could be run.
Pick the builder for the target platform, not the host platform. This is
necessary when running the test suite remotely on a different platform.
Unlike for Darwin, both Windows and Linux us the default builder, which
is why this went unnoticed on the remote-linux bots.
Pick the builder for the target platform, not the host platform. This is
necessary when running the test suite remotely on a different platform.
Unlike for Darwin, both Windows and Linux us the default builder, which
is why this went unnoticed on the remote-linux bots.
We emit a warning when running the test suite remotely that says the
libcxx arguments will be ignored, but because they're set outside the
conditional block, we're not actually do this. Fix the logic by moving
the configuration in the conditional else-block.
Extend support in LLDB for WebAssembly. This PR adds a new Process
plugin (ProcessWasm) that extends ProcessGDBRemote for WebAssembly
targets. It adds support for WebAssembly's memory model with separate
address spaces, and the ability to fetch the call stack from the
WebAssembly runtime.
I have tested this change with the WebAssembly Micro Runtime (WAMR,
https://github.com/bytecodealliance/wasm-micro-runtime) which implements
a GDB debug stub and supports the qWasmCallStack packet.
```
(lldb) process connect --plugin wasm connect://localhost:4567
Process 1 stopped
* thread #1, name = 'nobody', stop reason = trace
frame #0: 0x40000000000001ad
wasm32_args.wasm`main:
-> 0x40000000000001ad <+3>: global.get 0
0x40000000000001b3 <+9>: i32.const 16
0x40000000000001b5 <+11>: i32.sub
0x40000000000001b6 <+12>: local.set 0
(lldb) b add
Breakpoint 1: where = wasm32_args.wasm`add + 28 at test.c:4:12, address = 0x400000000000019c
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, name = 'nobody', stop reason = breakpoint 1.1
frame #0: 0x400000000000019c wasm32_args.wasm`add(a=<unavailable>, b=<unavailable>) at test.c:4:12
1 int
2 add(int a, int b)
3 {
-> 4 return a + b;
5 }
6
7 int
(lldb) bt
* thread #1, name = 'nobody', stop reason = breakpoint 1.1
* frame #0: 0x400000000000019c wasm32_args.wasm`add(a=<unavailable>, b=<unavailable>) at test.c:4:12
frame #1: 0x40000000000001e5 wasm32_args.wasm`main at test.c:12:12
frame #2: 0x40000000000001fe wasm32_args.wasm
```
This PR is based on an unmerged patch from Paolo Severini:
https://reviews.llvm.org/D78801. I intentionally stuck to the
foundations to keep this PR small. I have more PRs in the pipeline to
support the other features/packets.
My motivation for supporting Wasm is to support debugging Swift compiled
to WebAssembly:
https://www.swift.org/documentation/articles/wasm-getting-started.html
This controls whether tag checking is performed for loads and
stores, or stores only.
It requires a specific architecture feature which we detect
with a HWCAP3 and cpuinfo feature.
Live process tests look for this and adjust expectations
accordingly, core file tests are using an updated file with
this feature enabled.
The size of the core file has increased and there's nothing
I can do about that. Could be the presence of new architecure
features or kernel changes since I last generated them.
I can generate a smaller file that has the tag segment,
but that segment does not actually contain tag data. So
that's no use.
Currently, the "stopped" event returned when a breakpoint is hit will
always return only the ID of first breakpoint returned from
`GetStopReasonDataAtIndex`. This is slightly different from the
behaviour in `lldb`, where multiple breakpoints can exist at a single
instruction address and all are returned as part of the stop reason when
that address is hit.
This patch allows all multiple hit breakpoints to be returned in the
"stopped" event, both in the hitBreakpointIds field and in the
description, using the same formatting as lldb e.g. "breakpoint 1.1
2.1". I'm not aware of any effect this will have on debugger plugins; as
far as I can tell, it makes no difference within the VS Code UI - this
just fixes a minor issue encountered while writing an `lldb-dap` backend
for Dexter.
This commit adds completion support for the plugin commands. It will try
to complete partial namespaces to the full namespace string. If the
completion input is already a full namespace string then it will add all
the matching plugins in that namespace as completions.
This lets the user complete to the namespace first and then tab-complete
to the next level if desired.
```
(lldb) plugin list a<tab>
Available completions:
abi
architecture
(lldb) plugin list ab<tab>
(lldb) plugin list abi<tab>
(lldb) plugin list abi.<tab>
Available completions:
abi.SysV-arm64
abi.ABIMacOSX_arm64
abi.SysV-arm
...
```
From #148554 - compile tests with exceptions on Windows
(`-fno-exceptions` was added 11 years ago in
c7826524acda6a9c8816261d5c48b94dc92935ed). The variant test uses `try {}
catch {}` to create variants that are valueless by exception. On other
platforms, exceptions are enabled as well.
I have no clue why compiling with exceptions will optimize out
`a_long_guy` in the changed test (even with `-O0`). Taking the address
of that value will ensure it's kept.
This updates all the existing memory reference fields to use
`lldb::addr_t` directly.
A few places were using `std::string` and decoding the value in the
request handler and other places had unique ways of parsing addresses.
This unifies all of these references with the `DecodeMemoryReference`
helper in JSONUtils.h.
Additionally, for the types I updated, I tried to simplify the POD types
some and moved default values out of RequestHandlers and into the
protocol POD types.
This PR adds type summaries for
`std::{string,wstring,u8string,u16string,u32string}` from the MSVC STL.
See https://github.com/llvm/llvm-project/issues/24834 for the MSVC STL
issue.
The following changes were made:
- `dotest.py` now detects if the MSVC STL is available. It does so by
looking at the target triple, which is an additional argument passed
from Lit. It specifically checks for `windows-msvc` to not match on
`windows-gnu` (i.e. MinGW/Cygwin).
- (The main part): Added support for summarizing `std::(w)string` from
MSVC's STL. Because the type names from the libstdc++ (pre C++ 11)
string types are the same as on MSVC's STL, `CXXCompositeSummaryFormat`
is used with two entries, one for MSVC's STL and one for libstdc++.
With MSVC's STL, `std::u{8,16,32}string` is also handled. These aren't
handled for libstdc++, so I put them in `LoadMsvcStlFormatters`.
When there is a function that is inlined at the current program counter.
If you get the current `line_entry` using the program counter's address
it will point to the location of the inline function that may be in
another file. (this is in implicit step-in and should not happen what
step over is called).
Use the current frame to get the `line_entry`
# Benefit
This patch fixes:
1. After `platform select ios-simulator`, `platform process list` will
now print processes which are running in the iOS simulator. Previously,
no process will be listed.
2. After `platform select ios-simulator`, `platform attach --name
<name>` will succeed. Previously, it will error out saying no process is
found.
# Several bugs that is being fixed
1. During the process listing, add `aarch64` to the list of CPU types
for which iOS simulators are checked for.
2. Given a candidate process, when checking for simulators, the original
code will find the desired environment variable (`SIMULATOR_UDID`) and
set the OS to iOS, but then the immediate next environment variable will
set it back to macOS.
3. For processes running on simulator, set the triple's `Environment` to
`Simulator`, so that such processes can pass the filtering [in this
line](https://fburl.com/8nivnrjx). The original code leave it as the
default `UnknownEnvironment`.
# Manual test
**With this patch:**
```
royshi-mac-home ~/public_llvm/build % bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
240 matching processes were found on "ios-simulator"
PID PARENT USER TRIPLE NAME
====== ====== ========== ============================== ============================
40511 28844 royshi arm64-apple-ios-simulator FocusPlayground // my toy iOS app running on simulator
... // omit
28844 1 royshi arm64-apple-ios-simulator launchd_sim
(lldb) process attach --name FocusPlayground
Process 40511 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x0000000104e3cb70 libsystem_kernel.dylib`mach_msg2_trap + 8
libsystem_kernel.dylib`mach_msg2_trap:
-> 0x104e3cb70 <+8>: ret
... // omit
```
**Without this patch:**
```
$ bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
error: no processes were found on the "ios-simulator" platform
(lldb) process attach --name FocusPlayground
error: attach failed: could not find a process named FocusPlayground
```
# Unittest
See PR.
This fixes issues found in e066f35c6981c720e3a7e5883efc40c861b3b7, which
was later reverted. The problem was with "k" message which was sent with
sync_on_timeout flag set to true, so lldb was waiting for response,
which is currently not being sent by lldb-server. Not waiting for
response at all seems to be not a solution, because on MAC OS X lldb
waits for response from "k" to gracefully kill inferior.
This relands changes in #144424 for adding a count of DWO files
parsed/loaded and the total number of DWO files.
The previous PR was reverted in #145494 due to the newly added unit
tests failing on Windows and MacOS CIs since these platforms don't
support DWO. This change add an additional
`@add_test_categories(["dwo"])` to the new tests to
[skip](cd46354dbd/lldb/packages/Python/lldbsuite/test/test_categories.py (L56))
these tests on Windows/MacOS.
Original PR: #144424
### Testing
Ran unit tests
```
$ bin/lldb-dotest -p TestStats.py llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 24 tests in 211.391s
OK (skipped=3)
```
## Summary
A new `totalLoadedDwoFileCount` and `totalDwoFileCount` counters to
available statisctics when calling "statistics dump".
1. `GetDwoFileCounts ` is created, and returns a pair of ints
representing the number of loaded DWO files and the total number of DWO
files, respectively. An override is implemented for `SymbolFileDWARF`
that loops through each compile unit, and adds to a counter if it's a
DWO unit, and then uses `GetDwoSymbolFile(false)` to check whether the
DWO file was already loaded/parsed.
3. In `Statistics`, use `GetSeparateDebugInfo` to sum up the total
number of loaded/parsed DWO files along with the total number of DWO
files. This is done by checking whether the DWO file was already
successfully `loaded` in the collected DWO data, anding adding to the
`totalLoadedDwoFileCount`, and adding to `totalDwoFileCount` for all CU
units.
## Expected Behavior
- When binaries are compiled with split-dwarf and separate DWO files,
`totalLoadedDwoFileCount` would be the number of loaded DWO files and
`totalDwoFileCount` would be the total count of DWO files.
- When using a DWP file instead of separate DWO files,
`totalLoadedDwoFileCount` would be the number of parsed compile units,
while `totalDwoFileCount` would be the total number of CUs in the DWP
file. This should be similar to the counts we get from loading separate
DWO files rather than only counting whether a single DWP file was
loaded.
- When not using split-dwarf, we expect both `totalDwoFileCount` and
`totalLoadedDwoFileCount` to be 0 since no separate debug info is
loaded.
## Testing
**Manual Testing**
On an internal script that has many DWO files, `statistics dump` was
called before and after a `type lookup` command. The
`totalLoadedDwoFileCount` increased as expected after the `type lookup`.
```
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 29,
}
(lldb) type lookup folly::Optional<unsigned int>::Storage
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
...
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 2160,
}
```
**Unit test**
Added three unit tests that build with new "third.cpp" and "baz.cpp"
files. For tests with w/ flags `-gsplit-dwarf -gpubnames`, this
generates 2 DWO files. Then, the test incrementally adds breakpoints,
and does a type lookup, and the count should increase for each of these
as new DWO files get loaded to support these.
```
$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 20 tests in 211.738s
OK (skipped=3)
```
Currently if cmake_build_type is None we error with `AttributeError:
'NoneType' object has no attribute 'lower'` if the decorator
skipIfBuildType is used. This fixes the issue by first checking that
cmake_build_type is not None.
Replace `isTestSupported` function with `skipIfBuildType` annotation.
Test that uses the `IsTestSupported` function are no longer run, as the
size of lldb-dap binary is now more than `1mb`.
Update the broken test.
Fixes#108621
We could probably check if the test now passes on `linux arm` since it
was disabled because it timed out. I experienced the timeout after
replacing the `IsTestSupported` with `skipIfBuildType`.
Patch fixes the sync-on-timeout logic in lldb and switches to qEcho
based ping, instead of qC. This fixes vRun message case, when there is
no process yet and qC returns an error.
In DebugCommunication, we currently are using 2 thread to drive
lldb-dap. At the moment, they make an attempt at only synchronizing the
`recv_packets` between the reader thread and the main test thread. Other
stateful properties of the debug session are not guarded by a
locks/mutex.
To mitigate this, I am moving any state updates to the main thread
inside the `_recv_packet` method to ensure that between calls to
`_recv_packet` the state does not change out from under us in a test.
This does mean the precise timing of events has changed slightly as a
result and I've updated the existing tests that fail for me locally with
this new behavior.
I think this should result in overall more predictable behavior, even if
the test is slow due to the host workload or architecture differences.
---------
Co-authored-by: Ebuka Ezike <yerimyah1@gmail.com>
This adds new types for setExceptionBreakpoints and adds support for
`supportsExceptionFilterOptions`, which allows exception breakpoints to
set a condition.
While testing this, I noticed that obj-c exception catch breakpoints may
not be working correctly in lldb-dap.
This test was incorrectly disabled and bitrotted since then. This PR
fixes up the test and re-enables it.
- Build against the system libc++ (which can target the simulator)
- Bump the deployment target for iOS and tvOS on Apple Silicon
- Skip backdeploying to pre-Apple Silicon OS on Apple Silicon.
This adds a new 'CapabilitiesEventBody' type for having a well
structured type for the event and updates the 'restart' request
to dynamically set their capabilities.
Moving `threads` request to structured types. Adding helper types for
this and moving helpers from JSONUtils to ProtocolUtils.
---------
Co-authored-by: Ebuka Ezike <yerimyah1@gmail.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This reverts commit 4b6c608615a285d81132acf8e33b81b2ec2c9bf9 and
follow-up commits 159de3633640a5cb2d322ebe8cc4ec0c1c9a896d and
c9e1c52e2e75a91a44a98df818cc9bd11655e51d because this breaks
TestDAP_stepInTargets.py on Darwin.
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake
Attempt to improve tests by synchronously waiting for breakpoints to
resolve. Not sure if it will fix all the tests but I think it should
make the tests more stable
This reverts commit 8a49db35f45e56c92522c6079e51553e80c07aec.
Due to failures on Arm and AArch64 Linux:
https://lab.llvm.org/buildbot/#/builders/59/builds/18540https://lab.llvm.org/buildbot/#/builders/18/builds/16759
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 22, in assertEvaluateFailure
self.assertNotIn(
AssertionError: 'result' unexpectedly found in {'memoryReference': '0xFFFFF7CB3060', 'result': '0x0000000000000000', 'type': 'int *', 'variablesReference': 7}
FAIL: test_generic_evaluate_expressions (TestDAP_evaluate.TestDAP_evaluate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 228, in test_generic_evaluate_expressions
self.run_test_evaluate_expressions(enableAutoVariableSummaries=False)
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 117, in run_test_evaluate_expressions
self.assertEvaluateFailure("list") # local variable of a_function
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 22, in assertEvaluateFailure
self.assertNotIn(
AssertionError: 'result' unexpectedly found in {'memoryReference': '0xFFFFF7CB3060', 'result': '0x0000000000000000', 'type': 'int *', 'variablesReference': 7}
Config=aarch64-/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang
The second one is because our bots have the libc debug info package installed,
the first, no idea.
Improving the readability and correctness of DebugCommunication by
adding type annotations to many parts of the library and trying to
improve the implementation of a few key areas of the code to better
handle correctness.
Specifically, this refactored the
`DebugCommunication._handle_recv_packet` function to ensure consistency
with the reader thread when handling state changes and improved the
`DebugCommunication._recv_packet` helper to make it easier to follow by
adding some additional helpers.
#140107 changed the default argument of `disableASLR` of related
functions from `False` to `True`. libc++ CI has been stably failing for
`TestDAP_subtleFrames.py` (in bootstrapping-build) since then. The error
message "personality set failed: Operation not permitted" seems related
to ASLR.
This PR attempts to fix the CI failure by changing the default value of
`disableASLR` in `dap_server.py` to `False`.
In lldb-dap, we have existing tests that are known to be unstable when
lldb and lldb-dap are built in the Debug configuration.
This decorator lets us skip those tests in CI jobs that are to slow with
those configurations.
This was split out from #140777 to make the patches smaller.
