This updates all the existing memory reference fields to use
`lldb::addr_t` directly.
A few places were using `std::string` and decoding the value in the
request handler and other places had unique ways of parsing addresses.
This unifies all of these references with the `DecodeMemoryReference`
helper in JSONUtils.h.
Additionally, for the types I updated, I tried to simplify the POD types
some and moved default values out of RequestHandlers and into the
protocol POD types.
This PR adds type summaries for
`std::{string,wstring,u8string,u16string,u32string}` from the MSVC STL.
See https://github.com/llvm/llvm-project/issues/24834 for the MSVC STL
issue.
The following changes were made:
- `dotest.py` now detects if the MSVC STL is available. It does so by
looking at the target triple, which is an additional argument passed
from Lit. It specifically checks for `windows-msvc` to not match on
`windows-gnu` (i.e. MinGW/Cygwin).
- (The main part): Added support for summarizing `std::(w)string` from
MSVC's STL. Because the type names from the libstdc++ (pre C++ 11)
string types are the same as on MSVC's STL, `CXXCompositeSummaryFormat`
is used with two entries, one for MSVC's STL and one for libstdc++.
With MSVC's STL, `std::u{8,16,32}string` is also handled. These aren't
handled for libstdc++, so I put them in `LoadMsvcStlFormatters`.
When there is a function that is inlined at the current program counter.
If you get the current `line_entry` using the program counter's address
it will point to the location of the inline function that may be in
another file. (this is in implicit step-in and should not happen what
step over is called).
Use the current frame to get the `line_entry`
# Benefit
This patch fixes:
1. After `platform select ios-simulator`, `platform process list` will
now print processes which are running in the iOS simulator. Previously,
no process will be listed.
2. After `platform select ios-simulator`, `platform attach --name
<name>` will succeed. Previously, it will error out saying no process is
found.
# Several bugs that is being fixed
1. During the process listing, add `aarch64` to the list of CPU types
for which iOS simulators are checked for.
2. Given a candidate process, when checking for simulators, the original
code will find the desired environment variable (`SIMULATOR_UDID`) and
set the OS to iOS, but then the immediate next environment variable will
set it back to macOS.
3. For processes running on simulator, set the triple's `Environment` to
`Simulator`, so that such processes can pass the filtering [in this
line](https://fburl.com/8nivnrjx). The original code leave it as the
default `UnknownEnvironment`.
# Manual test
**With this patch:**
```
royshi-mac-home ~/public_llvm/build % bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
240 matching processes were found on "ios-simulator"
PID PARENT USER TRIPLE NAME
====== ====== ========== ============================== ============================
40511 28844 royshi arm64-apple-ios-simulator FocusPlayground // my toy iOS app running on simulator
... // omit
28844 1 royshi arm64-apple-ios-simulator launchd_sim
(lldb) process attach --name FocusPlayground
Process 40511 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x0000000104e3cb70 libsystem_kernel.dylib`mach_msg2_trap + 8
libsystem_kernel.dylib`mach_msg2_trap:
-> 0x104e3cb70 <+8>: ret
... // omit
```
**Without this patch:**
```
$ bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
error: no processes were found on the "ios-simulator" platform
(lldb) process attach --name FocusPlayground
error: attach failed: could not find a process named FocusPlayground
```
# Unittest
See PR.
This fixes issues found in e066f35c6981c720e3a7e5883efc40c861b3b7, which
was later reverted. The problem was with "k" message which was sent with
sync_on_timeout flag set to true, so lldb was waiting for response,
which is currently not being sent by lldb-server. Not waiting for
response at all seems to be not a solution, because on MAC OS X lldb
waits for response from "k" to gracefully kill inferior.
This relands changes in #144424 for adding a count of DWO files
parsed/loaded and the total number of DWO files.
The previous PR was reverted in #145494 due to the newly added unit
tests failing on Windows and MacOS CIs since these platforms don't
support DWO. This change add an additional
`@add_test_categories(["dwo"])` to the new tests to
[skip](cd46354dbd/lldb/packages/Python/lldbsuite/test/test_categories.py (L56))
these tests on Windows/MacOS.
Original PR: #144424
### Testing
Ran unit tests
```
$ bin/lldb-dotest -p TestStats.py llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 24 tests in 211.391s
OK (skipped=3)
```
## Summary
A new `totalLoadedDwoFileCount` and `totalDwoFileCount` counters to
available statisctics when calling "statistics dump".
1. `GetDwoFileCounts ` is created, and returns a pair of ints
representing the number of loaded DWO files and the total number of DWO
files, respectively. An override is implemented for `SymbolFileDWARF`
that loops through each compile unit, and adds to a counter if it's a
DWO unit, and then uses `GetDwoSymbolFile(false)` to check whether the
DWO file was already loaded/parsed.
3. In `Statistics`, use `GetSeparateDebugInfo` to sum up the total
number of loaded/parsed DWO files along with the total number of DWO
files. This is done by checking whether the DWO file was already
successfully `loaded` in the collected DWO data, anding adding to the
`totalLoadedDwoFileCount`, and adding to `totalDwoFileCount` for all CU
units.
## Expected Behavior
- When binaries are compiled with split-dwarf and separate DWO files,
`totalLoadedDwoFileCount` would be the number of loaded DWO files and
`totalDwoFileCount` would be the total count of DWO files.
- When using a DWP file instead of separate DWO files,
`totalLoadedDwoFileCount` would be the number of parsed compile units,
while `totalDwoFileCount` would be the total number of CUs in the DWP
file. This should be similar to the counts we get from loading separate
DWO files rather than only counting whether a single DWP file was
loaded.
- When not using split-dwarf, we expect both `totalDwoFileCount` and
`totalLoadedDwoFileCount` to be 0 since no separate debug info is
loaded.
## Testing
**Manual Testing**
On an internal script that has many DWO files, `statistics dump` was
called before and after a `type lookup` command. The
`totalLoadedDwoFileCount` increased as expected after the `type lookup`.
```
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 29,
}
(lldb) type lookup folly::Optional<unsigned int>::Storage
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
typedef std::conditional<true, folly::Optional<unsigned int>::StorageTriviallyDestructible, folly::Optional<unsigned int>::StorageNonTriviallyDestructible>::type
...
(lldb) statistics dump
{
...
"totalLoadedDwoFileCount": 2160,
}
```
**Unit test**
Added three unit tests that build with new "third.cpp" and "baz.cpp"
files. For tests with w/ flags `-gsplit-dwarf -gpubnames`, this
generates 2 DWO files. Then, the test incrementally adds breakpoints,
and does a type lookup, and the count should increase for each of these
as new DWO files get loaded to support these.
```
$ bin/lldb-dotest -p TestStats.py ~/llvm-sand/external/llvm-project/lldb/test/API/commands/statistics/basic/
----------------------------------------------------------------------
Ran 20 tests in 211.738s
OK (skipped=3)
```
Currently if cmake_build_type is None we error with `AttributeError:
'NoneType' object has no attribute 'lower'` if the decorator
skipIfBuildType is used. This fixes the issue by first checking that
cmake_build_type is not None.
Replace `isTestSupported` function with `skipIfBuildType` annotation.
Test that uses the `IsTestSupported` function are no longer run, as the
size of lldb-dap binary is now more than `1mb`.
Update the broken test.
Fixes#108621
We could probably check if the test now passes on `linux arm` since it
was disabled because it timed out. I experienced the timeout after
replacing the `IsTestSupported` with `skipIfBuildType`.
Patch fixes the sync-on-timeout logic in lldb and switches to qEcho
based ping, instead of qC. This fixes vRun message case, when there is
no process yet and qC returns an error.
In DebugCommunication, we currently are using 2 thread to drive
lldb-dap. At the moment, they make an attempt at only synchronizing the
`recv_packets` between the reader thread and the main test thread. Other
stateful properties of the debug session are not guarded by a
locks/mutex.
To mitigate this, I am moving any state updates to the main thread
inside the `_recv_packet` method to ensure that between calls to
`_recv_packet` the state does not change out from under us in a test.
This does mean the precise timing of events has changed slightly as a
result and I've updated the existing tests that fail for me locally with
this new behavior.
I think this should result in overall more predictable behavior, even if
the test is slow due to the host workload or architecture differences.
---------
Co-authored-by: Ebuka Ezike <yerimyah1@gmail.com>
This adds new types for setExceptionBreakpoints and adds support for
`supportsExceptionFilterOptions`, which allows exception breakpoints to
set a condition.
While testing this, I noticed that obj-c exception catch breakpoints may
not be working correctly in lldb-dap.
This test was incorrectly disabled and bitrotted since then. This PR
fixes up the test and re-enables it.
- Build against the system libc++ (which can target the simulator)
- Bump the deployment target for iOS and tvOS on Apple Silicon
- Skip backdeploying to pre-Apple Silicon OS on Apple Silicon.
This adds a new 'CapabilitiesEventBody' type for having a well
structured type for the event and updates the 'restart' request
to dynamically set their capabilities.
Moving `threads` request to structured types. Adding helper types for
this and moving helpers from JSONUtils to ProtocolUtils.
---------
Co-authored-by: Ebuka Ezike <yerimyah1@gmail.com>
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This reverts commit 4b6c608615a285d81132acf8e33b81b2ec2c9bf9 and
follow-up commits 159de3633640a5cb2d322ebe8cc4ec0c1c9a896d and
c9e1c52e2e75a91a44a98df818cc9bd11655e51d because this breaks
TestDAP_stepInTargets.py on Darwin.
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake
Attempt to improve tests by synchronously waiting for breakpoints to
resolve. Not sure if it will fix all the tests but I think it should
make the tests more stable
This reverts commit 8a49db35f45e56c92522c6079e51553e80c07aec.
Due to failures on Arm and AArch64 Linux:
https://lab.llvm.org/buildbot/#/builders/59/builds/18540https://lab.llvm.org/buildbot/#/builders/18/builds/16759
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 22, in assertEvaluateFailure
self.assertNotIn(
AssertionError: 'result' unexpectedly found in {'memoryReference': '0xFFFFF7CB3060', 'result': '0x0000000000000000', 'type': 'int *', 'variablesReference': 7}
FAIL: test_generic_evaluate_expressions (TestDAP_evaluate.TestDAP_evaluate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 228, in test_generic_evaluate_expressions
self.run_test_evaluate_expressions(enableAutoVariableSummaries=False)
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 117, in run_test_evaluate_expressions
self.assertEvaluateFailure("list") # local variable of a_function
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/tools/lldb-dap/evaluate/TestDAP_evaluate.py", line 22, in assertEvaluateFailure
self.assertNotIn(
AssertionError: 'result' unexpectedly found in {'memoryReference': '0xFFFFF7CB3060', 'result': '0x0000000000000000', 'type': 'int *', 'variablesReference': 7}
Config=aarch64-/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang
The second one is because our bots have the libc debug info package installed,
the first, no idea.
Improving the readability and correctness of DebugCommunication by
adding type annotations to many parts of the library and trying to
improve the implementation of a few key areas of the code to better
handle correctness.
Specifically, this refactored the
`DebugCommunication._handle_recv_packet` function to ensure consistency
with the reader thread when handling state changes and improved the
`DebugCommunication._recv_packet` helper to make it easier to follow by
adding some additional helpers.
#140107 changed the default argument of `disableASLR` of related
functions from `False` to `True`. libc++ CI has been stably failing for
`TestDAP_subtleFrames.py` (in bootstrapping-build) since then. The error
message "personality set failed: Operation not permitted" seems related
to ASLR.
This PR attempts to fix the CI failure by changing the default value of
`disableASLR` in `dap_server.py` to `False`.
In lldb-dap, we have existing tests that are known to be unstable when
lldb and lldb-dap are built in the Debug configuration.
This decorator lets us skip those tests in CI jobs that are to slow with
those configurations.
This was split out from #140777 to make the patches smaller.
Fix the handling of the `instructionOffset` parameter, which resulted in
always returning the wrong disassembly because VSCode always uses
`instructionOffset = -50` and expects 50 instructions before the given
address, instead of 50 bytes before
This is more straight forward refactor of the startup sequence that
reverts parts of ba29e60f9a2222bd5e883579bb78db13fc5a7588. Unlike my
previous attempt, I ended up removing the pending request queue and not
including an `AsyncReqeustHandler` because I don't think we actually
need that at the moment.
The key is that during the startup flow there are 2 parallel operations
happening in the DAP that have different triggers.
* The `initialize` request is sent and once the response is received the
`launch` or `attach` is sent.
* When the `initialized` event is recieved the `setBreakpionts` and
other config requests are made followed by the `configurationDone`
event.
I moved the `initialized` event back to happen in the `PostRun` of the
`launch` or `attach` request handlers. This ensures that we have a valid
target by the time the configuration calls are made. I added also added
a few extra validations that to the `configurationeDone` handler to
ensure we're in an expected state.
I've also fixed up the tests to match the new flow. With the other
additional test fixes in 087a5d2ec7897cd99d3787820711fec76a8e1792 I
think we've narrowed down the main source of test instability that
motivated the startup sequence change.
Adding an assert that the 'continue' request succeeds caused a number of
tests to fail. This showed a number of tests that were not specifying if
they should be stopped or not at key points in the test. This is likely
contributing to these tests being flaky since the debugger is not in the
expected state.
Additionally, I spent a little time trying to improve the readability of
the dap_server.py and lldbdap_testcase.py.
To improve logging this adjusts two properties of the existing tests:
* Forwards stderr from lldb-dap to the process in case errors are
reported to stderr.
* Adjusts `DebugAdapterServer.terminate` to close stdin and wait for the
process to exit instead of sending SIGTERM. Additionally, if we end up
with a non-zero exit status we now raise an error to note the unexpected
exit status.
With these changes, I did find one test case in
`TestDAP_console.test_diagnositcs` that was not waiting to ensure the
expected event had arrived by the time it performed an assert.
test_common is force-included into every compilation, which causes
problems when we're compiling assembly code, as we were in #138805.
This avoids that as we can include the header only when it's needed.
This PR changes how we treat the launch sequence in lldb-dap.
- Send the initialized event after we finish handling the initialize
request, rather than after we finish attaching or launching.
- Delay handling the launch and attach request until we have handled
the configurationDone request. The latter is now largely a NO-OP and
only exists to signal lldb-dap that it can handle the launch and
attach requests.
- Delay handling the initial threads requests until we have handled
the launch or attach request.
- Make all attaching and launching synchronous, including when we have
attach or launch commands. This removes the need to synchronize
between the request and event thread.
Background:
https://discourse.llvm.org/t/reliability-of-the-lldb-dap-tests/86125
Make stopOnAttach=False the default again and explicitly pass
stopOnAttach=True where the tests relies on that. I changed the default
in the launch sequence PR (#138219) because that was implicitly the
assumption (the tests never send the configurationDone request).
This PR changes how we treat the launch sequence in lldb-dap.
- Send the initialized event after we finish handling the initialize
request, rather than after we finish attaching or launching.
- Delay handling the launch and attach request until we have handled
the configurationDone request. The latter is now largely a NO-OP and
only exists to signal lldb-dap that it can handle the launch and
attach requests.
- Delay handling the initial threads requests until we have handled
the launch or attach request.
- Make all attaching and launching synchronous, including when we have
attach or launch commands. This removes the need to synchronize
between the request and event thread.
Background:
https://discourse.llvm.org/t/reliability-of-the-lldb-dap-tests/86125
# Summary
This patch makes the `request_attach` wait for events `process` and
`initialized` just like `request_launch`. This ensure the DAP session
can move forward somewhat correctly.
Recently `TestDap_attach.test_terminate_commands` became flaky.
It's hitting:
```
lldbsuite/test/tools/lldb-dap/dap_server.py", line 350, in send_recv
raise ValueError(desc)
ValueError: no response for "disconnect"
```
I took a look at the DAP msg from that test case and noticed:
- It's not using the regular attaching, instead it's using the
`attachCommands` to launch debug the binary and it will stop at entry.
- The `initialized` event returned after the `disconnect` request. Which
means lldb-dap didn't really get ready yet.
### NOTE
The `dap_server.py` is doing things to mimic the VSCode (or other dap
clients) but it had some assumptions. For example, it's still missing
the `configurationDone` request and response because it relies on a
continue action to trigger the `configurationDone` request.
# Test Plan
```
./bin/llvm-lit -va /Users/wanyi/llvm-upstream/llvm-project/lldb/test/API/tools/lldb-dap/attach/TestDAP_attach.py
./bin/llvm-lit -va /Users/wanyi/llvm-upstream/llvm-project/lldb/test/API/tools/lldb-dap/launch/TestDAP_launch.py
```
To test the `wait_for_events` timeout case
```
events = self.wait_for_events(["process", "initialized", "fake", "event"], 1)
if events:
raise ValueError(f'no events {",".join(events)} found for within timeout 1')
```
Observed
<img width="696" alt="image"
src="https://github.com/user-attachments/assets/bc97c0ef-d91f-4561-8272-4d36f5f5d4e6"
/>
### Also
Looks like some test cases should be re-enabled in
0b8dfb5762
But only comments was removed. The skip statements survived the change.
The module event indicates that some information about a module has
changed. The event is supported by the Emacs and Visual Studio DAP
clients. This PR adds support for emitting the event from lldb-dap.
Fixes#137058
