This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
A previous patch added the ability for us to tell if types were forcefully completed. This patch adds the ability to see which modules have forcefully completed types and aggregates the number of modules with forcefully completed types at the root level.
We add a module specific setting named "debugInfoHadIncompleteTypes" that is a boolean value. We also aggregate the number of modules at the root level that had incomplete debug info with a key named "totalModuleCountWithIncompleteTypes" that is a count of number of modules that had incomplete types.
Differential Revision: https://reviews.llvm.org/D138638
We have a statistic on each module named "debugInfoHadVariableErrors" which tracks when we have debug info, but an error prevented the variables from being displayed. This patch adds a new top level statistic named "totalModuleCountWithVariableErrors" which is a count of the modules that have "debugInfoHadVariableErrors" set to true.
Differential Revision: https://reviews.llvm.org/D138383
Now that we display an error when users try to get variables, but something in the debug info is preventing variables from showing up, track this with a new bool in each module's statistic information named "debugInfoHadVariableErrors".
This patch modifies the code to track when we have variable errors in a module and adds accessors to get/set this value. This value is used in the module statistics and we added a test to verify this value gets set correctly.
Differential Revision: https://reviews.llvm.org/D134508
Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.
Differential Revision: https://reviews.llvm.org/D128077
This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list.
So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files.
Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress.
For a module with a dSYM file, we can see the "symbolFilePath" is included:
```
"modules": [
{
"debugInfoByteSize": 1070,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 4873280600,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out",
"symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out",
"symbolTableIndexTime": 7.9999999999999996e-06,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 7.8999999999999996e-05,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx12.0.0",
"uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3"
},
```
And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats:
```
"modules": [
{
"debugInfoByteSize": 0,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 0,
"identifier": 4603526968,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out",
"symbolFileModuleIdentifiers": [
4604429832
],
"symbolTableIndexTime": 7.9999999999999996e-06,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0.000112,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx12.0.0",
"uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569"
},
```
And the .o file for 4604429832 looks like:
```
{
"debugInfoByteSize": 1028,
"debugInfoIndexLoadedFromCache": false,
"debugInfoIndexSavedToCache": false,
"debugInfoIndexTime": 0,
"debugInfoParseTime": 6.0999999999999999e-05,
"identifier": 4604429832,
"path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o",
"symbolTableIndexTime": 0,
"symbolTableLoadedFromCache": false,
"symbolTableParseTime": 0,
"symbolTableSavedToCache": false,
"triple": "arm64-apple-macosx"
}
```
Differential Revision: https://reviews.llvm.org/D119400
Add statistics about the memory usage of the string pool. I'm
particularly interested in the memory used by the allocator, i.e. the
number of bytes actually used by the allocator it self as well as the
number of bytes allocated through the allocator.
Differential revision: https://reviews.llvm.org/D117914
This patch add the ability to cache the manual DWARF indexing results to disk for faster subsequent debug sessions. Manual DWARF indexing is time consuming and causes all DWARF to be fully parsed and indexed each time you debug a binary that doesn't have an acceptable accelerator table. Acceptable accelerator tables include .debug_names in DWARF5 or Apple accelerator tables.
This patch breaks up testing by testing all of the encoding and decoding of required C++ objects in a gtest unit test, and then has a test to verify the debug info cache is generated correctly.
This patch also adds the ability to track when a symbol table or DWARF index is loaded or saved to the cache in the "statistics dump" command. This is essential to know in statistics as it can help explain why a debug session was slower or faster than expected.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115951
It is great to know how many times the target has stopped over its lifetime as each time the target stops, and possibly resumes without the user seeing it for things like shared library loading and signals that are not notified and auto continued, to help explain why a debug session might be slow. This is now included as "stopCount" inside each target JSON.
Differential Revision: https://reviews.llvm.org/D113810
This patch adds breakpoints to each target's statistics so we can track how long it takes to resolve each breakpoint. It also includes the structured data for each breakpoint so the exact breakpoint details are logged to allow for reproduction of slow resolving breakpoints. Each target gets a new "breakpoints" array that contains breakpoint details. Each breakpoint has "details" which is the JSON representation of a serialized breakpoint resolver and filter, "id" which is the breakpoint ID, and "resolveTime" which is the time in seconds it took to resolve the breakpoint. A snippet of the new data is shown here:
"targets": [
{
"breakpoints": [
{
"details": {...},
"id": 1,
"resolveTime": 0.00039291599999999999
},
{
"details": {...},
"id": 2,
"resolveTime": 0.00022679199999999999
}
],
"totalBreakpointResolveTime": 0.00061970799999999996
}
]
This provides full details on exactly how breakpoints were set and how long it took to resolve them.
Differential Revision: https://reviews.llvm.org/D112587
The new key/value pairs that are added to each module's stats are:
"debugInfoByteSize": The size in bytes of debug info for each module.
"debugInfoIndexTime": The time in seconds that it took to index the debug info.
"debugInfoParseTime": The time in seconds that debug info had to be parsed.
At the top level we add up all of the debug info size, parse time and index time with the following keys:
"totalDebugInfoByteSize": The size in bytes of all debug info in all modules.
"totalDebugInfoIndexTime": The time in seconds that it took to index all debug info if it was indexed for all modules.
"totalDebugInfoParseTime": The time in seconds that debug info was parsed for all modules.
Differential Revision: https://reviews.llvm.org/D112501
The new module stats adds the ability to measure the time it takes to parse and index the symbol tables for each module, and reports modules statistics in the output of "statistics dump" along with the path, UUID and triple of the module. The time it takes to parse and index the symbol tables are also aggregated into new top level key/value pairs at the target level.
Differential Revision: https://reviews.llvm.org/D112279
This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804.
This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like:
(lldb) statistics dump
{
"expressionEvaluation": {
"failures": 0,
"successes": 0
},
"firstStopTime": 0.34164492800000001,
"frameVariable": {
"failures": 0,
"successes": 0
},
"launchOrAttachTime": 0.31969605400000001,
"targetCreateTime": 0.0040863039999999998
}
The top level keys are:
"expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts.
"frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts.
"targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled.
"launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs.
"firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test.
This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class.
Differential Revision: https://reviews.llvm.org/D111686
Summary:
LLDB keeps statistics of how many expression evaluations are 'successful' and 'failed'
which are updated after each expression evaluation (assuming statistics are enabled).
From what I understand the idea is that this could be used to define how well LLDB's
expression evaluator is working.
Currently all expressions are considered successful unless the user passes an explicit
positive element counting to the expression command (with the `-Z` flag) and then passes
an expression that successfully evaluates to a type that doesn't support element counting.
Expressions that fail to parse, execute or any other outcome are considered successful
at the moment which means we nearly always have a 100% expression evaluation
success rate.
This patch makes that expressions that fail to parse or execute to count as failed
expressions.
We can't know whether the expression failed because of an user error
of because LLDB couldn't correctly parse/compile it, but I would argue that this is
still an improvement. Assuming that the percentage of valid user expressions stays
mostly constant over time (which seems like a reasonable assumption), then this
way we can still see if we are doing relatively better/worse from release to release.
Reviewers: davide, aprantl, JDevlieghere
Reviewed By: aprantl
Subscribers: abidh
Differential Revision: https://reviews.llvm.org/D76280
There is still the bug that empty lines seem to skip any following expressions
and it makes it harder to commend between all the comments. Let's make this
a normal test instead which is just slightly more verbose but can be properly
formatted.
Summary: Moves lldbsuite tests to lldb/test/API.
This is a largely mechanical change, moved with the following steps:
```
rm lldb/test/API/testcases
mkdir -p lldb/test/API/{test_runner/test,tools/lldb-{server,vscode}}
mv lldb/packages/Python/lldbsuite/test/test_runner/test lldb/test/API/test_runner
for d in $(find lldb/packages/Python/lldbsuite/test/* -maxdepth 0 -type d | egrep -v "make|plugins|test_runner|tools"); do mv $d lldb/test/API; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-vscode -maxdepth 1 -mindepth 1 | grep -v ".py"); do mv $d lldb/test/API/tools/lldb-vscode; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-server -maxdepth 1 -mindepth 1 | egrep -v "gdbremote_testcase.py|lldbgdbserverutils.py|socket_packet_pump.py"); do mv $d lldb/test/API/tools/lldb-server; done
```
lldb/packages/Python/lldbsuite/__init__.py and lldb/test/API/lit.cfg.py were also updated with the new directory structure.
Reviewers: labath, JDevlieghere
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71151
