1194 Commits

Author SHA1 Message Date
Johnny Chen
fc9e79fb95 Benchmark the turnaround time starting a debugger and run to the breakpoint with lldb vs. gdb.
An example (with /Developer/usr/bin/lldb vs. /usr/bin/gdb):

[13:05:04] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v +b -n -p TestCompileRunToBreakpointTurnaround.py
1: test_run_lldb_then_gdb (TestCompileRunToBreakpointTurnaround.CompileRunToBreakpointBench)
   Benchmark turnaround time with lldb vs. gdb. ... 
lldb turnaround benchmark: Avg: 4.574600 (Laps: 3, Total Elapsed Time: 13.723799)
gdb turnaround benchmark: Avg: 7.966713 (Laps: 3, Total Elapsed Time: 23.900139)
lldb_avg/gdb_avg: 0.574214
ok

----------------------------------------------------------------------
Ran 1 test in 55.462s

OK

llvm-svn: 142949
2011-10-25 20:08:03 +00:00
Johnny Chen
447a2507a4 Add a bench-history file to keep track of simple bench results.
llvm-svn: 142874
2011-10-24 23:18:17 +00:00
Johnny Chen
aeab25c70f Add more context information to the stop-hook mechanism by displaying the stop-hook
command in the '- Hook id' header.  This should improve readbility of the 'display'
command if, for example, we have issued 'display a' and 'display b' which turn into
"target stop-hook add -o 'expr -- a'" and "target stop-hook add -o 'expr -- b'".

Plus some minor change in TestAbbreviations.py to conditionalize the platform-specific
checkings of the "image list" output.

llvm-svn: 142868
2011-10-24 23:01:06 +00:00
Johnny Chen
51a4655201 Minor change.
llvm-svn: 142858
2011-10-24 22:06:48 +00:00
Johnny Chen
0deafe065a Test some lldb command abbreviations to make sure the common short spellings of
many commands remain available even after we add/delte commands in the future.

llvm-svn: 142857
2011-10-24 22:03:01 +00:00
Johnny Chen
7149b6f115 Add two new @expectedFailure decorators.
rdar://problem/10334911

llvm-svn: 142839
2011-10-24 20:06:23 +00:00
Jim Ingham
7e18e42235 Add "di" and "dis" aliases to "disassemble" so they will win over "display".
llvm-svn: 142834
2011-10-24 18:37:00 +00:00
Johnny Chen
9593622b08 Fix test regressions due to the addition of 'display' alias to the top level commands, which conflicts
with the original 'dis' -> 'disassemble' unique expansion.  Change it to 'disass' now.

llvm-svn: 142825
2011-10-24 18:24:54 +00:00
Johnny Chen
e5b190304d Removed the @expectedFailure decorators from test cases. They have been fixed with the r142717 check-in.
llvm-svn: 142823
2011-10-24 18:11:16 +00:00
Johnny Chen
b8da426285 Add bench.py as a driver script to run some benchmarks on lldb.
Add benchmarks for expression evaluations (TestExpressionCmd.py) and disassembly (TestDoAttachThenDisassembly.py).

An example:
[17:45:55] johnny:/Volumes/data/lldb/svn/trunk/test $ ./bench.py 2>&1 | grep -P '^lldb.*benchmark:'
lldb startup delay (create fresh target) benchmark: Avg: 0.104274 (Laps: 30, Total Elapsed Time: 3.128214)
lldb startup delay (set first breakpoint) benchmark: Avg: 0.102216 (Laps: 30, Total Elapsed Time: 3.066470)
lldb frame variable benchmark: Avg: 1.649162 (Laps: 20, Total Elapsed Time: 32.983245)
lldb stepping benchmark: Avg: 0.104409 (Laps: 50, Total Elapsed Time: 5.220461)
lldb expr cmd benchmark: Avg: 0.206774 (Laps: 25, Total Elapsed Time: 5.169350)
lldb disassembly benchmark: Avg: 0.089086 (Laps: 10, Total Elapsed Time: 0.890859)

llvm-svn: 142708
2011-10-22 00:57:05 +00:00
Johnny Chen
31fdfb1f33 Add a benchmark for measuring the response time of the 'frame variable' command.
Example (start the lldb inferior, break at the Driver::MainLoop() function, and
issue 'frame variable'):

 $ ./dotest.py -v +b -x '-F Driver::MainLoop()' -n -p TestFrameVariableResponse.py
----------------------------------------------------------------------
Collected 1 test

1: test_startup_delay (TestFrameVariableResponse.FrameVariableResponseBench)
   Test response time for the 'frame variable' command. ... 
lldb frame variable benchmark: Avg: 1.636897 (Laps: 20, Total Elapsed Time: 32.737944)
ok

----------------------------------------------------------------------
Ran 1 test in 65.105s

OK

llvm-svn: 142678
2011-10-21 20:19:51 +00:00
Johnny Chen
99904b33b7 Rephrase benchmark output display.
llvm-svn: 142676
2011-10-21 20:11:40 +00:00
Johnny Chen
4d4363ba56 Add a '-n' option to turn off printings of build dir, lldb version, svn info, and other headers
which happen before the listingings of test cases.

llvm-svn: 142668
2011-10-21 18:33:27 +00:00
Johnny Chen
f7a0062869 Fix the compilation warning while running the test case.
llvm-svn: 142663
2011-10-21 17:57:04 +00:00
Johnny Chen
edb677c3c7 Fix wrong directory name.
llvm-svn: 142629
2011-10-21 01:09:29 +00:00
Johnny Chen
7f63fdaa74 Add a benchmark for measuring start up delays of lldb, including:
o create a fresh target; and
o set the first breakpoint

Example (using lldb to set a breakpoint on lldb's Driver::MainLoop function):

./dotest.py -v +b -x '-F Driver::MainLoop()' -p TestStartupDelays.py

...

1: test_startup_delay (TestStartupDelays.StartupDelaysBench)
   Test start up delays creating a target and setting a breakpoint. ... 
lldb startup delays benchmark:
create fresh target: Avg: 0.106732 (Laps: 15, Total Elapsed Time: 1.600985)
set first breakpoint: Avg: 0.102589 (Laps: 15, Total Elapsed Time: 1.538832)
ok

llvm-svn: 142628
2011-10-20 22:37:45 +00:00
Johnny Chen
ff7fc9cfa4 Breakpoint specification can have the form '-n main', so it's not a good idea to
check that the option arg in '-x opt_arg' does not start with a '-' char.

llvm-svn: 142625
2011-10-20 22:16:24 +00:00
Johnny Chen
8241d378f8 Directory renaming: example -> expression.
llvm-svn: 142602
2011-10-20 18:58:28 +00:00
Johnny Chen
f4ca4b5f57 Directory renaming: example -> expression.
llvm-svn: 142601
2011-10-20 18:57:04 +00:00
Johnny Chen
38f9daa303 Parameterize the iteration count used when running benchmarks, instead of hard-coded inside the test case.
Add a '-y count' option to the test driver for this purpose.  An example:

 $  ./dotest.py -v -y 25 +b -p TestDisassembly.py

...

----------------------------------------------------------------------
Collected 2 tests

1: test_run_gdb_then_lldb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
gdb benchmark: Avg: 0.226305 (Laps: 25, Total Elapsed Time: 5.657614)
lldb benchmark: Avg: 0.113864 (Laps: 25, Total Elapsed Time: 2.846606)
lldb_avg/gdb_avg: 0.503146
ok
2: test_run_lldb_then_gdb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
lldb benchmark: Avg: 0.113008 (Laps: 25, Total Elapsed Time: 2.825201)
gdb benchmark: Avg: 0.225240 (Laps: 25, Total Elapsed Time: 5.631001)
lldb_avg/gdb_avg: 0.501723
ok

----------------------------------------------------------------------
Ran 2 tests in 41.346s

OK

llvm-svn: 142598
2011-10-20 18:43:28 +00:00
Johnny Chen
5a328cc8c6 Remove stale code.
llvm-svn: 142595
2011-10-20 17:49:44 +00:00
Johnny Chen
29b3f6b616 Remove stale code.
llvm-svn: 142594
2011-10-20 17:45:39 +00:00
Johnny Chen
2a6eab04b1 Up until now, we have been using pexpect to spawn an lldb process and use lldb commands to
bring the debugger to the desired state.

This patch makes BenchBase inherit from TestBase, instead of Base (which is a parent class of
TestBase).  This is so that we can also enjoy the Pythonic way of bringing the lldb debugger
to a desired state before running the benchmark and collect statistics.

llvm-svn: 142562
2011-10-20 01:35:57 +00:00
Johnny Chen
c54de2ca54 Missed this file when doing r142543 to temporarily relax the expected substrings for watchpoint creation output.
llvm-svn: 142549
2011-10-19 22:17:06 +00:00
Johnny Chen
d72b178a45 Temporarily relax the expected substrings for watchpoint creation output
due a bug in the decl file info of a global variable emitted by clang.

llvm-svn: 142543
2011-10-19 20:51:28 +00:00
Johnny Chen
2a80858903 Modify lldbtest.Base.runHooks() to now take the following keyword arguments:
child=None, child_prompt=None, use_cmd_api=False

By default, expect a pexpect spawned child and child prompt to be
supplied (use_cmd_api=False).  If use_cmd_api is true, ignore the child
and child prompt and use self.runCmd() to run the hooks one by one.

Modify existing client to reflect the change.

llvm-svn: 142532
2011-10-19 16:48:07 +00:00
Johnny Chen
a737ba55af Extract the run hooks functionality into the base class lldbtest.Base.
llvm-svn: 142469
2011-10-19 01:06:21 +00:00
Johnny Chen
9df05592f0 Add test cases for setting condition on a watchpoint for both command and API.
llvm-svn: 142291
2011-10-17 22:17:47 +00:00
Enrico Granata
a9dbf4325e this patch introduces a new command script import command which takes as input a filename for a Python script and imports the module contained in that file. the containing directory is added to the Python path such that dependencies are honored. also, the module may contain an __lldb_init_module(debugger,dict) function, which gets called after importing, and which can somehow initialize the module's interaction with lldb
llvm-svn: 142283
2011-10-17 21:45:27 +00:00
Johnny Chen
9c7b04cb26 Rename test file.
llvm-svn: 142242
2011-10-17 20:28:39 +00:00
Johnny Chen
8289e36506 Fix wrong docstring.
llvm-svn: 142241
2011-10-17 20:26:07 +00:00
Johnny Chen
16dcf718d3 Add a commnad to set a condition for a watchpoint. Example:
watchpoint modify -c 'global==5'

modifies the last created watchpoint so that the condition expression
is evaluated at the stop point to decide whether we should proceed with
the stopping.

Also add SBWatchpont::SetCondition(const char *condition) to set condition
programmatically.

Test cases to come later.

llvm-svn: 142227
2011-10-17 18:58:00 +00:00
Johnny Chen
ed456eb0a9 Add SBWatchpoint::GetError() API, which is not currently populated as yet.
llvm-svn: 141979
2011-10-14 19:15:48 +00:00
Johnny Chen
6283f4b67a Remove debug 'print watchpoint' statement.
llvm-svn: 141965
2011-10-14 18:10:00 +00:00
Johnny Chen
fc807f84ed Rename directory command_python to command_script, and the test file, too.
llvm-svn: 141963
2011-10-14 17:37:38 +00:00
Johnny Chen
f5be9e3759 Add a test case to exercise the newly added SB API:
lldb::SBWatchpoint
SBTarget::WatchAddress (lldb::addr_t addr, size_t size, bool read, bool write)

llvm-svn: 141931
2011-10-14 01:16:39 +00:00
Johnny Chen
4c3e140247 Fix wrong docstring with respect to API usage.
llvm-svn: 141928
2011-10-14 00:56:28 +00:00
Johnny Chen
01a678603a SBValue::Watch() and SBValue::WatchPointee() are now the official API for creating
a watchpoint for either the variable encapsulated by SBValue (Watch) or the pointee
encapsulated by SBValue (WatchPointee).

Removed SBFrame::WatchValue() and SBFrame::WatchLocation() API as a result of that.

Modified the watchpoint related test suite to reflect the change.

Plus replacing WatchpointLocation with Watchpoint throughout the code base.

There are still cleanups to be dome.  This patch passes the whole test suite.
Check it in so that we aggressively catch regressions.

llvm-svn: 141925
2011-10-14 00:42:25 +00:00
Johnny Chen
04a101d475 Add cleanups for 'command script add' to TestCommandPython.py.
llvm-svn: 141790
2011-10-12 17:50:41 +00:00
Johnny Chen
8a7f3f5bac Add a more generic stepping benchmark, which uses the '-k' option of the test driver
to be able to specify the runhook(s) to bring the debug session to a certain state
before running the benchmarking logic.  An example,

./dotest.py -v -t +b -k 'process attach -n Mail' -k 'thread backtrace all' -p TestRunHooksThenSteppings.py

spawns lldb, attaches to the 'Mail' application, does a backtrace for all threads, and then
runs the benchmark to step the inferior multiple times.

llvm-svn: 141740
2011-10-11 23:22:02 +00:00
Jim Ingham
32fc9601a1 Added some simple testcases for figuring out breakpoints by name in
classes & namespaces.

llvm-svn: 141629
2011-10-11 01:43:50 +00:00
Johnny Chen
999f7ab9bd Add '-k' option to the test driver to be able to specify a runhook, which is an lldb command
for the debugger to execute for certain kind of tests (for example, a benchmark).
A list of runhooks can be used to steer the debugger into the desired state before more
actions can be performed.

llvm-svn: 141626
2011-10-11 01:30:27 +00:00
Johnny Chen
541b98517e Remove test logic to check for clang and skip the rest due to insufficient debug info.
Recent changes in lldb inlining robustness seem to have fixed it.

llvm-svn: 141595
2011-10-10 23:26:54 +00:00
Johnny Chen
a10e6c1052 Add '-e' and '-x' options to the test driver to be able to specify an executable (full path)
and the breakpoint specification for the benchmark purpose.  This is used by TestSteppingSpeed.py
to benchmark the lldb stepping speed.  Without '-e' and 'x' specified, the test defaults to
run the built lldb against itself and stopped on Driver::MainLoop, then stepping for 50 times.

rdar://problem/7511193

llvm-svn: 141584
2011-10-10 22:03:44 +00:00
Jim Ingham
eb6ba39033 Fix the last testsuite regression from the apple-names stuff.
llvm-svn: 141468
2011-10-08 01:11:42 +00:00
Johnny Chen
a4e8baeb33 Add fuzz call for SBBreakpointLocation.GetAddress().
llvm-svn: 141443
2011-10-07 23:54:04 +00:00
Jim Ingham
4cda6e058b Move the responsibility for translating the various eFunctionNameType lookups to the
SymbolFIle (it was done mostly in the BreakpointResolverName resolver before.)  Then
tailor our searches to the way the indexed maps are laid out.  This removes a bunch 
of test case failures using indexed dSYM's.

llvm-svn: 141428
2011-10-07 22:23:45 +00:00
Johnny Chen
ebe51726b8 If we spawn an lldb process for test (via pexpect), do not load the init file unless told otherwise.
Set up self.lldbOption to be "--no-lldbibit" unless env variable NO_LLDBIBIT is defined and equals "NO".
Also add "-nx" to gdb spawned.

llvm-svn: 141384
2011-10-07 19:21:09 +00:00
Jim Ingham
de4b919d27 Turn the EvaluateExpression (..., True) tests back on.
llvm-svn: 141375
2011-10-07 18:02:54 +00:00
Johnny Chen
4f09a0f855 Add a test sequence for 'target variable' command when no arguments are given.
llvm-svn: 141264
2011-10-06 01:00:53 +00:00