built locally from the source tree. This is distinguished from self.lldbExec, which
can be used by test/benchmarks to measure the performances against other debuggers.
You can use environment variable LLDB_EXEC to specify self.lldbExec to the dotest.py
test driver, otherwise it is going to be populated with self.lldbHere.
Modify the regular tests under test dir, i.e., not test/benchmarks, to use self.lldbHere.
Also modify the benchmarks tests to use self.lldbHere when it needs an 'lldb' executable
with debug info to do the performance measurements.
llvm-svn: 138608
on lldb's Driver::MainLoop function which is ~1190 lines of x86 assembly code. This file is not
exercised during the normal test suite run, i.e., no +b option specified. So it should be ok.
The following is the benchmark result on my MBP running OSX Lion:
[17:38:46] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v +b -p TestFlintVsSlate
/Volumes/data/lldb/svn/trunk/build/Debug
LLDB-71
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 137008
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 137008
Last Changed Date: 2011-08-05 17:50:36 -0700 (Fri, 05 Aug 2011)
Session logs for test failures/errors/unexpected successes will go into directory '2011-08-08-17_38_52'
Command invoked: python ./dotest.py -v +b -p TestFlintVsSlate
----------------------------------------------------------------------
Collected 2 tests
1: test_run_41_then_42 (TestFlintVsSlateGDBDisassembly.FlintVsSlateGDBDisassembly)
Test disassembly on a large function with 4.1 vs. 4.2's gdb. ...
4.1 gdb benchmark: Avg: 0.205623 (Laps: 5, Total Elapsed Time: 1.028113)
4.2 gdb benchmark: Avg: 0.201970 (Laps: 5, Total Elapsed Time: 1.009849)
gdb_42_avg/gdb_41_avg: 0.982236
ok
2: test_run_42_then_41 (TestFlintVsSlateGDBDisassembly.FlintVsSlateGDBDisassembly)
Test disassembly on a large function with 4.1 vs. 4.2's gdb. ...
4.2 gdb benchmark: Avg: 0.202602 (Laps: 5, Total Elapsed Time: 1.013012)
4.1 gdb benchmark: Avg: 0.204418 (Laps: 5, Total Elapsed Time: 1.022089)
gdb_42_avg/gdb_41_avg: 0.991119
ok
----------------------------------------------------------------------
Ran 2 tests in 15.688s
OK
llvm-svn: 137092
Sample run on my OSX Lion (MacBook Pro):
1: test_run_gdb_then_lldb (TestDisassembly.DisassembleDriverMainLoop)
Test disassembly on a large function with lldb vs. gdb. ...
gdb benchmark: Avg: 0.201802 (Laps: 5, Total Elapsed Time: 1.009008)
lldb benchmark: Avg: 0.109569 (Laps: 5, Total Elapsed Time: 0.547843)
lldb_avg/gdb_avg: 0.542952
ok
2: test_run_lldb_then_gdb (TestDisassembly.DisassembleDriverMainLoop)
Test disassembly on a large function with lldb vs. gdb. ...
lldb benchmark: Avg: 0.109580 (Laps: 5, Total Elapsed Time: 0.547902)
gdb benchmark: Avg: 0.201587 (Laps: 5, Total Elapsed Time: 1.007936)
lldb_avg/gdb_avg: 0.543588
ok
llvm-svn: 136931
Modify lldbbench.py so that lldbtest.line_number() utility function is available to
BenchBase client as just line_number(), and modify lldbtest.py so that self.lldbExec
(the full path for the 'lldb' executable) is available to BenchBase client as well.
An example run of the test case on my MacBook Pro running Lion:
1: test_compare_lldb_to_gdb (TestRepeatedExprs.RepeatedExprsCase)
Test repeated expressions with lldb vs. gdb. ...
lldb_avg: 0.204339
gdb_avg: 0.205721
lldb_avg/gdb_avg: 0.993284
ok
llvm-svn: 136740
Stopwatch (self.swatch) within the BenchBase's setUp() instance method to be available
to all the child classes.
Use self.swatch to measure elapsed time in TestRepeatedExprs.py, which needs to be
modified later on to actually measure repeated expression evaluations within the
context of lldb as well as gdb.
llvm-svn: 136664
The test driver now takes an option "+b" which enables to run just the benchmarks tests.
By default, tests decorated with the @benchmarks_test decorator do not get run.
Add an example benchmarks test directory which contains nothing for the time being,
just to demonstrate the @benchmarks_test concept.
For example,
$ ./dotest.py -v benchmarks
...
----------------------------------------------------------------------
Collected 2 tests
1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
Test repeated expressions with gdb. ... skipped 'benchmarks tests'
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
Test repeated expressions with lldb. ... skipped 'benchmarks tests'
----------------------------------------------------------------------
Ran 2 tests in 0.047s
OK (skipped=2)
$ ./dotest.py -v +b benchmarks
...
----------------------------------------------------------------------
Collected 2 tests
1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
Test repeated expressions with gdb. ... running test_with_gdb
benchmarks result for test_with_gdb
ok
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
Test repeated expressions with lldb. ... running test_with_lldb
benchmarks result for test_with_lldb
ok
----------------------------------------------------------------------
Ran 2 tests in 0.270s
OK
Also mark some Python API tests which are missing the @python_api_test decorator.
llvm-svn: 136553
