Commits · 3397bb248fcd0425e1c6f15d7aff7bd6c95f122d · Roger Ferrer / llvm-epi-0.8

Sep 04, 2012

Patch from Filipe Cabecinhas that uses argparse in dotest.py instead of a hand... · 5ec9645f

Greg Clayton authored Sep 04, 2012

Patch from Filipe Cabecinhas that uses argparse in dotest.py instead of a hand coded option. I made a few modifications:

Changed the '-A' option to also have a long option of '--arch'. This is now specified multiple times to get multiple architectures.

Old: -A i386^x86_64
New: -A i386 -A x86_64
     --arch i386 --arch x86_64
     
Changed the '-C' option to also have a long option of '--compiler'. This is now specified multiple times to get multiple compiler.

Old: -C clang^gcc
New: -C clang -C gcc
     --compiler clang --compiler gcc
llvm-svn: 163141

5ec9645f

Aug 22, 2012
- Fix wrong directory pathname. · e3fe26c5
  Johnny Chen authored Aug 22, 2012
```
llvm-svn: 162372
```
  e3fe26c5
Aug 16, 2012
- Catch timestamps for the beginning and end of the test suite run. · a60c5ad8
  Johnny Chen authored Aug 16, 2012
```
llvm-svn: 162040
```
  a60c5ad8
Aug 08, 2012
- Indentation fix for dotest.py. · f2bac60f
  Filipe Cabecinhas authored Aug 08, 2012
```
llvm-svn: 161493
```
  f2bac60f
- Make dotest.py runnable from a directory other than lldb/test · d6cab1f6
  Filipe Cabecinhas authored Aug 08, 2012
```
llvm-svn: 161492
```
  d6cab1f6
May 22, 2012
- Fix a bug where if I just run: · 058bb990
  Johnny Chen authored May 22, 2012
```
    ./dotest.py

No progress bar appears.

llvm-svn: 157226
```
  058bb990
Apr 25, 2012
- Maked LLDB into a package so we can import things without poluting the global namespace. · 6d98f56c
  Greg Clayton authored Apr 25, 2012
```
Enrico will follow this up with fixing the data formatter test cases that are failing.

llvm-svn: 155514
```
  6d98f56c
Apr 24, 2012

Add a '-R' option, which is similar to '-r', except that the relocated... · 8f198c96

Johnny Chen authored Apr 24, 2012

Add a '-R' option, which is similar to '-r', except that the relocated directory, if exists, will be removed entirely
before running the test suite. A usage example looks like this:

test $ ./dotest.py -A x86_64 -R /tmp/x86_64 &
test $ ./dotest.py -A i386 -R /tmp/i386 &

where we would want to run the x86_64 and i386 archs concurrently but relocate the test suite to different directory
hierarchies in order not to stump on each other's intermediate files.

llvm-svn: 155491

8f198c96

Apr 19, 2012

LLDB test suite should also output the config info string along with the stack trace. · c9cb71a0

Johnny Chen authored Apr 19, 2012

rdar://problem/11283401

Example:

Collected 1 test

1: test_with_dwarf (TestCallStdStringFunction.ExprCommandCallFunctionTestCase)
   Test calling std::String member function. ... FAIL

======================================================================
FAIL: test_with_dwarf (TestCallStdStringFunction.ExprCommandCallFunctionTestCase)
   Test calling std::String member function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Volumes/data/lldb/svn/ToT/test/lldbtest.py", line 427, in wrapper
    return func(self, *args, **kwargs)
  File "/Volumes/data/lldb/svn/ToT/test/expression_command/call-function/TestCallStdStringFunction.py", line 34, in test_with_dwarf
    self.call_function()
  File "/Volumes/data/lldb/svn/ToT/test/expression_command/call-function/TestCallStdStringFunction.py", line 48, in call_function
    substrs = ['Hello world'])
  File "/Volumes/data/lldb/svn/ToT/test/lldbtest.py", line 1235, in expect
    msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : 'Hello world' returns expected result
Config=i386-clang
----------------------------------------------------------------------
Ran 1 test in 1.148s

FAILED (failures=1)

llvm-svn: 155157

c9cb71a0

Simplify the progress bar display when neither "-v" nor "-t" is passed to the test driver. · 39b8c356
Johnny Chen authored Apr 19, 2012
```
llvm-svn: 155152
```
39b8c356

Apr 16, 2012

Add the capability of supplying the pre/post-flight functions to the test suite such that · 44d24971

Johnny Chen authored Apr 16, 2012

the pre-flight code gets executed during setUp() after the debugger instance is available
and the post-flight code gets executed during tearDown() after the debugger instance has
done killing the inferior and deleting all the target programs.

Example:

[11:32:48] johnny:/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -A x86_64 -v -c ../examples/test/.lldb-pre-post-flight  functionalities/watchpoint/hello_watchpoint
config: {'pre_flight': <function pre_flight at 0x1098541b8>, 'post_flight': <function post_flight at 0x109854230>}
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-139
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 154753
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 154730
Last Changed Date: 2012-04-13 18:42:46 -0700 (Fri, 13 Apr 2012)


lldb.pre_flight: def pre_flight(test):
    __import__("lldb")
    __import__("lldbtest")
    print "\nRunning pre-flight function:"
    print "for test case:", test

lldb.post_flight: def post_flight(test):
    __import__("lldb")
    __import__("lldbtest")
    print "\nRunning post-flight function:"
    print "for test case:", test


Session logs for test failures/errors/unexpected successes will go into directory '2012-04-16-11_34_08'
Command invoked: python ./dotest.py -A x86_64 -v -c ../examples/test/.lldb-pre-post-flight functionalities/watchpoint/hello_watchpoint
compilers=['clang']

Configuration: arch=x86_64 compiler=clang
----------------------------------------------------------------------
Collected 2 tests

1: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... 
Running pre-flight function:
for test case: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)

Running post-flight function:
for test case: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
ok
2: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... 
Running pre-flight function:
for test case: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)

Running post-flight function:
for test case: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
ok

----------------------------------------------------------------------
Ran 2 tests in 1.584s

OK

llvm-svn: 154847

44d24971

Apr 12, 2012
- Make the default architectures to run as both 64 and 32-bit. · b127f00b
  Johnny Chen authored Apr 12, 2012
```
rdar://problem/11031264

llvm-svn: 154573
```
  b127f00b
Apr 06, 2012

Add a new option to the test driver, -N dsym or -N dwarf, in order to exclude tests decorated with · f1548d4f

Johnny Chen authored Apr 06, 2012

either @dsym_test or @dwarf_test to be executed during the testsuite run.  There are still lots of
Test*.py files which have not been decorated with the new decorator.

An example:

# From TestMyFirstWatchpoint.py ->
class HelloWatchpointTestCase(TestBase):

    mydir = os.path.join("functionalities", "watchpoint", "hello_watchpoint")

    @dsym_test
    def test_hello_watchpoint_with_dsym_using_watchpoint_set(self):
        """Test a simple sequence of watchpoint creation and watchpoint hit."""
        self.buildDsym(dictionary=self.d)
        self.setTearDownCleanup(dictionary=self.d)
        self.hello_watchpoint()

    @dwarf_test
    def test_hello_watchpoint_with_dwarf_using_watchpoint_set(self):
        """Test a simple sequence of watchpoint creation and watchpoint hit."""
        self.buildDwarf(dictionary=self.d)
        self.setTearDownCleanup(dictionary=self.d)
        self.hello_watchpoint()


# Invocation ->
[17:50:14] johnny:/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -N dsym -v -p TestMyFirstWatchpoint.py
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-137
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 154133
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 154109
Last Changed Date: 2012-04-05 10:43:02 -0700 (Thu, 05 Apr 2012)



Session logs for test failures/errors/unexpected successes will go into directory '2012-04-05-17_50_49'
Command invoked: python ./dotest.py -N dsym -v -p TestMyFirstWatchpoint.py
compilers=['clang']

Configuration: arch=x86_64 compiler=clang
----------------------------------------------------------------------
Collected 2 tests

1: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... skipped 'dsym tests'
2: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
   Test a simple sequence of watchpoint creation and watchpoint hit. ... ok

----------------------------------------------------------------------
Ran 2 tests in 1.138s

OK (skipped=1)
Session logs for test failures/errors/unexpected successes can be found in directory '2012-04-05-17_50_49'
[17:50:50] johnny:/Volumes/data/lldb/svn/ToT/test $ 

llvm-svn: 154154

f1548d4f

Mar 20, 2012

Add a '-E' option to the test driver for the purpose of specifying some extra CFLAGS · e344486e

Johnny Chen authored Mar 20, 2012

to pass to the toolchain in order to build the inferior programs to be run/debugged
duirng the test suite.  The architecture might dictate some special CFLAGS which are
more easily specified in a central place (like the command line) instead of inside
make rules.

For Example,

./dotest.py -v -r /shared/phone -A armv7 -E "-isysroot your_sdk_root" functionalities/watchpoint/hello_watchpoint

will relocate the particular test directory ('functionalities/watchpoint/hello_watchpoint' in this case) to a
new directory named '/shared/phone'.  The particular incarnation of the architecture-compiler combination of the
test support files are therefore to be found under:

/shared/phone.arch=armv7-compiler=clang/functionalities/watchpoint/hello_watchpoint

The building of the inferior programs under testing is now working.

The actual launching/debugging of the inferior programs are not yet working,
neither is the setting of a watchpoint on the phone.

llvm-svn: 153070

e344486e

Mar 12, 2012
- Disabled the i386 tests pending fixes in the JIT. · ddcda28f
  Sean Callanan authored Mar 12, 2012
```
llvm-svn: 152568
```
  ddcda28f
Mar 09, 2012

Add the capability on OS X to utilize 'xcrun' to locate the compilers used for... · 934c05d2

Johnny Chen authored Mar 09, 2012

Add the capability on OS X to utilize 'xcrun' to locate the compilers used for building the inferior programs
to be debugged while running the test suite. By default, compilers is set to ['clang'] and can be overridden
using the "-C compilerA^compilerB" option.

llvm-svn: 152367

934c05d2

Change the test driver so that, by default, it takes into consideration of... · 5a9a9883

Johnny Chen authored Mar 09, 2012

Change the test driver so that, by default, it takes into consideration of both 'x86_64' and 'i386' architectures
when building the inferior programs.

Example:

/Volumes/data/lldb/svn/ToT/test $ ./dotest.py -v functionalities/watchpoint
LLDB build dir: /Volumes/data/lldb/svn/ToT/build/Debug
LLDB-123
Path: /Volumes/data/lldb/svn/ToT
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 152244
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 152244
Last Changed Date: 2012-03-07 13:03:09 -0800 (Wed, 07 Mar 2012)

Session logs for test failures/errors/unexpected successes will go into directory '2012-03-08-16_43_51'
Command invoked: python ./dotest.py -v functionalities/watchpoint

Configuration: arch=x86_64
----------------------------------------------------------------------
Collected 21 tests

1: test_hello_watchlocation_with_dsym (TestWatchLocation.HelloWatchLocationTestCase)
Test watching a location with '-x size' option. ... ok
2: test_hello_watchlocation_with_dwarf (TestWatchLocation.HelloWatchLocationTestCase)
Test watching a location with '-x size' option. ... ok
3: test_hello_watchpoint_with_dsym_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
4: test_hello_watchpoint_with_dwarf_using_watchpoint_set (TestMyFirstWatchpoint.HelloWatchpointTestCase)
Test a simple sequence of watchpoint creation and watchpoint hit. ... ok
5: test_watchpoint_multiple_threads_with_dsym (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
Test that lldb watchpoint works for multiple threads. ... ok
6: test_watchpoint_multiple_threads_with_dwarf (TestWatchpointMultipleThreads.WatchpointForMultipleThreadsTestCase)
Test that lldb watchpoint works for multiple threads. ... ok
7: test_rw_disable_after_first_stop__with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint but disable it after the first stop. ... ok
8: test_rw_disable_after_first_stop_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint but disable it after the first stop. ... ok
9: test_rw_disable_then_enable_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint, disable initially, then enable it. ... ok
10: test_rw_disable_then_enable_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint, disable initially, then enable it. ... ok
11: test_rw_watchpoint_delete_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
Test delete watchpoint and expect not to stop for watchpoint. ... ok
12: test_rw_watchpoint_delete_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
Test delete watchpoint and expect not to stop for watchpoint. ... ok
13: test_rw_watchpoint_set_ignore_count_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
Test watchpoint ignore count and expect to not to stop at all. ... ok
14: test_rw_watchpoint_set_ignore_count_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
Test watchpoint ignore count and expect to not to stop at all. ... ok
15: test_rw_watchpoint_with_dsym (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint and expect to stop two times. ... ok
16: test_rw_watchpoint_with_dwarf (TestWatchpointCommands.WatchpointCommandsTestCase)
Test read_write watchpoint and expect to stop two times. ... ok
17: test_watchpoint_cond_with_dsym (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
Test watchpoint condition. ... ok
18: test_watchpoint_cond_with_dwarf (TestWatchpointConditionCmd.WatchpointConditionCmdTestCase)
Test watchpoint condition. ... ok
19: test_watchlocation_with_dsym_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
20: test_watchlocation_with_dwarf_using_watchpoint_set (TestWatchLocationWithWatchSet.WatchLocationUsingWatchpointSetTestCase)
Test watching a location with 'watchpoint set expression -w write -x size' option. ... ok
21: test_error_cases_with_watchpoint_set (TestWatchpointSetErrorCases.WatchpointSetErrorTestCase)
Test error cases with the 'watchpoint set' command. ... ok

----------------------------------------------------------------------
Ran 21 tests in 74.590s

Configuration: arch=i386
----------------------------------------------------------------------
Collected 21 tests

----------------------------------------------------------------------
Ran 21 tests in 67.059s

llvm-svn: 152357

5a9a9883

Jan 31, 2012

Add a period. · 5d0d607b
Johnny Chen authored Jan 31, 2012
```
llvm-svn: 149305
```
5d0d607b

Add a '-u ENV_VAR_NAME' option to the test driver, whose purpose is to unset the said · 8f8a2a77

Johnny Chen authored Jan 31, 2012

environment variable before starting the test runner which executes the test cases and
may spawn child processes.  An example:

    ./dotest.py -u MY_ENV1 -u MY_ENV2 -v -p TestWatchLocationWithWatchSet.py

llvm-svn: 149304

8f8a2a77

Jan 18, 2012

Add a '-X excluded-dir' option to the test driver to exclude a directory from... · bc774c11

Johnny Chen authored Jan 18, 2012

Add a '-X excluded-dir' option to the test driver to exclude a directory from consideration during test discovery.  For example:

    ./dotest.py -X types -v

from the test dir will ignore test cases under test/types.

llvm-svn: 148379

bc774c11

Jan 17, 2012

Add usage example of specifying the full path to the compiler used while... · 2e275c9e

Johnny Chen authored Jan 17, 2012

Add usage example of specifying the full path to the compiler used while building the debuggees for running the test suite.

llvm-svn: 148273

2e275c9e

Nov 18, 2011
- Add a simple progress bar when neither '-v' nor '-t' is specified. · 316651ab
  Johnny Chen authored Nov 18, 2011
```
llvm-svn: 144940
```
  316651ab
Nov 17, 2011

Add an option '-S' to skip the build and cleanup while running the test. · 0fddfb2c

Johnny Chen authored Nov 17, 2011

Use this option with care as you would need to build the inferior(s) by hand
and build the executable(s) with the correct name(s).  This option can be used
with '-# n' to stress test certain test cases for n number of times.

An example:

[11:55:11] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ ls
Makefile		TestValueAPI.pyc	linked_list
TestValueAPI.py		change_values		main.c
[11:55:14] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ make EXE=test_with_dsym
clang -gdwarf-2 -O0  -arch x86_64   -c -o main.o main.c
clang -gdwarf-2 -O0  -arch x86_64   main.o -o "test_with_dsym"
/usr/bin/dsymutil  -o "test_with_dsym.dSYM" "test_with_dsym"
[11:55:20] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ cd ../..
[11:55:24] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
LLDB build dir: /Volumes/data/lldb/svn/trunk/build/Debug
LLDB-89
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 144914
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 144911
Last Changed Date: 2011-11-17 09:22:31 -0800 (Thu, 17 Nov 2011)



Session logs for test failures/errors/unexpected successes will go into directory '2011-11-17-11_55_29'
Command invoked: python ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
----------------------------------------------------------------------
Collected 1 test

1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.163s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.200s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.198s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.199s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.239s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.215s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.105s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.098s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.195s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.197s

OK
[11:55:34] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 144919

0fddfb2c

Nov 01, 2011
- tests: Improve Makefile/dotest to properly set LLDB_HERE variable used in some · 16d88ff0
  Daniel Dunbar authored Oct 31, 2011
```
tests.

llvm-svn: 143394
```
  16d88ff0
Oct 28, 2011
- Fix a typo. · 6c75b61f
  Johnny Chen authored Oct 28, 2011
```
llvm-svn: 143207
```
  6c75b61f
- Print out the version of the locally built 'lldb' binary, not the one found in... · d9809f54
  Johnny Chen authored Oct 28, 2011
```
Print out the version of the locally built 'lldb' binary, not the one found in your PATH env variable.

llvm-svn: 143170
```
  d9809f54
Oct 25, 2011

Benchmark the turnaround time starting a debugger and run to the breakpoint with lldb vs. gdb. · fc9e79fb

Johnny Chen authored Oct 25, 2011

An example (with /Developer/usr/bin/lldb vs. /usr/bin/gdb):

[13:05:04] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v +b -n -p TestCompileRunToBreakpointTurnaround.py
1: test_run_lldb_then_gdb (TestCompileRunToBreakpointTurnaround.CompileRunToBreakpointBench)
   Benchmark turnaround time with lldb vs. gdb. ... 
lldb turnaround benchmark: Avg: 4.574600 (Laps: 3, Total Elapsed Time: 13.723799)
gdb turnaround benchmark: Avg: 7.966713 (Laps: 3, Total Elapsed Time: 23.900139)
lldb_avg/gdb_avg: 0.574214
ok

----------------------------------------------------------------------
Ran 1 test in 55.462s

OK

llvm-svn: 142949

fc9e79fb

Oct 22, 2011

Add bench.py as a driver script to run some benchmarks on lldb. · b8da4262

Johnny Chen authored Oct 22, 2011

Add benchmarks for expression evaluations (TestExpressionCmd.py) and disassembly (TestDoAttachThenDisassembly.py).

An example:
[17:45:55] johnny:/Volumes/data/lldb/svn/trunk/test $ ./bench.py 2>&1 | grep -P '^lldb.*benchmark:'
lldb startup delay (create fresh target) benchmark: Avg: 0.104274 (Laps: 30, Total Elapsed Time: 3.128214)
lldb startup delay (set first breakpoint) benchmark: Avg: 0.102216 (Laps: 30, Total Elapsed Time: 3.066470)
lldb frame variable benchmark: Avg: 1.649162 (Laps: 20, Total Elapsed Time: 32.983245)
lldb stepping benchmark: Avg: 0.104409 (Laps: 50, Total Elapsed Time: 5.220461)
lldb expr cmd benchmark: Avg: 0.206774 (Laps: 25, Total Elapsed Time: 5.169350)
lldb disassembly benchmark: Avg: 0.089086 (Laps: 10, Total Elapsed Time: 0.890859)

llvm-svn: 142708

b8da4262

Oct 21, 2011
- Add a '-n' option to turn off printings of build dir, lldb version, svn info, and other headers · 4d4363ba
  Johnny Chen authored Oct 21, 2011
```
which happen before the listingings of test cases.

llvm-svn: 142668
```
  4d4363ba
- Breakpoint specification can have the form '-n main', so it's not a good idea to · ff7fc9cf
  Johnny Chen authored Oct 20, 2011
```
check that the option arg in '-x opt_arg' does not start with a '-' char.

llvm-svn: 142625
```
  ff7fc9cf
Oct 20, 2011

Parameterize the iteration count used when running benchmarks, instead of... · 38f9daa3

Johnny Chen authored Oct 20, 2011

Parameterize the iteration count used when running benchmarks, instead of hard-coded inside the test case.
Add a '-y count' option to the test driver for this purpose.  An example:

 $  ./dotest.py -v -y 25 +b -p TestDisassembly.py

...

----------------------------------------------------------------------
Collected 2 tests

1: test_run_gdb_then_lldb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
gdb benchmark: Avg: 0.226305 (Laps: 25, Total Elapsed Time: 5.657614)
lldb benchmark: Avg: 0.113864 (Laps: 25, Total Elapsed Time: 2.846606)
lldb_avg/gdb_avg: 0.503146
ok
2: test_run_lldb_then_gdb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
lldb benchmark: Avg: 0.113008 (Laps: 25, Total Elapsed Time: 2.825201)
gdb benchmark: Avg: 0.225240 (Laps: 25, Total Elapsed Time: 5.631001)
lldb_avg/gdb_avg: 0.501723
ok

----------------------------------------------------------------------
Ran 2 tests in 41.346s

OK

llvm-svn: 142598

38f9daa3

Oct 11, 2011

Add '-k' option to the test driver to be able to specify a runhook, which is an lldb command · 999f7ab9

Johnny Chen authored Oct 11, 2011

for the debugger to execute for certain kind of tests (for example, a benchmark).
A list of runhooks can be used to steer the debugger into the desired state before more
actions can be performed.

llvm-svn: 141626

999f7ab9

Add '-e' and '-x' options to the test driver to be able to specify an executable (full path) · a10e6c10

Johnny Chen authored Oct 10, 2011

and the breakpoint specification for the benchmark purpose. This is used by TestSteppingSpeed.py
to benchmark the lldb stepping speed. Without '-e' and 'x' specified, the test defaults to
run the built lldb against itself and stopped on Driver::MainLoop, then stepping for 50 times.

rdar://problem/7511193

llvm-svn: 141584

a10e6c10

Sep 16, 2011
- Minor change for output message (less is better). · 98460191
  Johnny Chen authored Sep 16, 2011
```
llvm-svn: 139924
```
  98460191
- To be more paranoid, check cmd_output before searching in it. · 9c0a94ad
  Johnny Chen authored Sep 16, 2011
```
llvm-svn: 139920
```
  9c0a94ad
- Rephrase the checkDsymForUUIDIsNotOn() a bit so that in the normal case where automatic · 095ff6fd
  Johnny Chen authored Sep 16, 2011
```
lookup and cacheing of dSYMs is not enabled, the 'defaults read com.apple.DebugSymbols'
output is not shown.

llvm-svn: 139914
```
  095ff6fd
- Add a check for the test driver to make sure that, on Mac OS X, the automatic lookup and caching · 3cc78e60
  Johnny Chen authored Sep 16, 2011
```
of dSYMs is not turned on before running the test suite.

llvm-svn: 139896
```
  3cc78e60
Aug 26, 2011

Add a new attribute self.lldbHere, representing the fullpath to the 'lldb' executable · d890bfc9

Johnny Chen authored Aug 26, 2011

built locally from the source tree. This is distinguished from self.lldbExec, which
can be used by test/benchmarks to measure the performances against other debuggers.

You can use environment variable LLDB_EXEC to specify self.lldbExec to the dotest.py
test driver, otherwise it is going to be populated with self.lldbHere.

Modify the regular tests under test dir, i.e., not test/benchmarks, to use self.lldbHere.
Also modify the benchmarks tests to use self.lldbHere when it needs an 'lldb' executable
with debug info to do the performance measurements.

llvm-svn: 138608

d890bfc9

Aug 16, 2011
- Test driver should also report skipped tests because there were cases when tests were skipped · c5cc625c
  Johnny Chen authored Aug 15, 2011
```
due to incorrect skip-logic.

llvm-svn: 137678
```
  c5cc625c
Aug 13, 2011
- Fix a test driver bug where multiple filter specs (through the -f TestClassName.test_method_name · 60726fab
  Johnny Chen authored Aug 12, 2011
```
command line option) belonging to the same Test*.py file are ignored, except for the first one.

llvm-svn: 137540
```
  60726fab