Use this option with care as you would need to build the inferior(s) by hand
and build the executable(s) with the correct name(s).  This option can be used
with '-# n' to stress test certain test cases for n number of times.

An example:

[11:55:11] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ ls
Makefile		TestValueAPI.pyc	linked_list
TestValueAPI.py		change_values		main.c
[11:55:14] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ make EXE=test_with_dsym
clang -gdwarf-2 -O0  -arch x86_64   -c -o main.o main.c
clang -gdwarf-2 -O0  -arch x86_64   main.o -o "test_with_dsym"
/usr/bin/dsymutil  -o "test_with_dsym.dSYM" "test_with_dsym"
[11:55:20] johnny:/Volumes/data/lldb/svn/trunk/test/python_api/value $ cd ../..
[11:55:24] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
LLDB build dir: /Volumes/data/lldb/svn/trunk/build/Debug
LLDB-89
Path: /Volumes/data/lldb/svn/trunk
URL: https://johnny@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://johnny@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 144914
Node Kind: directory
Schedule: normal
Last Changed Author: gclayton
Last Changed Rev: 144911
Last Changed Date: 2011-11-17 09:22:31 -0800 (Thu, 17 Nov 2011)



Session logs for test failures/errors/unexpected successes will go into directory '2011-11-17-11_55_29'
Command invoked: python ./dotest.py -v -# 10 -S -f ValueAPITestCase.test_with_dsym
----------------------------------------------------------------------
Collected 1 test

1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.163s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.200s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.198s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.199s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.239s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.215s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.105s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.098s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 0.195s

OK
1: test_with_dsym (TestValueAPI.ValueAPITestCase)
   Exercise some SBValue APIs. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.197s

OK
[11:55:34] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 144919

tests.

llvm-svn: 143394

llvm-svn: 143207

Print out the version of the locally built 'lldb' binary, not the one found in your PATH env variable.

llvm-svn: 143170

An example (with /Developer/usr/bin/lldb vs. /usr/bin/gdb):

[13:05:04] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v +b -n -p TestCompileRunToBreakpointTurnaround.py
1: test_run_lldb_then_gdb (TestCompileRunToBreakpointTurnaround.CompileRunToBreakpointBench)
   Benchmark turnaround time with lldb vs. gdb. ... 
lldb turnaround benchmark: Avg: 4.574600 (Laps: 3, Total Elapsed Time: 13.723799)
gdb turnaround benchmark: Avg: 7.966713 (Laps: 3, Total Elapsed Time: 23.900139)
lldb_avg/gdb_avg: 0.574214
ok

----------------------------------------------------------------------
Ran 1 test in 55.462s

OK

llvm-svn: 142949

Add benchmarks for expression evaluations (TestExpressionCmd.py) and disassembly (TestDoAttachThenDisassembly.py).

An example:
[17:45:55] johnny:/Volumes/data/lldb/svn/trunk/test $ ./bench.py 2>&1 | grep -P '^lldb.*benchmark:'
lldb startup delay (create fresh target) benchmark: Avg: 0.104274 (Laps: 30, Total Elapsed Time: 3.128214)
lldb startup delay (set first breakpoint) benchmark: Avg: 0.102216 (Laps: 30, Total Elapsed Time: 3.066470)
lldb frame variable benchmark: Avg: 1.649162 (Laps: 20, Total Elapsed Time: 32.983245)
lldb stepping benchmark: Avg: 0.104409 (Laps: 50, Total Elapsed Time: 5.220461)
lldb expr cmd benchmark: Avg: 0.206774 (Laps: 25, Total Elapsed Time: 5.169350)
lldb disassembly benchmark: Avg: 0.089086 (Laps: 10, Total Elapsed Time: 0.890859)

llvm-svn: 142708

which happen before the listingings of test cases.

llvm-svn: 142668

check that the option arg in '-x opt_arg' does not start with a '-' char.

llvm-svn: 142625

Parameterize the iteration count used when running benchmarks, instead of hard-coded inside the test case.
Add a '-y count' option to the test driver for this purpose.  An example:

 $  ./dotest.py -v -y 25 +b -p TestDisassembly.py

...

----------------------------------------------------------------------
Collected 2 tests

1: test_run_gdb_then_lldb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
gdb benchmark: Avg: 0.226305 (Laps: 25, Total Elapsed Time: 5.657614)
lldb benchmark: Avg: 0.113864 (Laps: 25, Total Elapsed Time: 2.846606)
lldb_avg/gdb_avg: 0.503146
ok
2: test_run_lldb_then_gdb (TestDisassembly.DisassembleDriverMainLoop)
   Test disassembly on a large function with lldb vs. gdb. ... 
lldb benchmark: Avg: 0.113008 (Laps: 25, Total Elapsed Time: 2.825201)
gdb benchmark: Avg: 0.225240 (Laps: 25, Total Elapsed Time: 5.631001)
lldb_avg/gdb_avg: 0.501723
ok

----------------------------------------------------------------------
Ran 2 tests in 41.346s

OK

llvm-svn: 142598

for the debugger to execute for certain kind of tests (for example, a benchmark).
A list of runhooks can be used to steer the debugger into the desired state before more
actions can be performed.

llvm-svn: 141626

and the breakpoint specification for the benchmark purpose.  This is used by TestSteppingSpeed.py
to benchmark the lldb stepping speed.  Without '-e' and 'x' specified, the test defaults to
run the built lldb against itself and stopped on Driver::MainLoop, then stepping for 50 times.

rdar://problem/7511193

llvm-svn: 141584

llvm-svn: 139924

llvm-svn: 139920

lookup and cacheing of dSYMs is not enabled, the 'defaults read com.apple.DebugSymbols'
output is not shown.

llvm-svn: 139914

of dSYMs is not turned on before running the test suite.

llvm-svn: 139896

built locally from the source tree.  This is distinguished from self.lldbExec, which
can be used by test/benchmarks to measure the performances against other debuggers.

You can use environment variable LLDB_EXEC to specify self.lldbExec to the dotest.py
test driver, otherwise it is going to be populated with self.lldbHere.

Modify the regular tests under test dir, i.e., not test/benchmarks, to use self.lldbHere.
Also modify the benchmarks tests to use self.lldbHere when it needs an 'lldb' executable
with debug info to do the performance measurements.

llvm-svn: 138608

due to incorrect skip-logic.

llvm-svn: 137678

command line option) belonging to the same Test*.py file are ignored, except for the first one.

llvm-svn: 137540

DOTEST_SCRIPT_DIR=/path/leading/to/your/dotest/script/directory.

llvm-svn: 137492

There should be nothing unwanted there and a simpe main.cpp (generated from main.cpp.template)
which includes SB*.h should compile and link with the LLDB framework.

llvm-svn: 136894

The test driver now takes an option "+b" which enables to run just the benchmarks tests.
By default, tests decorated with the @benchmarks_test decorator do not get run.

Add an example benchmarks test directory which contains nothing for the time being,
just to demonstrate the @benchmarks_test concept.

For example,

$ ./dotest.py -v benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... skipped 'benchmarks tests'
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... skipped 'benchmarks tests'

----------------------------------------------------------------------
Ran 2 tests in 0.047s

OK (skipped=2)
$ ./dotest.py -v +b benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... running test_with_gdb
benchmarks result for test_with_gdb
ok
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... running test_with_lldb
benchmarks result for test_with_lldb
ok

----------------------------------------------------------------------
Ran 2 tests in 0.270s

OK

Also mark some Python API tests which are missing the @python_api_test decorator.

llvm-svn: 136553

to find out the tests which failed/errored and need re-running.  The dotest.py test driver
script is modified to allow specifying multiple -f testclass.testmethod in the command line
to accommodate the redo functionality.

An example,

 $ ./redo.py -n 2011-07-29-11_50_14
adding filterspec: TargetAPITestCase.test_find_global_variables_with_dwarf
adding filterspec: DisasmAPITestCase.test_with_dsym
Running ./dotest.py -v  -f TargetAPITestCase.test_find_global_variables_with_dwarf -f DisasmAPITestCase.test_with_dsym

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_dsym (TestDisasmAPI.DisasmAPITestCase)
   Exercise getting SBAddress objects, disassembly, and SBAddress APIs. ... ok
2: test_find_global_variables_with_dwarf (TestTargetAPI.TargetAPITestCase)
   Exercise SBTarget.FindGlobalVariables() API. ... ok

----------------------------------------------------------------------
Ran 2 tests in 15.328s

OK

llvm-svn: 136533

llvm-svn: 133836

Fix the log channel for gdb-remote.  It should be "gdb-remote" now, instead of "process.gdb-remote".

llvm-svn: 133534

This is to ensure that session files are written to the correct
directory if the -r option is given.

llvm-svn: 133483

This is required to run the test suite outside of the source tree
or on a case sensitive file system.

llvm-svn: 133460

rather than the current directory.

This is required to run the test suite outside of the source tree.

llvm-svn: 133458

llvm-svn: 132969

the session directory, which also includes the command line used to invoke
the test suite.

llvm-svn: 131501

llvm-svn: 131011

llvm-svn: 130254

llvm-svn: 130251

llvm-svn: 130240

llvm-svn: 129936

llvm-svn: 129470

the CommandInterpreter where it was always being used.

Make sure that Modules can track their object file offsets correctly to
allow opening of sub object files (like the "__commpage" on darwin).

Modified the Platforms to be able to launch processes. The first part of this
move is the platform soon will become the entity that launches your program
and when it does, it uses a new ProcessLaunchInfo class which encapsulates
all process launching settings. This simplifies the internal APIs needed for
launching. I want to slowly phase out process launching from the process
classes, so for now we can still launch just as we used to, but eventually
the platform is the object that should do the launching.

Modified the Host::LaunchProcess in the MacOSX Host.mm to correctly be able
to launch processes with all of the new eLaunchFlag settings. Modified any
code that was manually launching processes to use the Host::LaunchProcess
functions.

Fixed an issue where lldb_private::Args had implicitly defined copy 
constructors that could do the wrong thing. This has now been fixed by adding
an appropriate copy constructor and assignment operator.

Make sure we don't add empty ModuleSP entries to a module list.

Fixed the commpage module creation on MacOSX, but we still need to train
the MacOSX dynamic loader to not get rid of it when it doesn't have an entry
in the all image infos.

Abstracted many more calls from in ProcessGDBRemote down into the 
GDBRemoteCommunicationClient subclass to make the classes cleaner and more
efficient.

Fixed the default iOS ARM register context to be correct and also added support
for targets that don't support the qThreadStopInfo packet by selecting the
current thread (only if needed) and then sending a stop reply packet.

Debugserver can now start up with a --unix-socket (-u for short) and can 
then bind to port zero and send the port it bound to to a listening process
on the other end. This allows the GDB remote platform to spawn new GDB server
instances (debugserver) to allow platform debugging.

llvm-svn: 129351

which the testsuite is run against.

llvm-svn: 127782

Add pexpect-2.4 (a pure Python module for controlling and automating other programs) to the test directory.

http://pypi.python.org/pypi/pexpect/

llvm-svn: 127484

This provides a way to potentially provide conversational interactions with 'lldb' in the test suite.

llvm-svn: 127479

on the command line.  For example, use '-A x86_64^i386' to launch the inferior use both x86_64
and i386.

This is an example of building the debuggee using both clang and gcc compiers:

[17:30:46] johnny:/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -C clang^gcc -v -f SourceManagerTestCase.test_modify_source_file_while_debugging

Session logs for test failures/errors will go into directory '2011-03-03-17_31_39'
Command invoked: python ./dotest.py -C clang^gcc -v -f SourceManagerTestCase.test_modify_source_file_while_debugging

Configuration:  compiler=clang
----------------------------------------------------------------------
Collected 1 test

1: test_modify_source_file_while_debugging (TestSourceManager.SourceManagerTestCase)
   Modify a source file while debugging the executable. ... Command 'run' failed!

original content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

new content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello lldb.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after writing new content: 1299202305.0
content restored to: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after restore: 1299202307.0
ok

----------------------------------------------------------------------
Ran 1 test in 8.259s

OK

Configuration:  compiler=gcc
----------------------------------------------------------------------
Collected 1 test

1: test_modify_source_file_while_debugging (TestSourceManager.SourceManagerTestCase)
   Modify a source file while debugging the executable. ... original content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

new content: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello lldb.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after writing new content: 1299202307.0
content restored to: #include <stdio.h>

int main(int argc, char const *argv[]) {
    printf("Hello world.\n"); // Set break point at this line.
    return 0;
}

os.path.getmtime() after restore: 1299202309.0
ok

----------------------------------------------------------------------
Ran 1 test in 2.301s

OK
[17:31:49] johnny:/Volumes/data/lldb/svn/trunk/test $ 

llvm-svn: 126979