Navbar:

Color Console Output

Enable color output on the console with the ‘—color’ command-line option.

Screenshot

Regular Expressions

Choose which tests to run using regular expressions with the ‘—regex’ command-line option.

See screenshot above.

Glob Patterns

Choose which tests to run with Unix-style glob patterns with the ‘—glob’ command-line option.

Immediate Reports

Get immediate failure reports with the ‘—immediate’ command-line option.

prompt> testoob tests/suites.py --regex='test[A-D]|CaseFailure|test[0-9]' --immediate
..........F
======================================================================
FAIL: testFailure (suites.CaseFailure.testFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/suites.py", line 53, in testFailure
    def testFailure(self): self.fail()
AssertionError
======================================================================
....
----------------------------------------------------------------------
Ran 15 tests in 0.018s
FAILED (failures=1)
prompt>

XML Output

Output XML with the ‘—xml’ command-line option, and pass them to an automated system.

<testsuites>
  <testcase name="testSuccess (suites.CaseMixed)" time="0.0002">
    <result>success</result>
  </testcase>
  <testcase name="testError (suites.CaseMixed)" time="0.0014">
    <result>error</result>
    <error message="" type="exceptions.RuntimeError">
Traceback (most recent call last):
  ...
    </error>
  </testcase>
  <total_time value="0.0016"/>
</testsuites>

HTML Output

Output HTML for viewing test results in a web browser with the ‘—html’ command-line option.

Requires the 4Suite Python package.

The HTML output is generated with XSLT from the XML output. It’s possible to use an external XSLT processor, such as xsltproc, Xalan, or Saxon.

Name Time Result Info
test7 (suites.CaseDigits) 0.0002 Success
testError (suites.CaseError) 0.0016 error
Traceback (most recent call last):
File "../../tests/suites.py", line 59, in testError
def testError(self): raise RuntimeError
RuntimeError

Debug

Run pdb on failing tests with the ‘—debug’ command-line option.

prompt> testoob tests/suites.py -v --debug CaseMixed.testSuccess CaseMixed.testError
testSuccess (suites.CaseMixed.testSuccess) ... ok
testError (suites.CaseMixed.testError) ... ERROR

Debugging for error in test: testError (suites.CaseMixed.testError)
> /path/to/project/tests/suites.py(46)testError()
-> def testError(self): raise RuntimeError
(Pdb)

Verbose Asserts

Get full information on your asserts with the ‘—vassert’ command-line option.

Useful to see that you’re testing what you think you are.

prompt> testoob tests/suites.py --regex='test[0-1]' --vassert
test0 (suites.CaseDigits.test0) ... ok
  [ PASSED (assertEquals) first: "00" second: "00" ]
test1 (suites.CaseDigits.test1) ... ok
  [ PASSED (assertEquals) first: "11" second: "11" ]

----------------------------------------------------------------------
Ran 2 tests in 0.008s
OK
prompt>

Multiple Processes

Run tests in parallel in different processes with the ‘—processes’ command-line option.

This should work in most cases, even when the code isn’t thread-safe.

Multiple Threads

Run tests in parallel with the ‘—threads’ command-line option.

Requires Twisted.

Without threads:
prompt> testoob tests/timed_suite.py
...............
----------------------------------------------------------------------
Ran 15 tests in 16.174s
OK
prompt>
With threads:
prompt> testoob tests/timed_suite.py --threads=7
...............
----------------------------------------------------------------------
Ran 15 tests in 3.376s
OK
prompt>

NOTE: Make sure your tests are thread safe! If they’re not, don’t expect this option to work. If you’re using C extensions that aren’t thread safe, the tests can crash quite messily.

Repeat tests

Repeat each test a number of times with the ‘—repeat’ command-line option.

Some uses for this:

Wait Between Tests

Sleep between tests with the ‘—interval’ command line option.

Test Command-Line Utilities

Use testoob.testing.command_line to test command-line utilities. You can check an application’s standard output, standard error, and return code.

Checking that nmap detects a local HTTP server:
  def testNmapLocalHTTPServer(self):
    testoob.testing.command_line(["nmap", "localhost"],
                                 expected_output_regex="80/tcp   open  http")
Testing md5sum:
  def testMD5Sum(self):
    testoob.testing.command_line(
      ["md5sum"],
      input="abc\n",
      expected_output="0bee89b07a248e27c83fc3d5951213c1 *-\n")