Tuesday, October 28, 2008

Generating tests in Python unittest

There are many applications where you want to apply a code to a variety of data sets, and verify that you get the correct output. In this context, what you want is a test generator, which can dynamically create tests, based on the set of data sets that are available for testing.

Unfortunately, this does not appear to be a feature of unittest. The closest I have seen to this, is the support for test generators in the nose package, which extends unittest to provide test discovery mechanisms. However, that test generation feature is somewhat limited; it only applies to test functions that are Python generators, and not to similar class methods.

The following example shows how to directly insert new test methods into a unittest.TestCase class:


#
# A simple example for generating tests in the Python unittest framework
#

import glob
import unittest

#
# Defining the class that will contain the new tests
#
class TestCases(unittest.TestCase): pass

#
# A generic function that performs a test on a particular file
#
def perform_test(self,file):
if len(file) > 20:
self.fail("Failing in file \""+file+"\" because its name is too long.")

#
# Insert test methods into the TestCases class
#
for file in glob.glob("*")+glob.glob("*/*"):
tmp = file.replace("/","_")
tmp = tmp.replace("\\","_")
tmp = tmp.replace(".","_")
setattr(TestCases, "test_"+tmp, lambda self,x=file: perform_test(self,x))

#
# Apply these unittests
#
if __name__ == "__main__":
unittest.main()

In this example, the files in the current directory and in all subdirectories are used to create new test methods. For each file, a new test method is added to the TestCases class, which does a silly check to see if the file is too long.

2 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Nice hack! I tried __new__() construct to create test* methods, but unittest loader inspects them before creating an instance thus skipping the code.

    ReplyDelete