Generating tests in Python unittest

There are many applications where you want to apply a code to a variety of data sets, and verify that you get the correct output. In this context, what you want is a test generator, which can dynamically create tests, based on the set of data sets that are available for testing.

Unfortunately, this does not appear to be a feature of unittest. The closest I have seen to this, is the support for test generators in the nose package, which extends unittest to provide test discovery mechanisms. However, that test generation feature is somewhat limited; it only applies to test functions that are Python generators, and not to similar class methods.

The following example shows how to directly insert new test methods into a unittest.TestCase class:

#
# A simple example for generating tests in the Python unittest framework
#

import glob
import unittest

#
# Defining the class that will contain the new tests
#
class TestCases(unittest.TestCase): pass

#
# A generic function that performs a test on a particular file
#
def perform_test(self,file):
  if len(file) > 20:
      self.fail("Failing in file \""+file+"\" because its name is too long.")

#
# Insert test methods into the TestCases class
#
for file in glob.glob("*")+glob.glob("*/*"):
  tmp = file.replace("/","_")
  tmp = tmp.replace("\\","_")
  tmp = tmp.replace(".","_")
  setattr(TestCases, "test_"+tmp, lambda self,x=file: perform_test(self,x))

#
# Apply these unittests
#
if __name__ == "__main__":
  unittest.main()

In this example, the files in the current directory and in all subdirectories are used to create new test methods. For each file, a new test method is added to the TestCases class, which does a silly check to see if the file is too long.