Friday, January 20, 2012

Testing Open Source Software

Software testing is widely recognized as a best practice for software development. Software tests define expected functionality, and they can focus developer efforts by providing an objective assessment of the state of a software project. Additionally, software testing data can provide evidence that a software package can be reliably used. For example, when evaluating whether to try out open source software, I routinely look for software testing data to confirm which platforms the software will run on, the versions of associated software that is used, and test coverage statistics that indicate how much of the the code is tested.

Unfortunately, most open source software projects do not publish software test data.  I suspects that this indicates that a small fraction of OSS projects have robust test suites.  However, this also reflects another aspect of the OSS community:  hosting facilities for open source software do not support web-based testing facilities, like Jenkins, that can be used by developers to remotely launch jobs on test machines with a variety of different configurations. This is not totally unexpected, since testing can be computationally intensive.

Recently, I learned about CloudBees, which provides cloud services for building, running and managing Java applications. Happily CloudBees makes its Dev@cloud service freely available to open source projects! This includes the Jenkins testing service, which provides a limited number of CPU hours each month for testing an OSS project.  For example, the CxxTest project now hosts tests on a CloudBees Jenkins server.  Cool!

No comments:

Post a Comment