William E. Hart's Blog

Software testing is widely recognized as a best practice for software development . Software tests define expected functionality, and they can focus developer efforts by providing an objective assessment of the state of a software project. Additionally, software testing data can provide evidence that a software package can be reliably used. For example, when evaluating whether to try out open source software, I routinely look for software testing data to confirm which platforms the software will run on, the versions of associated software that is used, and test coverage statistics that indicate how much of the the code is tested. Unfortunately, most open source software projects do not publish software test data.  I suspects that this indicates that a small fraction of OSS projects have robust test suites.  However, this also reflects another aspect of the OSS community:  hosting facilities for open source software do not support web-based testing facilities, like Jenk...

I earlier mentioned that I developed a plugin framework within my PyUtilib software.  This is now called the PyUtilib Component Architecture (PCA).  The PCA is derived from the Trac component architecture, and it supports advanced features like nonsingleton components, namespaces and caching of component interactions. The PCA includes an independent, self-contained framework core that can be easily integrated into other applications, as well as a variety of extension packages with commonly used components. See The PyUtilib Component Architecture for further details.

Coopr is a collection of Python optimization-related packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models. The following are key Coopr capabilities that are driving Coopr development: Pyomo: Formulate algebraic models within Python's modern programming language PySP: Generic solvers for stochastic programming problems COLIN: Scripts that simplify IO between optimizers and black-box applications SUCASA: Customize MIP solvers to expose model structure to the MIP solver engine See https://software.sandia.gov/trac/coopr/wiki/GettingStarted for instructions for getting started with Coopr. An installation script, coopr_install, is provided to simplify the installation of Coopr packages along with the third-party Python packages that they depend on. This installer can also automatically install extension packages from Coin Bazaar. See http://groups.google.com/group/coopr-forum/topics for online discussions of Coopr. Two ...

At PyCon 2009, Aaron Maxwell gave a presentation about the use of BuildBot to support an automated software QA infrastructure. Listening to his talk ( online ) made me think more carefully about the reasons I am not using BuildBot, which I took a look at several years ago.  After working with a custom automated build tool for a few years, I have recently begun using Hudson to automate software quality processes for a variety of open source software packages.  Hudson automates the following QA activities for these packages: portability tests - building packages with different compilers, language versions and compute platforms continuous integration - rapid builds and software tests to provide developers continuous feedback integration tests - builds that test the integration of different software tools archiving QA statistics - test histories, code coverage statistics, build times, etc. managing third-party builds - building third-party libraries that my codes depend on Althou...

The gcovr command provides a utility for running the gcov command and summarizing code coverage results. This command is inspired by the Python coverage.py package, which provides a similar utility in Python. Further, gcovr can be viewed as a command-line alternative of the lcov utility, which runs gcov and generates an HTML output. The gcovr command currently generates two different types of output: Text Summary For each file that generates gcov statistics, gcovr will summarize the number of lines covered, the percentage of coverage and enumerate the lines that are not covered. Cobertura XML An XML summary of the coverage statistics can be generated in a format that is consistent with Cobertura. I find the text summary quite convenient for interactive assessment of coverage, especially as I design tests to improve coverage. The Cobertura summary can be used by continues build tools like Hudson . For example, see the acro-utilib coverage report that was generated with gcovr, ...

After blogging about Python plugin frameworks earlier this year, I wound up implemented a new framework in the PyUtilib software package.  The PyUtilib wiki provides a detailed description of the PyUtilib Plugin Framework , but here's a brief summary: This framework is derived from the Trac plugin framework (a.k.a. component architecture) Provides support for both singleton and non-singleton plugin instances Includes utilities for managing plugins within namespaces The core framework is defined in a single Python module, which can be used independently from PyUtilib PyUtilib also includes commonly use plugins, such as A config-file reader/writer based on ConfigParser Loading utilities for eggs and modules A file manager for temporary files Although I initially resisted the urge to develop my own framework, I was led to develop this because (1) I wanted a light-weight framework like that provided by Trac, but (2) Trac's framework is not particularlly modular within the Trac so...

I have not blogged much this fall because I have been busy managing a variety of software releases.  I plan to include further details in upcoming blogs, but I thought I would summarize these releases here: acro 2.0 - Acro is A Common Repository for Optimizers that integrates a rich variety of optimization libraries and solvers that have been developed for large-scale engineering and scientific applications. Acro was developed to facilitate the design, development, integration and support of optimization software libraries. Thus, Acro includes both individual optimization solvers as well as optimization frameworks that provide abstract interfaces for flexible interoperability of solver components. Furthermore, many solvers included in Acro can exploit parallel computing resources to solve optimization problems more quickly. utilib 4.0 - Utilib is a library of general-purpose C++ utilities, similar in spirit to the Boost libraries. While generally treated as an Acro package, Utilib i...

In the past year, I have increasingly been using Python to develop a variety of OR-related scientific software. In particular, the Coopr library has been a major focus of this software development. Recently, I have written a paper that will appear in the proceedings of the INFORMS Computing Society Conference 2009 : W. Hart, Python Optimization Modeling Objects (Pyomo) , Proc. INFORMS Computing Society Conference, 2009, (to appear). In this paper, I describe Pyomo, an open-source tool for modeling optimization applications in Python. A key goal of Pyomo is to provide an open-source math programming modeling capability. Although open-source optimization solvers are widely available in packages like COIN-OR , surprisingly few open-source tools have been developed to model optimization applications. Pyomo has been developed in Python because it is a well-used modern programming language that provides a robust foundation for developing and applying scientific software. In this paper, I...

Much of my work involves the development of open-source software. Recently, I have been challenged to justify this in several different projects. I recently stumbled across Dave Wheeler's paper , which provides a nice quantitative analysis of the advantages of open-source software.