William E. Hart's Blog

Working with and developing Python software is complicated by several factors: Many users do not have the administrative privileges that are needed to install packages in the Python site packages directory, where Python packages are most easily installed Python developers often need to manage many Python projects in the same development environment The Python virtualenv package resolves these issues by allowing users to generate a local Python installation. Thus, a user can create a Python installation into which they can install Python packages using standard tools like easy_install and pip. Unfortunately, getting started with virtualenv is complicated by the fact that you need to install this package to create virtual Python installations. But, if you do not have administrative privileges, then you are stuck. This is the simplest process that I have seen: hg clone http://bitbucket.org/ianb/virtualenv virtualenv/virtualenv.py ENV Unfortunately, the hg command may not be...

I almost gave the following presentation at the INFORMS Annual Meeting: Installing Python Software Packages: The Good, The Bad and the Ugly That is, I was scheduled to give this talk but my session co-organizer ran over and I had to summarize these slides in 5 minutes! Anyway, these slides describe different strategies for installing Python software.  Although I am a big fan of Python software development, robust strategies for software installation remains a challenge.  This talk describes several different installation scenarios: The Good: the user has administrative privileges Installing on Windows with an installer executable Installing with Linux application utility Installing a Python package from the PyPI repository Installing a Python package from source The Bad: the user does not have administrative privileges Using a virtual environment to isolate package installations Using an installer executable on Windows with a virtual environment The Ugly: the ...

The role of the FAST software repository has evolved over the past few years, and it is time to rethink what the goal of FAST is. See Rethinking the goal of FAST for some perspective about FAST. The upshot is that FAST has been reorganized to support an ensemble of independent software packages related to agile software development. One important change in this reorganization has been the decomposition of the FAST Python software into independent components. Some elements of FAST are deprecated, and the rest have been spread out into various packages. See the FAST Blog for further details.

The gcovr command provides a utility for running the gcov command and summarizing code coverage results. This command is inspired by the Python coverage.py package, which provides a similar utility in Python. Further, gcovr can be viewed as a command-line alternative of the lcov utility, which runs gcov and generates an HTML output. Recently, the gcovr script has been broken out into a separate software package, which is managed at https://software.sandia.gov/svn/public/fast/gcovr This package has been uploaded to the Python PyPI repository to facilitate it installation with easy_install: easy_install gcovr Alternatively, the gcovr script can be downloaded directly: https://software.sandia.gov/svn/public/fast/gcovr/trunk/scripts/gcovr See the gcovr Trac page for further details about this tool. Although gcovr is increasingly used to generate coverage statistics within Hudson, the gcovr wiki documents a command-line text summary that I personally find very useful when developing...

A while back I developed EXACT , a Python package for executing computational experiments using an XML-defined process. EXACT was designed to fill a particular niche in software testing: performing computational tests that involve the application of solvers to a suite of test problems. This sort of testing arises a lot when doing functionality testing for scientific software. Unfortunately, EXACT was too complex: The XML specification was complex and difficult to read Experiments with many factors were assigned generic experiment IDs It was hard to replication the execution of specific experiments The experimental results were captured in XML results files that were difficult to browse Even my close collaborators struggled to setup, run and analyze computational experiments! {sigh} I have recently developed the pyutilib.autotest Python package to provide a simpler alternative to EXACT. This package uses a YAML test configuration file to specify the solvers and problems that are ...

The PyUtilib Component Architecture (PCA) is a component architecture in Python that is derived from the Trac component framework . One important extension was to support both singleton and non-singleton plugins. Trac plugins are singletons that are created immediately when the plugin module is loaded (Python is a wonderful language for supporting this type of capability). Singleton plugins are well-suited for Trac, since it is a persistent application, but many other applications need to employ plugins "on demand". Consequently, the PCA supports non-singleton plugins, which need to be explicitly constructed by the end-user. The PCA is widely used in the Coopr project, and most plugins are non-singletons. Until recently, the PCA did not directly support plugin construction on demand. That is, the user needed to know the plugin class name, and plugin construction was done explicitly by the user. However, most plugins in Coopr can be best described as named services . ...

This is just a heads-up that I have setup blogs for the Coopr and PyUtilib software projects: Coopr: https://software.sandia.gov/trac/coopr/blog PyUtilib: https://software.sandia.gov/trac/pyutilib/blog These blogs will document releases and end-user advice and examples. Additionally, there will hopefully be other authors than just me adding content to these blogs. But, I guess we'll see.

I earlier mentioned that I developed a plugin framework within my PyUtilib software.  This is now called the PyUtilib Component Architecture (PCA).  The PCA is derived from the Trac component architecture, and it supports advanced features like nonsingleton components, namespaces and caching of component interactions. The PCA includes an independent, self-contained framework core that can be easily integrated into other applications, as well as a variety of extension packages with commonly used components. See The PyUtilib Component Architecture for further details.

Coopr is a collection of Python optimization-related packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models. The following are key Coopr capabilities that are driving Coopr development: Pyomo: Formulate algebraic models within Python's modern programming language PySP: Generic solvers for stochastic programming problems COLIN: Scripts that simplify IO between optimizers and black-box applications SUCASA: Customize MIP solvers to expose model structure to the MIP solver engine See https://software.sandia.gov/trac/coopr/wiki/GettingStarted for instructions for getting started with Coopr. An installation script, coopr_install, is provided to simplify the installation of Coopr packages along with the third-party Python packages that they depend on. This installer can also automatically install extension packages from Coin Bazaar. See http://groups.google.com/group/coopr-forum/topics for online discussions of Coopr. Two ...