tag:blogger.com,1999:blog-28143391904888791402024-03-13T03:13:27.683-07:00William E. Hart's BlogLessons learned and musing about software tools, software testing, computational experiments, optimization, operations research, and other interesting stuff that I run across...Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.comBlogger43125tag:blogger.com,1999:blog-2814339190488879140.post-75456110978978648072012-02-01T22:41:00.000-08:002012-02-01T22:43:15.724-08:00Science Fiction Comes to LifeOne of my favorite authors is Vernor Vinge, who explores how the evolution of technology impacts future societies. Vernor is a former professor of computer science, so it is perhaps no surprise that his tech focus appeals to me. The novel <a href="http://en.wikipedia.org/wiki/The_Peace_War">The Peace War</a> imagines a post-apocalyptic society that is recovering from a world war that was prompted, in part, by a new force field generating device (called the Bobbler).<br />
<br />
There are all kinds of futuristic technologies described in this book, which is typical for Vinge's literature. In one scene, the protagonist is about to be ambushed by a group of bad guys. He hands a gun to his teenage companion, who takes out the bad guys without hardly aiming the gun. The trick is that the gun has computer-guided bullets, which the protagonist directed using his laptop.<br />
<br />
This sounded like standard sci-fi tech fantasy until I heard about the self-guided bullet that has been developed by <a href="http://www.sandia.gov/">Sandia National Laboratories</a>. Mashable has a fun video describing the work (<a href="http://news.yahoo.com/self-guided-bullet-strikes-target-mile-away-video-132957445.html">see Yahoo News</a>). Although I work at Sandia, the team working on this didn't invite me to the test range. Perhaps I'll get to help test out the Bobbler. <span style="font-family: Verdana,sans-serif;">;)</span>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-82294535164413878482012-01-20T22:20:00.000-08:002012-01-20T22:20:36.824-08:00Testing Open Source Software<a href="http://en.wikipedia.org/wiki/Software_testing">Software testing</a> is widely recognized as a <a href="http://www.ibm.com/developerworks/websphere/library/techarticles/0306_perks/perks2.html">best practice for software development</a>. Software tests define expected functionality, and they can focus developer efforts by providing an objective assessment of the state of a software project. Additionally, software testing data can provide evidence that a software package can be reliably used. For example, when evaluating whether to try out open source software, I routinely look for software testing data to confirm which platforms the software will run on, the versions of associated software that is used, and test coverage statistics that indicate how much of the the code is tested.<br />
<br />
Unfortunately, most open source software projects do not publish software test data. I suspects that this indicates that a small fraction of OSS projects have robust test suites. However, this also reflects another aspect of the OSS community: hosting facilities for open source software do not support web-based testing facilities, like <a href="http://jenkins-ci.org/">Jenkins</a>, that can be used by developers to remotely launch jobs on test machines with a variety of different configurations. This is not totally unexpected, since testing can be computationally intensive. <br />
<br />
Recently, I learned about <a href="http://www.cloudbees.com/">CloudBees</a>, which provides cloud services for building, running and managing Java applications. Happily <a href="http://www.cloudbees.com/foss">CloudBees makes its Dev@cloud service freely available to open source projects</a>! This includes the Jenkins testing service, which provides a limited number of CPU hours each month for testing an OSS project. For example, the <a href="http://cxxtest.com/">CxxTest</a> project now hosts tests on a <a href="https://cxxtest.ci.cloudbees.com/">CloudBees Jenkins server</a>. Cool!Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-43014351188284617282012-01-08T07:34:00.000-08:002012-01-08T07:34:48.579-08:00A Different Model for Writing Blog Posts<style>
<!--
/* Font Definitions */
@font-face
{font-family:"MS 明朝";
panose-1:0 0 0 0 0 0 0 0 0 0;
mso-font-charset:128;
mso-generic-font-family:roman;
mso-font-format:other;
mso-font-pitch:fixed;
mso-font-signature:1 134676480 16 0 131072 0;}
@font-face
{font-family:"MS 明朝";
panose-1:0 0 0 0 0 0 0 0 0 0;
mso-font-charset:128;
mso-generic-font-family:roman;
mso-font-format:other;
mso-font-pitch:fixed;
mso-font-signature:1 134676480 16 0 131072 0;}
@font-face
{font-family:Cambria;
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0in;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:Cambria;
mso-ascii-font-family:Cambria;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"MS 明朝";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Cambria;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:Cambria;
mso-ascii-font-family:Cambria;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"MS 明朝";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Cambria;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<br />
<div class="MsoNormal">
This is a blog that I have been meaning to write for some
time.<span style="mso-spacerun: yes;"> </span>I occasionally take a look at the
download statistics for this blog, and recently I was prompted to do this by
other bloggers who were reporting their end-of-year statistics (e.g. see <a href="http://punkrockor.wordpress.com/2012/01/02/year-in-review/">Laura McLay’s review of the Punk Rock OR blog</a>). </div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Unlike Laura, I do not have impressive download statistics
to report about the many blogs I have written in 2011; frankly, I did not
create many posts. However, an interesting pattern has emerged regarding this
blog’s readership:<span style="mso-spacerun: yes;"> </span>there are a few key
blog posts that are frequently downloaded.<span style="mso-spacerun: yes;">
</span>For example, my most frequently downloaded blog post is a survey of
Python plugin software, which I wrote in 2009.<span style="mso-spacerun: yes;">
</span>I suspect that other bloggers have seen the same thing; they have a few
posts that are very popular because people do web searches on that topic.<span style="mso-spacerun: yes;"> </span>However, it is worth stepping back and
thinking about the implications of this when writing a blog.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
When I first started blogging, I imagined that readers would
view my blog the way that I view Laura’s blog.<span style="mso-spacerun: yes;">
</span>They would use a RSS feeder to collect and view blog updates.<span style="mso-spacerun: yes;"> </span>These would be read shortly after they were
published, and afterwards they might be used as a reference.<span style="mso-spacerun: yes;"> </span>This led me to create blogs that referenced
each other as part of a larger conversation on a topic.<span style="mso-spacerun: yes;"> </span>For example, after blogging about <a href="http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html">Python Plugin Frameworks</a>, I had several
follow-up blogs, including a brief description of <a href="http://wehart.blogspot.com/2009/07/pyutilib-plugins.html">PyUtilib Plugins</a> that I had developed.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
However, I have realized that my blogs are more likely to be
found through internet searches focused on a topic.<span style="mso-spacerun: yes;"> </span>Consequently, the <a href="http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html">Python Plugin Frameworks</a>
post gets frequently read while the <a href="http://wehart.blogspot.com/2009/07/pyutilib-plugins.html">PyUtilib Plugins</a> post rarely gets
read.<span style="mso-spacerun: yes;"> </span>Readers are finding my blog posts
after searching for “python plugins”. The narrower topic covered by the
<a href="http://wehart.blogspot.com/2009/07/pyutilib-plugins.html">PyUtilib Plugins</a> post is not frequently referenced on the internet, and
consequently it is not strongly associated with the more general topic of
Python plugins; for example, I did not see it in the first three pages of a
google search for “python plugin”.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This suggests a different model for writing blog posts that
has already begun to affect my blogging.<span style="mso-spacerun: yes;">
</span>Since blog posts are individual artifacts that may have enduring value
to readers, updating a blog post with new content makes more sense than
creating a new post that continues the previous discussion.<span style="mso-spacerun: yes;"> </span>For example, I’ve updated the <a href="http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html">Python PluginFrameworks</a> post to include references to PyUtilib’s plugins.<span style="mso-spacerun: yes;"> </span>This may confuse readers of RSS feeds, and I
do not know that RSS feeds will automatically update their feed to capture updates
like this. I would assume not.<span style="mso-spacerun: yes;"> </span>However,
this is clearly a strategy that will enhance the long-term impact of a blog
post on a specific topic.</div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com3tag:blogger.com,1999:blog-2814339190488879140.post-41830207308914671762012-01-07T22:09:00.000-08:002012-01-22T13:52:36.632-08:00The Pyomo Book is Coming SoonThe Python Optimization Modeling Objects (Pyomo) package is an open source tool for modeling optimization applications in Python. Pyomo can be used to define symbolic problems, create concrete problem instances, and solve these instances with standard solvers. Pyomo provides a capability that is commonly associated with algebraic modeling languages such as AMPL, AIMMS, and GAMS, but Pyomo's modeling objects are embedded within a full-featured high-level programming language with a rich set of supporting libraries. Pyomo leverages the capabilities of the Coopr software library, which integrates Python packages for defining optimizers, modeling optimization applications, and managing computational experiments.<br />
<br />
Of course, there is very little online documentation describing Pyomo. However, the first book on Pyomo is set to be published in February!<br />
<blockquote class="tr_bq">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://images.springer.com/cda/content/image/cda_displayimage.jpg?SGWID=0-0-16-1227219-0" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="http://images.springer.com/cda/content/image/cda_displayimage.jpg?SGWID=0-0-16-1227219-0" width="132" /></a></div>
<i>Pyomo - Optimization Modeling in Python</i>. William E. Hart, Carl Laird, Jean-Paul Watson and David L. Woodruff. Springer, 2012.</blockquote>
Here are some links if you want to learn more:<br />
<ul>
<li><a href="http://www.amazon.com/Pyomo-Optimization-Modeling-Springer-Applications/dp/1461432251">Amazon web page</a></li>
<li><a href="http://www.springer.com/mathematics/book/978-1-4614-3225-8">Springer web page</a></li>
</ul>
Enjoy!Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-49166098960226556752012-01-07T21:25:00.000-08:002012-01-07T21:36:55.146-08:00A Pythonic C++ ParserIf you google for "python C++ parser", you will find a variety of internet discussions related to parsing C++ in Python. C++ cannot be parsed by a LALR parser and it is well-known that parsing C++ is a nontrivial task. Thus, these discussions generally fall into one of several categories:<br />
<ul>
</ul>
<ol>
<li>It is too hard to parse C++ in Python, so use a package like <a href="http://www.gccxml.org/HTML/Index.html">GCC_XML</a> that does this for you. If you really need to do something in Python, write a wrapper to GCC_XML.</li>
<li>It is too hard to perform a complete parse of C++ in Python, but we can use a LALR parser to collect gross structural information from C++ files. The <a href="http://pypi.python.org/pypi/CppHeaderParser/">CppHeaderParser</a> is an example of this type of package, which uses the <a href="http://www.dabeaz.com/ply/">ply</a> parser to collect information about classes in header files. </li>
</ol>
<ul>
</ul>
In the recent release of <a href="http://cxxtest.com/">CxxTest</a>, I included a LALR C++ parser that is similar to <a href="http://pypi.python.org/pypi/CppHeaderParser/">CppHeaderParser</a>. CxxTest is a unit testing framework for C++ that is similar in
spirit to JUnit, CppUnit, and xUnit. CxxTest is easy to use because it
does not require precompiling a CxxTest testing library, it employs no
advanced features of C++ (e.g. RTTI) and it supports a very flexible
form of test discovery.<br />
<br />
CxxTest performs test discovery by searching C++
header files for CxxTest test classes. The default process for
test discovery is a simple process that analyzes each line in a
header file sequentially, looking for a sequence of lines that
represent class definitions and test method definitions.
<br />
<br />
I added a new test discovery mechanism in CxxTest 4.0 that is based
on the a parser for the
<a href="http://www.computing.surrey.ac.uk/research/dsrg/fog/">Flexible Object
Generator (FOG)</a> language, which is a superset of C++. The grammar
for the FOG language was adapted to parse C++ header files to
identify class definitions and class inheritance relationships,
class and namespace nesting of declarations, and class methods.
This allows CxxTest to identify test classes that are defined
with complex inheritance relationships.
<br />
<p>
As I noted earlier, the CxxTest FOG parser is similar to the parser in CppHeaderParser. Based on my limited knowledge of CppHeaderParser, here are some points of contrast between these two capabilities:<br />
<ol>
<li>The FOG parser is embedded in CxxTest, while the CppHeaderParser is a stand-alone package. Although I implemented the FOG parser as a separate component in CxxTest, I did not have specific design requirements that led me to make this a separate package. (Interested parties should give me a buzz...)</li>
<li>The FOG parser is a specifically focused on the features required by CxxTest, and thus it does not parse out much of the information that CppHeaderParser provides (return values, argument types, etc).</li>
<li>The FOG parser was specifically designed to capture class inheritance relationships. It is not clear to me that the CppHeaderParser does this.</li>
<li>The FOG parser is based on a superset of C++. Thus, it can robustly parse C++ method and function definitions. The examples provided by CppHeaderParser suggest that it can parser function and method declarations, but not headers that include their definitions. (Of course, the FOG parser ignores these definitions, but that's the point. The parser can do that.)</li>
<li>The FOG parser has been tested on a large set of C and C++ test files that are used to test the <a href="http://scottmcpeak.com/elkhound/sources/elsa/">ELSA</a> compiler. This is a much more extensive test suite than is used to develop CppHeaderParser.</li>
</ol>
The point of this comparison is that the FOG parser may be of interest for other C++ parsing applications. It has not been developed for general use, but it could easily be adapted to provide a more general capability. <br />
<ol>
</ol>
</p>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com1tag:blogger.com,1999:blog-2814339190488879140.post-21322179073428104682011-10-05T23:14:00.000-07:002011-10-05T23:14:22.796-07:00OptimJ is now free, but Ateji is closed<a href="http://www.ateji.com/">Atjei</a>, the creater of OptimJ, is now closed. The OptimJ product is a Java extension that supports a simplified syntax for specifying optimization models. It is sad to see this product be abandoned like this. I do not know of any other optimization modeling tool that directly supports Java.<br />
<br />
<a href="http://yetanothermathprogrammingconsultant.blogspot.com/2011/10/ateji-defunct-optimj-now-free.html">Erwin Kalvelagen</a> noted the demise of Ajeti, and commented that "<span class="Apple-style-span" style="color: #333333; font-family: Verdana, Arial, sans-serif; font-size: 13px; line-height: 16px;">I believe that some of the more complex data issues in practical modeling are often better (i.e. more efficiently) dealt with in a specialized language than in a traditional programming language."</span> Pyomo is similar to OptimJ, in that it supports optimization modeling in Python. However, a premise that has guided Pyomo development is that users <i>will</i> want to perform modeling in a full-featured programming language.<br />
<br />
Clearly, traditional programming languages are more verbose and complex than a domain-specific language. However, Python is arguably <i>much</i> simpler to use than Java. In fact, I have gotten feedback from Pyomo users that suggests that they were not aware that they were developing models in Python; they simply thought that it was another modeling language!<br />
<br />
Of course, I am a programmer who likes to do optimization. My selfish reason for creating Pyomo is that I find domain-specific languages quite constraining. In fact, I had thought that the use of Python for modeling and optimization would be attractive to optimization researchers. Python is a great language for prototyping a complex idea, usually without losing too much performance. I and other Pyomo developers have implemented complex optimizers in Python that directly interface with Pyomo models, and in most cases the runtime is dominated by LP subproblems. Thus, there has been little motivation to develop custom, highly-optimized codes in languages like C++ based on these solvers.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-7395896210148334332011-09-23T22:26:00.000-07:002013-05-05T13:51:07.258-07:00Using AsciiDoc for Mathematical PublicationsTechnical writing is an integral part of my research in computer science and operations research. I have a long history using LaTeX, which is very well suited for writing technical articles that contain mathematical equations as well as code snippets. Although LaTeX can readily generate postscript and PDF output files, I have been unimpressed with tools that generate HTML from LaTeX source. Thus, I was intrigued by <a href="http://www.methods.co.nz/asciidoc/">AsciiDoc</a>, which promises to generate PDF, HTML and eBook formats. AsciiDoc is used to provide online documentation for software projects, and authors can publish book through O'Rielly using this tool. Thus, this is a well-developed document generation tool.<br />
<br />
I have successfully prototyped a draft book, <i>Getting Started with Coopr</i>, and you can browse <a href="https://software.sandia.gov/svn/public/coopr/coopr.doc/trunk/GettingStarted/current">the subversion repository for this document here</a>. Note that the <b>Makefile</b> file specifies build targets for PDF, HTML and eBook files.<br />
<br />
The advantage of AsciiDoc is that you can use a simple markup language to generate complex documents in a variety of formats. Since this is a generic document-generation process, it is reasonable to expect that there will be limited control of document formatting. (If you want a lot of control, you should just use LaTeX!) However, there are several major limitations to the document generation and format control that limit what you can do with AsciiDoc:<br />
<br />
<ul>
<li><u>Portable Mathematical Equations:</u> There is only limited support for generating eBook documents that contain mathematical equations. I noticed that the ePUB standard was just updated this month to support MathML, so it is not clear that e-readers can handline MathML right now. Additionally, AsciiDoc does not support the generation of the MathML XML from a high-level description (e.g. LaTeX math equations). Thus, a user cannot easily prepare a document that generates both PDF (using LaTeX under the hood) and ePUB (using MathML under the hood). I guess we will have to wait a few more years to see robust publishing of mathematics for eBooks.<br /><br /> </li>
<li><u>Formatting Mathematics:</u> For whatever reason, the default formatting of mathematical environments in HTML is not centered or indented (as it is in LaTeX). Thus, it is much more difficult to read HTML documents containing mathematics. I tried resolving this using an AsciiDoc filter, without luck. I wound up rewriting the LatexMath macros to enforce this different formatting in HTML. Unfortunately, these revised macros do not precisely match the syntax used by AsciiDoc. {sigh}<br /> <br /> </li>
<li><u>Document Authors:</u> The AsciiDoc markup language does not provide a convenient way to create a document with multiple authors. Yes, I am not kidding. There is a docbook configuration file that you can provide, which only works if the document generation process goes through docbook; in my example, that works for ePUB and PDF files. Thus, there does not appear to be a single, portable way for specifying multiple authors.<br /> </li>
<li><u>Citations</u>: It is noteworthy that none of the examples of online books referenced in the online AsciiDoc documentation contain citations or a bibliography. The default format for bibliographies in AsciiDoc PDF files is as a numbered chapter or section, which differs from the normal convention in LaTeX (which I much prefer). Thus, my AsciiDoc book uses the <i>colophon</i> section, which is not numbered. However, that means that it does not show up in the table of contents. {sigh}<br /><br />Another issue with citations is that the examples provided by AsciiDoc do not correctly generate hyperlinks in the PDF file. Basically, the <i>bibliography</i> section type provided by AsciiDoc does not work well with the <b>dblatex</b> tool used to generate the PDF. My solution was to not use the <i>bibliography</i> section type!<br /><br />Finally, the examples provided by AsciiDoc include citations in a list environment, which means that the PDF output contains a numbered list followed by a bracket citation reference. Again, my solution was to avoid using the list; the bibliography is simply a sequence of paragraphs, each of which is a citation with its associated anchor.</li>
</ul>
Despite these issues, I am planning to continue developing the Coopr documentation with AsciiDoc. The lack of support for mathematical equations is a problem for ePUB documents, but most readers will be using this document to refer to the examples in python. However, I would not consider using AsciiDoc for developing more complex documents, like a book intended for publication. There is too much customization that would be needed to get past the current limitations.<br />
<br />
<br />
<ul>
</ul>
Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com5tag:blogger.com,1999:blog-2814339190488879140.post-23153109234538853012011-08-05T06:03:00.000-07:002011-08-05T06:04:22.982-07:00Coopr Download FunI stumbled across the following site, which provides download statistics for PyPI Python optimization packages:<br />
<br />
<a href="http://taichino.appspot.com/pypi_ranking/keyword/optimization">http://taichino.appspot.com/pypi_ranking/keyword/optimization</a><br />
<br />
It was fun to see the different download package statistics. I'm quite curious that coopr.pysos has the highest number of downloads, since IMHO this package doesn't have much interesting functionality... Hmmm...<br />
<br />
Another surprise for me is that so many optimization Python packages are _not_ on this list! I am maintaining the following list of links for Python optimization packages:<br />
<br />
<a href="https://software.sandia.gov/trac/coopr/wiki/Documentation/RelatedProjects">https://software.sandia.gov/trac/coopr/wiki/Documentation/RelatedProjects</a><br />
<br />
Perhaps these packages were not tagged with the 'optimization' label ... or perhaps they were not downloaded enough to make the list?Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-13470656774387540842011-07-23T09:25:00.000-07:002011-07-23T09:25:36.128-07:00Python Optimization PackagesI have been maintaining a list of Python optimization packages for a while now on the Coopr Trac pages: see <a href="https://software.sandia.gov/trac/coopr/wiki/Documentation/RelatedProjects">https://software.sandia.gov/trac/coopr/wiki/Documentation/RelatedProjects</a>.Today, I noticed that if you google for "python optimization packages", this page does not show up right away. Perhaps Trac pages are index differently? I'm not sure.<br />
<br />
Anyway, I thought I'd add this reference here to help others find this list that I'm maintaining...Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com3tag:blogger.com,1999:blog-2814339190488879140.post-10379839742094753512011-07-23T09:21:00.000-07:002011-07-23T09:26:17.393-07:00Coopr 3.0.4362 ReleaseWe are pleased to announce the release of Coopr 3.0 (3.0.4362). Coopr is a collection of Python software packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models.<br />
<br />
The following are highlights of this release:<br />
<br />
- Solvers<br />
* More sophisticated logic for solver factory to find ASL and OS solvers<br />
* Various solver interface improvements<br />
* New Solver results object for efficient representation of variable values<br />
* New support for asynchronous progressive hedging<br />
<br />
- Modeling<br />
* Changes in rule semantics to limit rule return values<br />
* Changes in the expected order of rule arguments<br />
* Constant sums or products can now be used as constraint bounds<br />
* Added full support for the !ConstraintList modeling component.<br />
<br />
- Usability enhancements<br />
* More explicit output from runph and runef commands<br />
* Added support in runef to write the extensive form in NL format<br />
* Add controls for garbage collection in PH<br />
<br />
- Other<br />
* Efficiency improvements in generation of NL and LP files.<br />
* Significant efficiency improvements in parsing of Pyomo Data Files.<br />
* More robust MS Windows installer (does not use virtual python<br />
* environment)<br />
<br />
Note that this is a major release of Coopr that changes the expected formulation of Coopr models. See the Coopr blog for further details about deprecated functionality, which will be disabled in future releases.<br />
<br />
See <a href="https://software.sandia.gov/trac/coopr/wiki/GettingStarted">https://software.sandia.gov/trac/coopr/wiki/GettingStarted</a> for instructions for getting started with Coopr. Installers are available for MS Windows and Unix operating systems to simplify the installation of Coopr packages along with the third-party Python packages that they depend on. These installers can also automatically install extension packages from Coin Bazaar.<br />
<br />
Enjoy!<br />
<br />
- Coopr Developer Team<br />
- <a href="mailto:coopr-developers@googlecode.com">coopr-developers@googlecode.com</a><br />
- <a href="https://software.sandia.gov/trac/coopr/wiki/Documentation/Developers">https://software.sandia.gov/trac/coopr/wiki/Documentation/Developers</a><br />
<br />
<br />
-----------<br />
About Coopr<br />
-----------<br />
<br />
Coopr is a collection of Python software packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models.<br />
<br />
A key driver for Coopr development is Pyomo, an open source tool for modeling optimization applications in Python. Pyomo can be used to define symbolic problems, create concrete problem instances, and solve these instances with standard solvers. Thus, Pyomo provides a capability that is commonly associated with algebraic modeling languages like AMPL and GAMS.<br />
<br />
Coopr has also proven an effective framework for developing high-level optimization and analysis tools. For example, the PySP package provides generic solvers for stochastic programming. PySP leverages the fact that Pyomo's modeling objects are embedded within a full-featured high-level programming language, which allows for transparent parallelization of subproblems using Python parallel communication libraries.<br />
<br />
Coopr development is hosted by Sandia National Laboratories and COIN-OR:<br />
<br />
* <a href="https://projects.coin-or.org/Coopr">https://projects.coin-or.org/Coopr</a><br />
* <a href="https://software.sandia.gov/coopr">https://software.sandia.gov/coopr</a> <br />
<br />
See <a href="http://groups.google.com/group/coopr-forum/">http://groups.google.com/group/coopr-forum/</a> for online discussions of Coopr.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-35591172519084172232011-03-28T08:58:00.000-07:002011-03-28T08:58:17.551-07:00<!--[if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:TrackMoves/> <w:TrackFormatting/> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:DoNotPromoteQF/> <w:LidThemeOther>EN-US</w:LidThemeOther> <w:LidThemeAsian>X-NONE</w:LidThemeAsian> <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> <w:SplitPgBreakAndParaMark/> <w:DontVertAlignCellWithSp/> <w:DontBreakConstrainedForcedTables/> <w:DontVertAlignInTxbx/> <w:Word11KerningPairs/> <w:CachedColBalance/> </w:Compatibility> <w:DoNotOptimizeForBrowser/> <m:mathPr> <m:mathFont m:val="Cambria Math"/> <m:brkBin m:val="before"/> <m:brkBinSub m:val="--"/> <m:smallFrac m:val="off"/> <m:dispDef/> <m:lMargin m:val="0"/> <m:rMargin m:val="0"/> <m:defJc m:val="centerGroup"/> <m:wrapIndent m:val="1440"/> <m:intLim m:val="subSup"/> <m:naryLim m:val="undOvr"/> </m:mathPr></w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267"> <w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/> <w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/> <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/> <w:LsdException Locked="false" Priority="39" Name="toc 1"/> <w:LsdException Locked="false" Priority="39" Name="toc 2"/> <w:LsdException Locked="false" Priority="39" Name="toc 3"/> <w:LsdException Locked="false" Priority="39" Name="toc 4"/> <w:LsdException Locked="false" Priority="39" Name="toc 5"/> <w:LsdException Locked="false" Priority="39" Name="toc 6"/> <w:LsdException Locked="false" Priority="39" Name="toc 7"/> <w:LsdException Locked="false" Priority="39" Name="toc 8"/> <w:LsdException Locked="false" Priority="39" Name="toc 9"/> <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/> <w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/> <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/> <w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/> <w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/> <w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/> <w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/> <w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/> <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/> <w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/> <w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/> <w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/> <w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/> <w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/> <w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/> <w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/> <w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/> <w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/> <w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/> <w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/> <w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/> <w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/> <w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/> <w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/> <w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/> <w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/> <w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/> <w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/> <w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/> <w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/> <w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/> <w:LsdException Locked="false" Priority="37" Name="Bibliography"/> <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/> </w:LatentStyles> </xml><![endif]--><!--[if gte mso 10]> <style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
</style> <![endif]--> <br />
<div class="MsoPlainText"><a href="https://software.sandia.gov/trac/coopr/blog/20110325-Coopr2.5.3890">Coopr 2.5(2.5.3890) has just been released</a>! Coopr is a collection of Python software packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models.</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">The following are highlights of this release:</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">- Solvers</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* MIP solver interface updates to use appropriate objective names</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Added support for suffixes in GUROBI solver interface</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Improved diagnostic analysis of PH solver for the extensive form</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">- Usability enhancements</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Improved robustness of coopr_install</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Fixed Coopr installation problem when using easy_install</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Added a script to launch the CooprAge GUI.</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* LP files now are written with the true objective name</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Rework of pyomo command line to create a concise output</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Many efficiency improvements during model generation!</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Many improvements to diagnostic output and error handling</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Expressions like "model.p > 1" can now be used within generation rules</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">- Modeling</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Added support for generalized disjunctive programs (in coopr.gdp)</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Constraints can now be specified in "compound" form:<span style="mso-spacerun: yes;"> </span>lb <= expr <= ub</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Significant robustness enhancements for model expressions</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Improved error handling for constraint generation</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">- Other</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Python 2.5 is deprecated due to performance issues</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* Python versions 2.6 and 2.7 are supported</div><div class="MsoPlainText"><span style="mso-spacerun: yes;"> </span>* New MS Windows installer is now available</div><div class="MsoPlainText"><br />
</div><div class="MsoPlainText">See the <a href="https://software.sandia.gov/trac/coopr">Coopr wiki</a> for further details!</div><div class="MsoPlainText"> </div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-25309083163728099392010-11-26T16:59:00.000-08:002010-11-26T16:59:22.895-08:00Getting Started with a Virtual Python InstallationWorking with and developing Python software is complicated by several factors: <br />
<ul><li>Many users do not have the administrative privileges that are needed to install packages in the Python site packages directory, where Python packages are most easily installed </li>
<li>Python developers often need to manage many Python projects in the same development environment </li>
</ul>The Python <a href="http://virtualenv.openplans.org/">virtualenv</a> package resolves these issues by allowing users to generate a local Python installation. Thus, a user can create a Python installation into which they can install Python packages using standard tools like easy_install and pip.<br />
<br />
Unfortunately, getting started with virtualenv is complicated by the fact that you need to install this package to create virtual Python installations. But, if you do not have administrative privileges, then you are stuck. This is the simplest process that I have seen:<br />
<pre class="wiki">hg clone http://bitbucket.org/ianb/virtualenv
virtualenv/virtualenv.py ENV</pre>Unfortunately, the <tt>hg</tt> command may not be preinstalled on your machine. It is not available on any of mine...<br />
<br />
The <a href="https://software.sandia.gov/trac/pyutilib/browser/pyutilib.virtualenv">pyutilib.virtualenv</a> package contains the <a href="https://software.sandia.gov/trac/pyutilib/export/HEAD/pyutilib.virtualenv/stable/scripts/vpy_install">vpy_install</a> script, which can bootstrap the setup of virtualenv. This script extends the functionality of the virtualenv driver script to allow direct, and it customization of this initial environment. The <a class="source" href="https://software.sandia.gov/trac/pyutilib/export/HEAD/pyutilib.virtualenv/stable/scripts/vpy_install">vpy_install</a> script can be easily downloaded from the <a href="https://software.sandia.gov/trac/pyutilib/wiki/Documentation/vpy">PyUtilib wiki</a>, or it can be downloaded with commands like wget or curl:<br />
<pre class="wiki">wget http://doiop.com/pyutilib/vpy_install
vpy_install ENV</pre>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-77391858091882256902010-11-10T21:22:00.000-08:002010-11-10T21:23:30.590-08:00Installing Python Software Packages: The Good, The Bad and the UglyI almost gave the following presentation at the INFORMS Annual Meeting:<br />
<br />
<div style="text-align: center;"><a href="http://www.slideshare.net/WilliamHart1/10-11hart-installing-pythonsoftware">Installing Python Software Packages: The Good, The Bad and the Ugly</a></div><br />
That is, I was scheduled to give this talk but my session co-organizer ran over and I had to summarize these slides in 5 minutes!<br />
<br />
Anyway, these slides describe different strategies for installing Python software. Although I am a big fan of Python software development, robust strategies for software installation remains a challenge. This talk describes several different installation scenarios:<br />
<br />
The Good: the user has administrative privileges<br />
<ol><li>Installing on Windows with an installer executable</li>
<li>Installing with Linux application utility</li>
<li>Installing a Python package from the PyPI repository</li>
<li>Installing a Python package from source</li>
</ol><br />
The Bad: the user does not have administrative privileges<br />
<ol><li>Using a virtual environment to isolate package installations</li>
<li>Using an installer executable on Windows with a virtual environment</li>
</ol><ol></ol><br />
The Ugly: the user needs to install an extension package from source<br />
<ol><li>Installing a Python extension package from source</li>
<li>PyCoinInstall – Managing builds for Python extension packages</li>
</ol>The last item referring to PyCoinInstall describes a utility being developed for the COIN-OR software, which is used within the operations research community. COIN-OR includes a variety of Python and C++ software packages, and this script uses a simple plug-in system to support the management of package builds and installation. Of course, this mostly does the easy packages right now...<br />
<br />
While preparing these slides I realized that the vpy_install script I use for setting up virtual environments is probably of broader interest. I'll say more about this in a little bit. In the meantime, checkout the slides!Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-64068957893852485192010-08-31T19:31:00.000-07:002010-08-31T19:39:47.689-07:00Reworking FASTThe role of the FAST software repository has evolved over the past few years, and it is time to rethink what the goal of FAST is. See <a href="https://software.sandia.gov/trac/fast/blog/20100815-RethinkingFast">Rethinking the goal of FAST</a> for some perspective about FAST. The upshot is that FAST has been reorganized to support an ensemble of independent software packages related to agile software development.<br /><br />One important change in this reorganization has been the decomposition of the FAST Python software into independent components. Some elements of FAST are deprecated, and the rest have been spread out into various packages. See the <a href="https://software.sandia.gov/trac/fast/blog/20100815-RethinkingFast">FAST Blog</a> for further details.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-85192030051058142742010-08-29T13:20:00.000-07:002010-08-31T19:55:51.897-07:00Update for gcovrThe <a href="https://software.sandia.gov/trac/fast/wiki/Documentation/gcovr">gcovr</a> command provides a utility for running the <a href="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html">gcov</a> command and summarizing code coverage results. This command is inspired by the Python <a href="http://nedbatchelder.com/code/coverage/">coverage.py</a> package, which provides a similar utility in Python. Further, <a href="https://software.sandia.gov/trac/fast/wiki/Documentation/gcovr">gcovr</a> can be viewed as a command-line alternative of the <a href="http://ltp.sourceforge.net/coverage/lcov.php">lcov</a> utility, which runs gcov and generates an HTML output.<br /><br />Recently, the gcovr script has been broken out into a separate software package, which is managed at<br /><br /><div style="text-align: center;">https://software.sandia.gov/svn/public/fast/gcovr<br /></div><br />This package has been uploaded to the Python PyPI repository to facilitate it installation with easy_install:<br /><br /><div style="text-align: center;">easy_install gcovr<br /></div><br />Alternatively, the gcovr script can be downloaded directly:<br /><br /><div style="text-align: center;"><a href="https://software.sandia.gov/svn/public/fast/gcovr/trunk/scripts/gcovr">https://software.sandia.gov/svn/public/fast/gcovr/trunk/scripts/gcovr</a><br /></div><br />See the <a href="https://software.sandia.gov/trac/fast/wiki/gcovr">gcovr Trac page</a> for further details about this tool. Although gcovr is increasingly used to generate coverage statistics within Hudson, the gcovr wiki documents a command-line text summary that I personally find very useful when developing tests to improve code coverage.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-14468849966547305022010-06-28T21:19:00.000-07:002010-06-28T21:48:19.070-07:00A New Python Package: pyutilib.autotestA while back I developed <a href="https://software.sandia.gov/trac/fast/wiki/Documentation/exact">EXACT</a>, a Python package for executing computational experiments using an XML-defined process. EXACT was designed to fill a particular niche in software testing: performing computational tests that involve the application of solvers to a suite of test problems. This sort of testing arises a lot when doing functionality testing for scientific software.<br /><br />Unfortunately, EXACT was too complex:<br /><ul><li>The XML specification was complex and difficult to read</li><li>Experiments with many factors were assigned generic experiment IDs<br /></li><li>It was hard to replication the execution of specific experiments</li><li>The experimental results were captured in XML results files that were difficult to browse</li></ul>Even my close collaborators struggled to setup, run and analyze computational experiments! {sigh}<br /><br />I have recently developed the <a href="https://software.sandia.gov/trac/pyutilib/browser/pyutilib.autotest">pyutilib.autotest</a> Python package to provide a simpler alternative to EXACT. This package uses a <a href="http://yaml.org/">YAML</a> test configuration file to specify the solvers and problems that are exercised in a test. The tests setup and executed with Python's unittest package, which allows the user to apply a rich set of testing methods. The user can specify a test driver in Python that defines how each test is executed, using the solver and problem options that are specified in the configuration file.<br /><br />Here's a simple example of a YAML test specification, where the goal is to apply the UNIX cat command with different options to different test files:<pre>driver: example<br /><br />solvers:<br /> cat:<br /> cat2:<br /> name: cat<br /> cat_options: -n<br /><br />problems:<br /> f1:<br /> file: file1.txt<br /> f2:<br /> file: file2.txt<br /><br />suites:<br /> suite1:<br /> solvers:<br /> cat:<br /> cat2:<br /> problems:<br /> f1:<br /> f2:<br /></pre>See the <a href="http://pypi.python.org/pypi/pyutilib.autotest">pyutilib.autotest PyPI documentation</a> for further details.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-19236260411434512832010-06-21T14:24:00.000-07:002010-06-21T14:40:03.731-07:00Dynamic Service Creation in the PyUtilib Component ArchitectureThe <a href="https://software.sandia.gov/trac/pyutilib/export/1831/pyutilib.component.doc/trunk/doc/plugin/pca.pdf">PyUtilib Component Architecture (PCA)</a> is a component architecture in Python that is derived from the <a href="http://trac.edgewall.org/wiki/TracDev/ComponentArchitecture">Trac component framework</a>. One important extension was to support both singleton and non-singleton plugins. Trac plugins are singletons that are created immediately when the plugin module is loaded (Python is a wonderful language for supporting this type of capability). Singleton plugins are well-suited for Trac, since it is a persistent application, but many other applications need to employ plugins "on demand". Consequently, the PCA supports non-singleton plugins, which need to be explicitly constructed by the end-user.<br /><br />The PCA is widely used in the <a href="https://software.sandia.gov/trac/coopr">Coopr</a> project, and most plugins are non-singletons. Until recently, the PCA did not directly support plugin construction on demand. That is, the user needed to know the plugin class name, and plugin construction was done explicitly by the user. However, most plugins in Coopr can be best described as <span style="font-style: italic;">named services</span>. For example, Coopr optimizers are implemented as plugins and each optimizer has a unique name.<br /><br />The PCA has recently been extended to support the <span style="font-weight: bold;font-size:85%;" ><span style="font-family:courier new;">alias</span></span> function, which can be used to declare the name of a plugin with respect to a particular interface. For example, consider the following definitions of the <span style="font-weight: bold;font-size:85%;" ><span style="font-family:courier new;">MyPlugin</span></span> and <span style=";font-family:courier new;font-size:85%;" ><span style="font-weight: bold;">YourPlugin</span></span> plugin classes:<br /><pre><br />class ITask(Interface):<br /><br /> def print(self):<br /><br />class MyPlugin(Plugin):<br /><br /> implements(ITask)<br /> alias('my', ITask)<br /><br /> def print(self):<br /> print "My Plugin"<br /><br />class YourPlugin(Plugin):<br /><br /> implements(ITask)<br /> alias('your', ITask)<br /><br /> def print(self):<br /> print "Your Plugin"<br /></pre><br />These plugins are non-singletons, so they are only registered and used if they are constructed. The <span style="font-weight: bold;font-size:85%;" ><span style="font-family:courier new;">alias</span></span> function allows these plugins to be constructed with a factory class that is supported by the PCA. For example:<br /><pre><br />factory = CreatePluginFactory(ITask)<br /><br />p1 = factory('my')<br />p2 = factory('your')<br /></pre><br />The <span style="font-weight: bold;font-size:85%;" ><span style="font-family:courier new;">CreatePluginFactory</span></span> class is provided by the PCA, and it provides a functor that manages the creation of plugins whose names are registered with the <span style="font-weight: bold;font-size:85%;" ><span style="font-family:courier new;">alias</span></span> function.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-84415895556408292902010-06-21T12:19:00.000-07:002010-06-21T12:32:54.818-07:00Blogs for Coopr and PyUtilibThis is just a heads-up that I have setup blogs for the <a href="https://software.sandia.gov/trac/coopr">Coopr</a> and <a href="https://software.sandia.gov/trac/pyutilib">PyUtilib</a> software projects:<br /><ul><li>Coopr: <a href="https://software.sandia.gov/trac/coopr/blog">https://software.sandia.gov/trac/coopr/blog</a><br /></li><li>PyUtilib: <a href="https://software.sandia.gov/trac/pyutilib/blog">https://software.sandia.gov/trac/pyutilib/blog</a><br /></li></ul>These blogs will document releases and end-user advice and examples. Additionally, there will hopefully be other authors than just me adding content to these blogs. But, I guess we'll see.Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-48274940939456266992010-04-16T14:19:00.000-07:002010-04-16T14:20:40.344-07:00PyUtilib Component Architecture<div xmlns='http://www.w3.org/1999/xhtml'>I earlier mentioned that I developed a plugin framework within my PyUtilib software. This is now called the PyUtilib Component Architecture (PCA). The PCA is derived from the Trac component architecture, and it supports advanced features like nonsingleton components, namespaces and caching of component interactions. The PCA includes an independent, self-contained framework core that can be easily integrated into other applications, as well as a variety of extension packages with commonly used components.<br/><br/>See <a href='https://software.sandia.gov/trac/pyutilib/export/1754/pyutilib.component.doc/trunk/doc/plugin/pca.pdf'>The PyUtilib Component Architecture</a> for further details.</div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-65372990995742852482010-04-16T14:07:00.001-07:002010-04-16T14:12:38.896-07:00Recent Coopr Developments<div xmlns="http://www.w3.org/1999/xhtml"><p>Coopr is a collection of Python optimization-related packages that supports a diverse set of optimization capabilities for formulating and analyzing optimization models. The following are key Coopr capabilities that are driving Coopr development:<br /></p><ul><li>Pyomo: Formulate algebraic models within Python's modern programming language </li><li>PySP: Generic solvers for stochastic programming problems<br /></li><li>COLIN: Scripts that simplify IO between optimizers and black-box applications<br /></li><li>SUCASA: Customize MIP solvers to expose model structure to the MIP solver engine<br /></li></ul><p>See <a href="http://www.google.com/url?sa=D&q=https://software.sandia.gov/trac/coopr/wiki/GettingStarted&usg=AFQjCNEl3LOuH9LxcoSunA8gz8nBbnErxQ" rel="nofollow" target="_blank">https://software.sandia.gov/trac/coopr/wiki/GettingStarted</a> for instructions for getting started with Coopr. An installation script, coopr_install, is provided to simplify the installation of Coopr packages along with the third-party Python packages that they depend on. This installer can also automatically install extension packages from Coin Bazaar. See <a href="http://www.google.com/url?sa=D&q=http://groups.google.com/group/coopr-forum/topics&usg=AFQjCNFFSWBlpKNbsrQmtMHWpjfNHfETGA" rel="nofollow" target="_blank">http://groups.google.com/group/coopr-forum/topics</a> for online discussions of Coopr. </p><p>Two recent Coopr developments are noteworthy:<br /></p><p>I. Coopr development is now co-hosted by Sandia National Laboratories and COIN-OR: <a href="http://www.google.com/url?sa=D&q=https://projects.coin-or.org/Coopr&usg=AFQjCNGgFCHw6GkTl8G5lGR6cgIxf-rhFg" rel="nofollow" target="_blank"><br /></a></p><ul><li><a href="http://www.google.com/url?sa=D&q=https://projects.coin-or.org/Coopr&usg=AFQjCNGgFCHw6GkTl8G5lGR6cgIxf-rhFg" rel="nofollow" target="_blank">https://projects.coin-or.org/Coopr</a><br /></li><li><a href="http://www.google.com/url?sa=D&q=https://software.sandia.gov/coopr&usg=AFQjCNFxxg1bUmeJduZIXfzrpYm6TPS_cw" rel="nofollow" target="_blank">https://software.sandia.gov/coopr</a><br /></li></ul>Coopr includes wrappers for COIN-OR solvers, and Coopr directly leverages the COIN-OR Coin<br />Bazaar project to host plug-in packages for Coopr (e.g., see the coopr.neos package in Coin Bazaar).<br /><br />II. We have recently released Coopr 2.3. This release includes<br /><ul><li> A preliminary Gurobi solver interface<br /></li><li> Extended syntax for data command files:<br /> o 'include' command to load other data command files<br /> o 'import' command to load data from other sources<br /> o namespaces to next data declarations<br /></li><li> The coopr_install script can install packages from Coin Bazaar<br /></li><li> New conversion scripts to generate LP or NL files from Pyomo models<br /></li><li> Solvers now extract standard suffix information<br /></li><li> Various fixes to PySP solvers<br /></li><li> Full support for PyPI. You can install from PyPI using the 'Coopr' package name.<br /></li></ul><p>Enjoy!<br /></p></div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-25106608703174590692009-10-21T05:06:00.001-07:002009-10-21T05:06:43.664-07:00MINLP Test Problems<div xmlns='http://www.w3.org/1999/xhtml'>Ignacio Grossmann and Jon Lee recently announced the <a href='http://www.minlp.org/'>CMU-IBM Cyberinfrastructure Collaborative site for MINLP</a>. The goal of this web site is to create a library of optimization problems in different application areas in which one or several alternative models are presented with their derivation. In addition, each model has one or several instances that can serve to test various algorithms. This effort is different from other test problem collections by requiring a description of the problem, and encouraging the contribution of alternate modeling formulations. Thus, the actual models in this collection may be MILP or NLP formulations that simplify a nonlinear problem, including simplifications of other MINLP formulations.<br/><br/>As it happens, Cindy Phillips, Regan Murray and I are working on a paper that describes our work on sensor placement for water security, where we describe various MILP formulations for this nonlinear application. I guess we should try to add our models to this repository!<br/><br/><div class='zemanta-pixie'><img src='http://img.zemanta.com/pixy.gif?x-id=2884f2e4-f199-8044-a440-9ec0df4ddc76' alt='' class='zemanta-pixie-img'/></div></div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-15595298884235439472009-10-20T15:16:00.001-07:002009-10-20T15:17:47.326-07:00Using easy_install to download source files<div xmlns='http://www.w3.org/1999/xhtml'>Python's <a href='http://pypi.python.org/pypi/setuptools/'>setuptools </a>package includes the <font face='Courier New'>easy_install</font> script, which provides a convenient mechanism for installing a Python package from the <a href='http://pypi.python.org/pypi'>PyPi </a>repository. Normally, <font face='Courier New'>easy_install </font>installs a Python package in the Python site packages directory. However, I recently discovered that <font face='Courier New'>easy_install</font> can download the source for Python package. For example, the following command downloads the <a href='http://pypi.python.org/pypi/Coopr/'>Coopr </a>optimization package into the <font face='Courier New'>coopr </font>directory:<br/><br /><blockquote>easy_install -q --editable --build-directory . Coopr</blockquote><br/>I had to browse a variety of web pages before I figured this syntax out. Enjoy!</div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-18587456392359212392009-10-14T19:04:00.001-07:002009-10-14T19:04:15.021-07:00Book Recommendation: Software IP<div xmlns='http://www.w3.org/1999/xhtml'>I have been working with software-related intellectual property issues for several years now. I finally broke down and bought a book to help me get the big picture. The following book has been remarkably helpful:<br/><blockquote>Intellectual Property and Open Source: A practical guide to protecting code<br/>Van Lindberg<br/>O'Reilly, 2008<br/></blockquote>I have been quite surprised how well Lindberg describes the complex legal issues related to intellectual property law. Lindberg is a lawyer and software developer, and he uses computer science analogies that are quite straightforward.<br/><br/><br/><br/><div class='zemanta-pixie'><img src='http://img.zemanta.com/pixy.gif?x-id=90f3125e-a924-84ee-b750-2b13888ebe41' alt='' class='zemanta-pixie-img'/></div></div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-39621675580317865632009-10-05T20:44:00.001-07:002009-10-05T20:44:52.919-07:00Applying the CBC Presolver<div xmlns='http://www.w3.org/1999/xhtml'>Here's a fun fact that I wanted to archive... In a recent COIN-OR email exchange on coin-discuss, John Forrest suggested the following command-line for applying the CBC preprocessor to an MPS file:<br/><br/> cbc xxxxx.mps -preprocess save -heuristic off -maxnode -1 -solve<br/><br/>This command will save the cbc-presolved model in the file presolved.mps. The -heuristic off and -maxnode -1 options make cbc stop as quickly as possible.</div>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com0tag:blogger.com,1999:blog-2814339190488879140.post-28348898474933208182009-07-17T11:15:00.000-07:002009-07-17T11:18:11.871-07:00Using Open-Source Tools to Manage Software Quality<font face="sans-serif">At PyCon 2009, Aaron Maxwell gave a presentation about the use of <a href="http://buildbot.net/trac">BuildBot</a> to support an automated software QA infrastructure. Listening to his talk (<a href="http://us.pycon.org/2009/conference/schedule/event/22/">online</a>) made me think more carefully about the reasons I am not using BuildBot, which I took a look at several years ago. After working with a custom automated build tool for a few years, I have recently begun using <a href="https://hudson.dev.java.net/">Hudson </a>to automate software quality processes for a variety of open source software packages. Hudson automates the following QA activities for these packages:<br /></font><ul><li><font face="sans-serif">portability tests - building packages with different compilers, language versions and compute platforms</font></li><li><font face="sans-serif">continuous integration - rapid builds and software tests to provide developers continuous feedback</font></li><li><font face="sans-serif">integration tests - builds that test the integration of different software tools</font></li><li><font face="sans-serif">archiving QA statistics - test histories, code coverage statistics, build times, etc.</font></li><li><font face="sans-serif">managing third-party builds - building third-party libraries that my codes depend on<br /></font></li></ul><font face="sans-serif">Although I am reasonably happy with Hudson, I must admit that I did not immediately decide that it was perfect for my needs the first time I looked at it. However, as the scope of my QA needs has grown, it has become critical to have a flexible, extensible strategy for automating software QA activities. The following high-level issues have proven to be major considerations when assessing the viability of tools like BuildBot and Hudson:<ul><li><u>GUI/web interface</u><br />GUI and web interfaces are key to ensuring that developers regularly use the QA data that is being generated. Interactive interrogation of QA current data facilitates effective use of this data, and GUI interfaces are very important when developers do not all have access to the same computing platforms. These interfaces can also convey valuable QA in a concise manner, such as graphical representations of QA history (e.g test failure/successes of time).</li><br /><li><u>Extensibility</u><br />Any automation framework is going to need to be customized to adapt to local operating constraints. Thus, the extensibility of the automation framework is a key issue for end-users. A particularly effective strategy for supporting this flexibility is with plugin components, which are supported in Hudson.</li><br /><li><u>Loosely Coupled QA Tools</u><br />Hudson uses a <i>standards-based approach</i> for integrating QA information. QA activities can be initiated in a very generic manner, using shell commands whose scope is not restricted. If the QA information is provided in a standard format, then Hudson can integrate it into its QA summaries. For example, Hudson recognizes testing results that are documented with the xUnit XML format, and code coverage results that are documented with the Cobertura XML format. This strategy supports a loose coupling between the QA processes and the use of Hudson, which allows for the application of a heterogeneous set of QA tools, including tools for different test harnesses and programming languages!</li><br /><li><u>Compute Resource Management</u><br />Coordinating of a large number of QA activities requires scalable strategies for managing computing resources. Frameworks like Hudson provide basic resource management strategies, including dynamic scheduling of continuous integration builds on a build farm. More generally, scalable automated testing tools need to support strategies like fractional factorial test designs, which test many build options (configuration, platform, compiler, etc) with a small number of builds. Also, management of daemon clients also becomes an issue for large build farms (e.g. notification of exceptional events like running out of disk space).</li><br /><li><u>Ease of Use</u><br />It is worth restating that ease-of-use is a major factor in practice. Developers will not use QA frameworks unless they add value to the development process. Further, it can be difficult to convince an organization to support the maintenance of automated QA frameworks on a large build farm.</li></ul>As a final note, the <a href="https://software.sandia.gov/trac/acro/wiki/Development/Resources">Acro Developer Resources page</a> summarizes the QA tools that the Acro project is using with Hudson to support software development. It is noteworthy that this effort includes QA processes for both C/C++ software and Python software. On another project, we have also used Hudson to summarize tests of Matlab code.<br /><br />P.S. I want to thank John Siirola for brainstorming about this blog. John has done most of the work setting up the Hudson server that we are using for Acro and related open source software development.</font>Bill Harthttp://www.blogger.com/profile/10166929153462801784noreply@blogger.com2