Technistas

I recently got a chance to work on a project using Collabnet Subversion and OpenMake Meister and put together a short demo on how to get the two tools to work together doing continuous integration. You can view it at http://www.openmakesoftware.com/flashdemo/Meister-SVN/omsvn_small/omsvn_small.html

Meister like most CI tools has several ways to kick off a CI build. You can do a scheduled build, or you can poll the SCM system. The third way of doing a CI build is to call the build from a Subversion hook. In the demo I show two of these methods: a scheduled build in Meister, and calling Meister from the Subversion post commit hook.

The setup is pretty simple. I have a repository in Subversion that has working copies for developers, and what I’ll call a ‘hands off’ working copy that only the build process uses (meaning, no developers are ever in that copy making changes. It receives changes strictly through a ’svn update’ command run by the CI process). In Meister, I have a workflow that knows how to build a small DOS application from some code in the repository.

In the demo, I first show Meister running a build on a schedule. Meister updates the ‘hands off’ working copy and then compiles and links the code. In the second case, I turn off the scheduler, and instead activate the post commit hook in the Subversion repository. The hook code calls the Meister command line, which looks like this:

java -cp c:\openmake-meister\client\bin\omcmdline.jar com.openmake.cmdline.Main
-BUILD "WINDOWS BUILD WITH SVN"

The same workflow runs in both cases. The advantage of running from the hook is that you are always guaranteed that every transaction in Subversion gets built. On the other hand, setting a scheduler to run every hour is easy and might be more appropriate for shops with less frequent code changes. In both cases Meister is driving the build with its dependency analysis engine, so the builds are fast and highly parallelized.

Overall it was pretty easy both to get the Subversion repository configured, and to get the Meister workflow up and running. The Meister command line lets you do things like set environment variables (not shown above), so you can control the workflow at a fine level of detail.

Happy Building!
- Matt

I never cease to be amazed at how software development management ignores the build problem. In a recent article in SDTimes, Alex Handy reports on a survey undertaken by RedMonk that found Java developers spend nearly 8 minutes per hour doing software builds. In round numbers, that’s an hour a day, 5 hours a week, and for a typical 48 week year, nearly 30 work days every year. Let me repeat that. Your Java developers, who are presumably paid $80-120k per year, are spending 30 days per year staring at their screens waiting for builds. If we take the middle of this range, this is a productivity loss of over $12,000 per year per developer. For a small 5-10 person team, you could hire an extra developer just with the savings from improving your build process.

The article goes on to say that Java developers “have learned to build smaller portions of their projects at a time, or to compile small pieces of code and inject them into running applications”. I’m trying to think of two worse practices in professional software development. If you build a smaller portion of your project, you risk having unsatisfied dependencies, and thus wasting time when your runtime fails to operate properly. Similarly, injecting code into a running application is dangerous even in a test environment for similar reasons – you haven’t spent any time understanding the dependencies, and therefore are at risk of dependency-related failures.

As my readers know, I work for a software company (OpenMake Software) that sells a software build and workflow automation product. We have tens of thousands of Java developers using our product through our Eclipse plugin, which gives you true incremental builds that take dependencies into account. You can build, test and do a test deploy all from within Eclipse, and not have to resort to trickery such as injecting partial code into your test server. Since the cost of Meister (which I am not at liberty to publish here) is significantly less per developer than the annual productivity loss calculated above, it makes sense for development managers to move their Java developers away from the wild west of desktop builds, and consider a formal build system. Anything less and you’re ignoring one of your primary missions as a manager: controlling the development process.

Happy Building!

- Matt

Archiva (http://archiva.apache.org) is an open source repository manager that lets users access binary and other objects for use during software builds and deploys. Its functionality is similar in many ways to traditional version control systems, and OpenMake Meister (http://www.openmakesoftware.com) integrates with it as such. There are three ways to integrate with Archiva using Meister.

1.       Via the file system. Archiva stores objects transparently in a structured way on the file system. To include the contents of an Archiva repository, create an entry in a Meister Dependency Directory with an appropriate name, that contains a path to the repository location. For example, the default archiva installation includes JUnit 3.8.1. Creating a Dependency directory with the name ‘JUNIT381’ and the value ‘C:\tools\archiva-1.2.2\data\repositories\internal\junit\junit\3.8.1’ will enable Meister to include any libraries found in this directory as part of Meister builds. I don’t really recommend this method, since someday the transparency of the files may change in Archiva. But it works in a quick and dirty way.

2.       Through a webdav client. The native interface to Archiva is WebDAV (see http://www.ietf.org/rfc/rfc2518.txt for the WebDAV RFC). A Meister activity can be easily created to execute a command line GET operation against the Archiva repository. For example, if you use the BitKinex file transfer client, you can retrieve junit-3.8.1.jar by using a Meister activity that runs the following command:

URL=http://localhost:8080/archiva/repository/internal/junit/junit/3.8.1/junit-3.8.1.jar

bitkinex.exe cp /noinfo /force $(URL) c:\temp

This retrieves the file and copies it to c:\temp, where it can be used as part of a build. This is probably the best solution. When building, you really want a local copy of the files, at the very least for audit purposes, and there are plenty of WebDAV clients out there.

3.       Mapping a web drive. Most operating systems support mapping a network drive to a web location. For example, on Windows, you can add access to the JUnit 3.8.1 repository by mapping a drive in DOS as:

net use Z: “http://localhost/archiva/repository/internal/junit/junit/3.8.1”

Once this is complete, you can access the files in the repository simply by referencing drive Z: in a Meister Dependency Directory. One restriction, on Windows XP your repository must be running on port 80, as Windows does not support web drives on any other port. This is also a pretty good method, but it eats up drive letters fairly quickly if you have many libraries that you want to uniquely map.

I’ve tried all three methods in my build lab, and while I don’t think I would use Archiva in a production environment (my bias is towards actual SCM systems or a managed file system for storing 3rd party build dependencies), it definitely ‘worked’ and was fairly easy to integrate with.

Happy Building!

- Matt

After a long break from blogging, I’m back. I’d like to invite all my viewers to a webinar that I’m hosting on Wednesday October 21, 2pm EST. The webinar is titled ‘A Tale of Four Builds’. In it I take a single piece of code and build it using four different build technologies. This survey of build methodologies moves from manual, error-prone processes to highly controlled and repeatable processes. If you’re a software engineer, development manager or build and tools manager, this webinar will help you sort out the pros and cons of the various build technologies that are common in today’s software environment.

Hope to see you there. To register, go to: https://www1.gotomeeting.com/register/554771256

This is a cross post of a blog I did for the company I work for, AccuRev, Inc. In it I describe how to use the recently released m2eclipse Maven integration with the AccuRev software configuration management (SCM) product. Maven is a very cool build and project management tool, and the combination of working with Maven via m2eclipse and Accuev via the AccuBridge for Eclipse plugin is pretty powerful. If you are interested, please click here to view the original post.