…wherein I feel the pain of being a generalist

I’ve lately been in a position of offering occasional advice to Lee Spector, a former professor of mine, on various topics related to Clojure, which he’d recently discovered and (as far as I can tell) adopted with some enthusiasm.  I think I’d been of some help to him – that is, until the topic of build tooling came up.

He wanted to “export” a Processing sketch – written in Clojure against the Processing core library and the clj-processing wrapper – to an applet jar, which is the most common deployment path in that sphere.  Helpfully, the Processing “IDE” (I’m not sure what it’s actually called; the Processing.app that one launches on OS X that provides a Java-ish code editor and an integrated build/run environment) provides a one-button-push export feature that wraps up a sketch into an applet jar and an HTML file one can copy to a web server for easy viewing.

It’s an awesome, targeted solution and clearly hits the sweet spot for people using the Processing kit.

Stepping out of the manicured garden of Processing.app comes with a cost, though; you’ve lost that vertically-integrated user experience, and have to tangle with everything the JVM ecosystem has to throw at you.  There is no big, simple button to push to get your ready-to-deploy artifact out of your development environment.

So, Lee asked for some direction on how to regain that simple deployment process; my response pointing at the various build tooling options in the JVM neighborhood ended up provoking pain more than anything else due to some fundamental mismatches between our expectations and background.  You can read the full thread here, but I’ll attempt to distill the useful bits here.

Do I really need to bother with this ‘build’ shit?

Building software for deployment is a damn tricky problem, but it’s far more of a people problem than a technical problem: the diversity and complexity of solutions is such that the only known-good solution is immersive exposure to documentation, examples, and community.

In response, Lee eventually compared the current state of affairs with regard to build/deployment issues in Clojure and on the JVM as if one were asking a novelist to learn how to construct a word processor’s interface before being able to write:

This is, I know, a caricature, but imagine a word processing app that came with no GUI because hey, people have different GUI preferences and a lot of people are going to want things to look different. So here’s a word processing app but if you really want to use it and actually see your document you have to immerse yourself in the documentation, examples, and community of GUI design and libraries. This is not helpful to the novelist who wants a word processor! On the other hand if you provide a word processor with a functioning GUI but also make it customizable, or even easy to swap in entirely different GUI components, then that’s all for the good. I (and many others, including many who are long-term/professional programmers, but just not in this ecosystem) are like the novelists here. We want a system that allows us to write Clojure code and make it go (including producing an executable), and any default way of doing that that works will be great. Requiring immersive exposure to documentation, examples, and community to complete a basic step in “making it go” seems to me to be an unnecessary impediment to a large class of potential users.

My response was probably inevitable, as steeped in the ethos of the JVM as I am; nevertheless, Lee’s perspective ended up allowing me to elucidate more clearly than I ever have why I use tools like Maven rather than far simpler (and yes, more approachable) tools like Leiningen, Cake, et al.:

At one time, there were only a few modes of distribution (essentially: build executable for one, perhaps two platforms, send over the wire, done).  That time is long past though, and software developers cannot afford to be strict domain experts that stick to their knitting and write code: the modes of distribution of one’s software are at least as critical as the software itself. Beyond that, interests of quality and continuity have pushed the development and adoption of practices like continuous integration and deployment, which require a rigor w.r.t. configuration management and build tooling as serious as one pays to one’s “real” domain.

To match up with your analogy, programmers are not simply novelists, but must further concern themselves with printing, binding, and channel distribution.

Within that context, I much prefer tools and practices that can ramp from fairly simple cases (as described in my blog), up to the heights of build automation, automated functional testing, and continuous deployment.  One should not have to switch horses at various stages of the growth of a project just to accommodate changing tooling requirements.  Thus, I encourage the use of maven, which has (IMO) the least uneven character along that spectrum; ant, with the caveat that you’ll strain to beat it into shape for more sophisticated tasks, especially in larger projects; and gradle, which appears to be well on its way to being competitive with maven in most if not all circumstances.

In all honesty, I envy Lee and those with similar sensibilities…

The first step to recovery is realizing you have a problem

The complexity that is visited upon us when writing software is enough; in an ideal world, we shouldn’t have to develop all this extraneous expertise in how to build, package, and deploy that software as well.  There are a few things in software that I know how to do really well that make me slightly unique, and I wish I could concentrate on those rather than becoming a generalist in this, yet another vector, which is fundamentally a means to an end.  History and circumstance seem to be stacked against me at the moment, though.

Especially in comparison with monocultures like the .NET and iOS worlds, which have benevolent stewards that helpfully provide well-paved garden paths for such mundane activities, those of us aligned with more “open” platforms like the JVM, Ruby, Python, etc. are constantly pulled in multiple directions by the allure of the shiniest tech on the world and the dreary reality that our vision consistently outpaces our reach when it comes to harnessing the gnarly underbelly of that snazzy kit in any kind of sensible way.  Along the way, the most pernicious thing happens: like the apocryphal frog in a warming pot, we find ourselves lulled into thinking that the state of affairs is normal and reasonable and perfectly sane.

Of course, there’s nothing sane about it…but I’m afraid that doesn’t mean a real solution is at hand.  Perhaps knowing that I’m the frog now is progress enough for now.

Hosting Maven Repos on Github

UPDATE: If you’re using Clojure and Leiningen, read no further. Just use s3-wagon-private to deploy artifacts to S3. (The deployed artifacts can be private or public, depending on the scheme you use to identify the destination bucket, i.e. s3://... vs. s3p://....)


Hosting Maven repos has gotten easier and easier over the years.  We’ve run the free version of Nexus for a couple of years now, which owns all the other options feature-wise as far as I can tell, and is a cinch to get set up and maintain.  There’s a raft of other free Maven repository servers, using plain-Jane FTP, and various recipes on the ‘nets to serve up a Hudson instance’s local Maven repository for remote usage.  Finally, Sonatype began offering free Maven repository hosting (via Nexus) for open source projects earlier this year, which comes with automatic syncing/promotion to Maven Central if you meet the attendant requirements.

Despite all these options, I continue to run into people that are intimidated by the notion of running a Maven repo to support their own projects – something that is increasingly necessary in the Clojure community, where all of the build tools at hand (clojure-maven-plugin, Leiningen, Clojuresque [a Gradle plugin], and those brave souls that use Ant + Ivy) require Maven-style artifact repositories.  Some recent discussions on this topic reminded me of a technique I came across a few months ago for hosting a maven repository on Google Code (also available for those that use Kenai).  This approach (ab)uses a Google Code subversion repo as a maven repo (over either webdav or svn protocols). At the time, I thought that it would be nice to do the same with Github, since I’ve long since sworn off svn, but I didn’t pursue it any further then.

So, perhaps people might find hosting Maven artifacts on Github more approachable than running a proper Maven repository.  Thankfully, it’s remarkably easy to get setup; there’s no rocket science here, and the approach is fundamentally the same as using a subversion repository as a Maven repo, but a walkthrough is warranted nonetheless.  I’ll demonstrate the workflow using clutch, the Clojure CouchDB library that I’ve contributed to a fair bit.

1. Set up/identify your Maven repo project on Github

You need to figure out where you’re going to deploy (and then host) your project artifacts.  I’ve created a new repo, and checked it out at the root of my dev directory.

[catapult:~/dev] chas% git clone git@github.com:cemerick/cemerick-mvn-repo.git
Initialized empty Git repository in ~/dev/cemerick-mvn-repo/.git/
warning: You appear to have cloned an empty repository.

Because Maven namespaces artifacts based on their group and artifact IDs, you should probably have only one Github-hosted Maven repository for all of your projects and other miscellaneous artifact storage.  I can’t see any reason to have a repository-per-project.

2. Set up separate snapshots and releases directories.

Snapshots and releases should be kept separate in Maven repositories.  This isn’t a technical necessity, but will generally be expected by your repo’s consumers, especially if they’re familiar with Maven.  (Repository managers such as Nexus actually require that individual repositories’ types be declared upon creation, as either snapshot or release.)

[catapult:~/dev] chas% cd cemerick-mvn-repo/
[catapult:~/dev/cemerick-mvn-repo] chas% mkdir snapshots
[catapult:~/dev/cemerick-mvn-repo] chas% mkdir releases

3. Deploy your project’s artifacts to your Maven repo

A properly-useful pom.xml contains a <distributionManagement> configuration that specifies the repositories to which one’s project artifacts should be deployed.  If you’re only going to use Github-hosted Maven repositories, then we just need to stub this configuration out (doing this will not be necessary in the future1):


Usually, URLs provided here would describe a Maven repository server’s API endpoint (e.g. a webdav URL, etc).  That’s obviously not available if Github is going to be hosting the contents of the Maven repos, so I’m just using the root URLs where my git Maven repos will be hosted from; as a side effect, this will cause mvn deploy to fail if I  don’t provide a path to my clone of the Github Maven repo.

Now let’s run the clutch build and deploy our artifacts (which handily implies running all of the project’s tests), providing a path to our repo’s clone directory using the altDeploymentRepository system property (heavily edited console output below)2:

[catapult:~/dev/cemerick-mvn-repo/] chas% cd ../vendor/clutch
[catapult:~/dev/vendor/clutch] chas% mvn -DaltDeploymentRepository=snapshot-repo::default::file:../../cemerick-mvn-repo/snapshots clean deploy
[INFO] Building jar: ~/dev/vendor/clutch/target/clutch-0.2.3-SNAPSHOT.jar
[INFO] Using alternate deployment repository snapshot-repo::default::file:../../cemerick-mvn-repo/snapshots
[INFO] Retrieving previous build number from snapshot-repo
Uploading: file:../../cemerick-mvn-repo/snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.jar
729K uploaded  (clutch-0.2.3-SNAPSHOT.jar)

That looks happy-making.  Let’s take a look:

[catapult:~/dev/vendor/clutch] chas% find ~/dev/cemerick-mvn-repo/snapshots

That there is a Maven repository.  Just to briefly dissect the altDeploymentRepository argument:


It’s a three-part descriptor of sorts:

  1. snapshot-repo is the ID of the repository we’re defining, and can refer to one of the repositories specified in the <distributionManagement> section of the pom.xml.  This allows one to change a repository’s URL while retaining other <distributionManagement> configuration that might be set.
  2. default is the repository type; unless you’re monkeying with Maven 1-style repositories (hardly anyone is these days), this is required.
  3. file:../../cemerick-mvn-repo/snapshots is the actual repository URL, and has to be relative to the root of your project, or absolute. No ~ here, etc.

4. Push to Github

Remember that your Maven repo is just like any other git repo, so changes need to be committed and pushed up in order to be useful.

[catapult:~/dev/cemerick-mvn-repo] chas% git add *
[catapult:~/dev/cemerick-mvn-repo] chas% git commit -m "clutch 0.2.3-SNAPSHOT"
[master f177c06] clutch 0.2.3-SNAPSHOT
 12 files changed, 164 insertions(+), 2 deletions(-)
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.jar
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.jar.md5
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.jar.sha1
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.pom
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.pom.md5
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/clutch-0.2.3-SNAPSHOT.pom.sha1
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/maven-metadata.xml
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/maven-metadata.xml.md5
 create mode 100644 snapshots/com/ashafa/clutch/0.2.3-SNAPSHOT/maven-metadata.xml.sha1
[catapult:~/dev/cemerick-mvn-repo] chas% git push origin master
Counting objects: 24, done.
Delta compression using 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (19/19), 669.07 KiB, done.
Total 19 (delta 1), reused 0 (delta 0)
To git@github.com:cemerick/cemerick-mvn-repo.git
 f57ccba..f177c06  master -> master

5. Use your new Maven repository

Your repository’s root will be at


Just append snapshots or releases to that root URL, as appropriate for your project’s dependencies.

You can use your Github-hosted Maven repository in all the same ways as you would use a “normal” Maven repo – configure projects to depend on artifacts from it, proxy and aggregate it with Maven repository servers like Nexus, etc. The most common case of projects depending upon artifacts in the repo only requires a corresponding <repository> entry in their pom.xml, e.g.:



  1. Administrative simplicity: there are no servers to maintain, no additional accounts to obtain (i.e. compared to using Sonatype’s OSS Nexus hosting service), and the workflow is very familiar (at least to those that use git).
  2. Configuration simplicity: compared to (ab)using Google Code, Kenai, or any other subversion host as a Maven repository, the project configuration described above is far simpler.  Subversion options require adding a build extension (for either wagon-svn or wagon-webdav) and specifying svn credentials in one’s ~/.m2/settings.xml.
  3. Tighter alignment with the “real world”: ideally, every artifact would be in Maven Central, and every project would use Hudson and deploy SNAPSHOT artifacts to a Maven repo.  In reality, if you want to depend upon the bleeding edge of most projects – which aren’t regularly built in a continuous integration environment and which don’t regularly throw off artifacts from HEAD – having an artifact repository that is (roughly) co-located with the source repository that you control is very handy.  This is true even if you have your own hosted Maven repo, such as Nexus; especially for SNAPSHOTs of “vendor” artifacts, it’s often easier to simply deploy to a local clone of your git-hosted Maven repo and push that than it is to recall the URL for your Maven server’s third-party snapshots repository, or constantly be adding/modifying a <distributionManagement> element to the projects’ pom.xml.

Caveats / Cons

  1. This practice may be considered harmful by some.  Quoting here from comments on another description of how to host Maven repositories via svn:

    This basically introduces microrepository. Users of these projects require either duplicate entries in their pom: one for the dependency and one for the repository, or put great burden on the maintainer of the local repository to add each microrepository by hand….So please instead of this solution have your Maven build post artifacts to a real repository like Java’s, Maven’s or Sontatype’s.
    – tbee

    Please do NOT use this approach. Sonatype provide free hosting of a Maven Repo for open source projects and BONUS! you get syncing to Maven Central too for free!!!
    Stephen Connolly

    tbee’s point is that, the more “microrepositories” there are, the more work there is for those that maintain their own “proper” Maven repository servers, all of which provide a proxying feature that allows users of such servers (almost always deployed in a corporate environment as a hedge against network issues, artifact rot, and security/provenance issues, among other things) to specify only one source repository in their projects’ pom.xml configurations, even if artifacts are actually originating from dozens or hundreds of upstream repositories.  I don’t really buy that argument, insofar as the cat is definitely out of the bag vis á vis a proliferation of Maven repositories. Our Nexus install proxies around 15 Maven repositories in addition to Maven Central, and I suspect that that’s a very, very low compared to the typical Nexus site. I’ll bet there are hundreds – perhaps thousands – of moderately-active Maven repositories in the world already.

    I agree with Stephen that if you are willing to get set up with deployment rights to Sonatype’s OSS Maven repo, you should do so.  Not everyone is though – and I’d rather see people using dependency management than not, and sharing SNAPSHOT builds rather than not. In any case, if you use this method, know that you’ve strayed from the Maven pack to some extent.

  2. The notion of having to commit the results of a mvn deploy invocation is very foreign.  This is particularly odd for release artifacts, the deployment of which are ostensibly transactional (yes, the git commit / git push workflow is atomic as far as other users of the git/Maven repository are concerned, but I’m referring to the deployer’s perspective here).  The subversion-based Maven hosting arrangements don’t suffer from this workflow oddity, which is nice.  I suppose one could add a post-commit hook to immediately push results, but that’s just illustrating the law of conservation of strangeness, sloughing off unusual semantics from Maven-land to the Realm of git.  You could fall back to using Github’s svn write support, but then you’re back in subversion-land, with the configuration complexity I noted earlier.
  3. Without a proper Maven repository server receiving deployed artifacts, there will never be any indexes offered by these git-hosted Maven repos.  The same goes for subversion-hosted Maven repos as well. Such indexes are a bit of a niche feature, but are well-liked by those using Maven-capable tooling (such as the Eclipse and NetBeans IDEs).  It would be possible to generate and update those indexes in conjunction with the deployment process, but that would likely require a new Maven plugin – a configuration complication, and perhaps enough additional work to make deploying to Sonatype’s OSS repo or hosting one’s own Maven repository worthwhile.


  1. The fact that Maven currently requires a stubbed-out <distributionManagement> configuration is a known bug, slated to be fixed for Maven 2.5. Specifying the deployment repository location via the altDeploymentRepository will then be sufficient.
  2. An alternative to using the altDeploymentRepository option would be to make the git Maven repository a submodule of the project repository.  This would require that the git Maven repo be the canonical Maven repo for the project, and would imply that all the usual git submodule gymnastics be used when deploying to the git Maven repo.  I’ve not tried this workflow myself, but might be worth experimenting with.

Why using Maven for Clojure builds is a no-brainer

Put simply:

It’s the community, stupid.


If you have to pick, choose function over form (at least when it comes to build tools).

Ahem. Sorry, let’s start from the beginning.

Like any group of super-smart programmers using a relatively new language, a lot of folks in the Clojure community have looked at existing build tools (the JVM space is the relevant one here, meaning primarily Maven and Ant, although someone will bark if I don’t mention Gradle, too), and felt a rush of disdain. I’d speculate that this came mostly because of XML allergies, but perhaps also in part because when one has a hammer as glorious as Clojure, it’s hard to not want to use it to beat away at every problem in sight. Ruby has rake, and python has easy_install, so it seems natural that Clojure should have its own build system that leverages the language’s stellar capabilities – “just think of how simple builds could be given macros and such”, one might think.

I can sympathize with that perspective, and I admit that I, too, once thought that a Clojure-based build system was an obvious move. This notion runs off the rails pretty quickly for one reason:

Builds inherently involve stitching together lots of bits from disparate sources. Clojure is great for building amazingly flexible and featureful programs up from first principles, but don’t confuse that foundational capability with being able to easily deploy a Compojure web application to any of 12 app servers, or building Windows installers (or Windows executables, even), or tagging revisions in your SCM of choice, or easily using continuous integration servers like Hudson.

You can either help reimplement all of these things – or, if you’re lucky enough to have access to a build tool that has a community that has built all these things already, you can use that.

Handily enough, Clojure is a JVM language, so using all of the goodness that’s been built up over the years in Maven-land is extraordinarily easy to do. This means you have to write less code, and you get to use more mature, well-tested, well-supported code and tools, allowing you to focus on building awesome Clojure apps, not dicking around with implementing shell invocation, or Java compilation, or deployment via scp, or whatever “simple” build task you need today that’s been in Maven’s quiver for 5 years.

As if that weren’t enough, Sonatype has its Polyglot Maven project, where they are working on making it possible to drive Maven builds from your favorite language, be it Clojure, Ruby, Groovy, or Scala. For now, I stick to using XML POM files (they’re incredibly well-supported by tons of JVM-land tools – code completion on dependency version numbers FTW); while I love s-expressions, I’m too happy to trade off a pinch of syntactic elegance in exchange for tons more capability.

OK, enough blather. Demo-time!

With that said, let’s see what building a Clojure app using Maven looks like. First, the demo, where I start with the simplest Maven POM for building Clojure projects, and add in a Maven plugin to wrap my application into an OS X .app bundle (also available in HD):

For reference purposes, here’s that simplest of all Maven POM files you can use for your own projects:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <name>Easy Clojure Maven setup</name>



Further Reading

If you’re going to use Maven for your Clojure builds, here’s some links:

  1. Please make sure you check out the documentation on clojure-maven-plugin, which is where all of the Clojure-specific goals come from.
  2. You’ll do yourself a world of good by keeping the Maven books ready at hand (not the old one published years ago, BTW, the newer ones available online or through lulu). Yup, there’s a lot of material there. No, you don’t need to know it all to become super-productive with Maven.

Sane web development with Compojure, Jetty, and Maven

I find myself slipping back into web development in the new year. I’ve known this was coming for some time, so I’ve had a fair chance to carefully choose my weapons:

What has really tied this all together is Maven (and a couple of plugins for it), which has enabled me to fill in a couple of gaps in what is otherwise the most pleasant web development environment I’ve ever used (where Pylons was the prior champ, FWIW).

The biggest gap is in automatic application reloading/redeployment – in concrete terms, when I save a Clojure source file, my application should be reloaded nearly immediately, thereby avoiding any code-build-deploy cycle. To be precise, this capability is built into Jetty (as it is in many other Java-based app servers). The question is, how to most readily utilize it.

I came across this post by Jim Downing, which describes how to set up a Maven project for a Compojure application, enabling development-mode app reloading using the maven-jetty-plugin (the formatting on that post appears to have degraded since it was published; you can check out the project described in the post here). This certainly appears to fit the bill; unfortunately, the setup that Jim describes there doesn’t quite work for me – when I save a source file, the application is automatically redeployed, but no changes are picked up.

Thankfully, the fix is easy. Below is the relevant section of my pom.xml, configuring maven-jetty-plugin to add my Clojure source root as an extra classpath element. This allows Clojure, running in the jetty application server, to find and load any Clojure source files that are newer than their AOT-compiled counterparts in the usual target/classes directory (note the webAppConfig/extraClasspath elements):

            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">

With that, I’m just a mvn jetty:run away (or, really, a single click away in NetBeans) from having a development process identical to paster serve --reload, with the added benefit of Clojurey goodness.

♬♫The more you know…♪♬

(Apologies to those who aren’t familiar with American pop culture.)

If you want to compile Clojure code (and really, if you’re involved in a project of any size or importance, you should be, if only to avoid forcing Clojure to generate bytecode at runtime, which will slow down the sort of rapid development enabled by automatic app redeployment as describe above), do me a favor and use clojure-maven-plugin. (The post I reference above manually invokes the Clojure compiler using ant’s exec task, but that was what you had to do back in July 2009.) It’s a great piece of kit, and additionally serves as a perfect gateway drug to Maven – which, despite the controversy, and my own quibbles with various aspects of it, will eventually save your bacon in any larger project.

Mavenization of NetBeans Platform projects

Over the past month, I’ve been gradually porting all of our projects’ builds from Ant to Maven. Everything’s gone swimmingly, especially given the excellent clojure-maven-plugin, which allowed me to cleave off all of our comparatively complicated ant scripts for building and testing Clojure code. One part that did require some work was the porting of the builds associated with our NetBeans Platform-based applications – so, I thought I’d post a couple of hints to help others over the rough spots.

Extemporaneous and Lengthy Background

To be clear, the rough spots in question aren’t associated with the actual Mavenization of the NetBeans Platform-based projects – that’s a relatively straightforward affair, with archetypes available in the NetBeans IDE to get one started, and very well-documented goals available, all provided by the NBM Maven Plugin. Given an existing ant-based build process, I found the actual porting of the build fairly straightforward.

The dicey part had to do with having a set of Platform artifacts available to build against. Under the ant-based build regime, it was common for those building on top of the NB RCP to keep a set of RCP artifacts available in every build environment. This was always a pain (for potentially-obvious reasons that I don’t really want to get into now), and the general non-composability of the ant-based build process drove NB RCP users (and the Platform developers themselves) to extreme lengths of hacking to get stuff working properly. (BTW, just so everyone knows, I’m not picking on Fabrizio here – he’s just the one who appears to have pushed the envelope more than anyone else vis á vis improving the composability of the ant-based RCP build process.)

One great thing about the NBM Maven Plugin is that it cuts this knot quite elegantly, making it possible to treat NetBeans Modules (NBMs) as first-class citizens within the maven world. So, if you have a maven repository that contains NBMs (like this one hosted by the NetBeans folks themselves), you can readily add NBM dependencies just like you would jar dependencies from maven central:


…and the NBM plugin will take care of using those NBM dependencies as appropriate:

  • injecting the NBMs’ associated jars into the project’s compile classpath
  • adding the NBMs as runtime dependencies of whatever NBM(s) your project/application produces
  • adding the NBMs to the (optional) “update site” associated with your NB RCP application (making remote updating of that application in the field trivial)

And, to complete the cycle, the nbm-maven-plugin provides a nbm packaging type, so that you can build NBMs independently, deploy them as you’d expect, and then compose them without any ceremony into however many NB RCP applications you’d like. No suite-chaining, no special platform or cluster artifacts in every build environment, nothing at all different from what one is used to in any other jvm/maven environment.

The Rough Spot

All of the above works flawlessly (at least it has for me in my ~month of usage). The key prerequisite though, is having access to a repository that contains the Platform NBMs that you’d like to use. The repository that I linked to above does not track NetBeans releases in lockstep (e.g. at the time of this posting, the http://bits.netbeans.org/maven2 repo has NBMs from NetBeans v6.5 and v6.7, but not v6.7.1, or the recently-released v6.8). The solution is to populate your own maven repository with those NBM artifacts.

Deploying NetBeans Platform artifacts to your own repository

This might have been a tedious process, were it not for another handy goal from the NBM Maven Plugin, populate-repository, which will push all of the artifacts produced by a NetBeans Platform build (the NBMs themselves, their sources, javadoc, and appropriate non-NetBeans dependency metadata) into your own maven repository.

There’s a fair bit of configuration and setup that goes into this though. A HOWTO is provided by the nbm-maven-plugin project, but there are a number of things that it leaves unspoken. So, here’s a dump of what I did to successfully populate a Nexus maven repo with a full set of NetBeans Platform artifacts:

  1. Pull the NetBeans Platform sources from the associated hg repo (I used the release68 repo, as we’re targeting v6.8 of the NB RCP now). It appears that populating your repo with NB RCP artifacts from a binary download is possible, but then you’ll not have the associated javadoc, source artifacts, etc.
  2. Build the entire project – I’m sure it’s possible to restrict the build to certain clusters, but I don’t see any reason to optimize this process since doing so only saves a little bit of disk.
    1. You must set your JAVA_HOME environment variable to point to a Sun JDK, especially in linux environments that often come with non-Sun JDKs (I’m looking at you, Ubuntu, with your cute gcj JDK). Not doing this will result in very strange compilation errors.
    2. You must set your ANT_OPTS environment variable to specify a higher-than-default maximum heap (export ANT_OPTS=-Xmx1024m worked for me).
    3. Within the top-level of your NetBeans Platform source checkout, run ant; ant nbms build-source-zips build-javadoc – this will build everything you care about in order to populate your maven repo.
  3. You want to have the NBMs in your repository to have appropriate dependency relationships established with third-party artifacts, right? Achieving this is easy if you have Nexus:
    1. unzip sonatype-work/nexus/storage/central/.index/nexus-maven-repository-index.zip somewhere (I used /tmp/nexus-index).
    2. set the nexusIndexDirectory property in the last step to that the path where you unzipped central’s index; the nbm-maven-plugin will search that Lucene index to find dependencies referred to within the Platform’s NBMs
  4. set MAVEN_OPTS to specify a higher-than-default maximum heap (export MAVEN_OPTS=-Xmx512m worked for me). I’m not sure why this would be required, but I got OutOfMemoryErrors with max heap set to anything less than 512MB. Perhaps searching the maven central repo index is what pushed allocation so high.
  5. Make sure you don’t have a pom.xml in your current directory. Bad things will happen.
  6. Decide on a version number for the deployed artifacts, and use it as the value of the forcedVersion property. I used RELEASE68 to go along with the pattern established at http://bits.netbeans.org/maven2; 6.8 makes more sense to me, but if/when the NetBeans maven repo comes up to date with the NetBeans release schedule, sticking with their convention will allow us to use that authoritative repository with no changes to our projects.
  7. Assuming you’re deploying to a release repository, make absolutely sure that you’ve (temporarily) enabled redeployment for that repository! nbm-maven-plugin deploys some NBMs multiple times (presumably while traversing various dependency graphs), and not enabling redeployment will result in errors (400 errors from Nexus, specifically – I can’t say what might happen with different repository managers).
  8. Now for the big finish:
    mvn org.codehaus.mojo:nbm-maven-plugin:3.1:populate-repository -DforcedVersion=RELEASE68 -DnetbeansInstallDirectory=nbbuild/netbeans -DnetbeansSourcesDirectory=nbbuild/build/source-zips -DnexusIndexDirectory=/tmp/nexus-index -DnetbeansJavadocDirectory=nbbuild/build/javadoc -DnetbeansNbmDirectory=nbbuild/nbms -DdeployUrl=<nexus_repo_url> -DskipLocalInstall=true

Whew! Let that sucker run for a while, and you should be left with a maven repository fully populated with NetBeans Platform artifacts.