Continuous Deployment of Clojure Web Applications: Clojure Conj slides and an epic screencast

It was a privilege and honor to give a talk at last month’s (first (clojure-conj)) on a topic that is often a pain point for many Clojure developers and teams, the continuous deployment of Clojure web applications.

Based on discussions I’ve had with Clojure developers over the past year or two, it seems that many (if not most) find the domain of application deployment to be incredibly complicated (it is), populated with too many potential “solutions” to choose from (there are), of which few seem to provide gentle on-ramps for new users (it’s true).  My talk was probably the least technical and most high-level of the conference, but based on the feedback I’ve heard so far (crazy selection bias in play!), many people seem to have come away from it with a better grasp on the problem overall, and perhaps some reasonably concrete potential paths to sanity.

Here are the slides from that talk (download/view as PDF):

(Special credit goes to Brian Carper, who was a good enough sport to allow me to call out his clever/scary app deployment architecture based on emacs, swank, screen, apache, mod_proxy, and jetty as the ultimate example of what not to do. ;-))

One of those paths is a Clojure-based toolchain, provided primarily by pallet and jclouds (which I’ve mentioned before).  If you’re already a Clojure programmer, it’s a very easy option to slip into compared to, say, Chef or Puppet. I provided an outline of using pallet and jclouds in my high-level talk, but I wanted to provide an addendum to that with a very detailed, step-by-step introduction to them, along with how one can “close the loop” on continuous deployment using the Hudson continuous integration server.

I figured I’d knock out a short-ish screencast, but quickly discovered that there was way more material than I anticipated…and ended up with a 50-minute screencast that provides a bunch of background on pallet and jclouds, walks through a basic web application build and deployment process, and sets up Hudson from scratch to drive the whole thing.  In hindsight, the whole thing may be far too long (and maybe too introductory?) for most people’s tastes; in any case, I do hope some find it helpful.

Please heed the disclaimer included in the sample project:

…what is contained herein and discussed in the screencast is by no means an ideal web application deployment process.  It is only meant to be an easy-to-understand first step into using pallet, jclouds, and Hudson. You are strongly encouraged to learn about these tools and customize (or likely replace) this basic approach for your own needs.

Without any further ado, here’s the sample project repository:

and the accompanying screencast (or watch in HD on Vimeo):

As an aside, I’d be remiss if I didn’t mention the excellent introduction to pallet that Nate Young posted just yesterday.

Finally, here are some links to additional resources you may find helpful if you do decide to start using jclouds and pallet for your application deployments:

  • The main pages for pallet and jclouds
  • Pallet’s formal documentation; insofar as typical usage of jclouds is via pallet (for us lucky Clojure developers, anyway), what’s here covers nearly all you need to know to get started.
  • Take a look at the pallet-examples repository for more sample projects.
  • Hugo Duncan (the lead of the pallet project) blogs about pallet from time to time.
  • When you do need to stray outside of pallet proper to use jclouds- (and cloud-) specific APIs, you’ll find some good Clojure wrappers waiting for you.  These include the blobstore API wrapper, as well as wrappers for Amazon’s EBS and Elastic IP service (disclosure: I wrote the latter two). It doesn’t look like autodoc for these wrappers is published, unfortunately; thankfully, the relevant namespaces (org.jclouds.blobstore,, and org.jclouds.elastic-ip) are well-documented. ns-publics and doc are your friends here.
  • One will inevitably start using jcloud’s APIs directly; its javadocs are helpful, though there’s a ton of stuff there that you fundamentally don’t need to worry about as a user. NodeMetadata is the most important common class (the class of the object in pallet’s request map in the :target-node slot).
  • The #jclouds and #pallet channels on freenode are always populated with helpful people. Don’t be afraid to ask. :-)

4 thoughts on “Continuous Deployment of Clojure Web Applications: Clojure Conj slides and an epic screencast

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s