Enabling rich(er) interactions in the Clojure REPL

I love the Clojure REPL.  I’ve never been more productive in any other environment.  In particular, I happen to like the REPL and general development experience provided by Counterclockwise, the Eclipse Clojure plugin.

That said, the Clojure REPL is far from perfect.  A long time ago, I banged out this page describing what my ideal Clojure development environment would look like.  Much of what’s there is still an accurate reflection of my ideal end-state, and much of it is now available in Counterclockwise.  But, some things remain unfulfilled.

Like, when you are working with image data.  I do, all the time.  I’m parsing documents, producing rasterizations of them, performing all sorts of transformations and analysis on them, often in the REPL. Yet, I end up needing to have other tools around to do the simplest of things…like, oh, to look at an image.  Other fancier things should be possible, but let’s start with that and see where we end up.

This corresponds with this bullet in the aforementioned description of an ideal Clojure environment:

  • Configurable pretty-printing of output
    • This also means being able to “print” non-textual data, such as having images drawn inline into the REPL if so desired, etc.
    • See Factor’s REPL for an example

So, I implemented opt-in “rich content” Clojure REPL interactions in nREPL (a tool-agnostic network Clojure REPL server & client implementation), with a first spike of client support for such “rich content” in Counterclockwise.  The result is something that I think can be the basis for breaking a lot of new ground in how we use the [Clojure] REPL.

Video demo follows, with some discussion of the implementation and ways clients can add their own response types and such dynamically (direct link for HD video):

The code for this is currently on nREPL HEAD (the SNAPSHOT coordinates are [org.clojure/tools.nrepl "0.0.6-SNAPSHOT"]), and this branch of Counterclockwise (which, along with a couple of refinements, will hopefully be rolled into the next release). It’s all bleeding-edge stuff, so I’m sure there’ll be lots of change to come, but it’s a start.

Go poke and prod at my approach, let me know what you think — and, hey, it’d be awesome if someone knocked out rich content REPL  support for another tool, be it jark (which already uses nREPL), vimclojure, Enclojure (both of which are planned to use nREPL eventually AFAIK?), or others.

McCarthy on the W3C

I happened to listen to John McCarthy’s keynote at OOPSLA 2007.  There are some real gems in there for those interested in the history of Lisp, though I admit that I mostly blanked out through the exposition of his Elephant programming language — probably not something amenable to a purely audio delivery, at least for me.

Anyway, he delivers a great zinger at the end, a tangential response to a question about the aims of Semantic Web technologies as they relate to his work on Elephant:

When w3c decided to not use [s-expressions], but instead imitate SGML [for HTML], that showed a certain capacity to make mistakes — which, probably, they haven’t lost.

Nice.

The beauty of letterpress and craft and old arts faithfully renewed

Having worked primarily with PDF documents and all the minutiae of their fonts and such over the years, I’ve come to have a great appreciation for typography. This appreciation has led me down some interesting paths, most notably when I visited the Wilson Printing Office a few years ago, originally built in 1816 in what is now Old Deerfield Village (about a half hour’s drive north). It’s a quaint old building in a quaint old village, exactly what you’d expect in New England:

Inside, you can find a very old, manually-operated, movable-type letterpress. It’s entirely functional, and luckily, visitors are allowed to operate the behemoth.

I was reminded of this recently when I stumbled across this video, where the proprietor of Firefly Press (located in Somerville, MA) talks about his love of letterpress, the state of his craft, and how he expects it to die eventually, simply because people will forget how to do it:

The artfulness, the care, and the precision of the work exhibited there is remarkable. Seeing it makes me want to retire and build a letterpress from scratch, and start pumping out lovingly-crafted stationary and such (although remember, self-sufficiency is the road to poverty).

As Sisyphean as it might seem, I try to bring as much of that spirit as I can to what I do. Despite the sometimes soul-sucking pop culture of software development, the drumbeat of get-it-done-fast that comes on every vector, and the never-ending treadmill of “new” technologies that parade across social news sites, I try to bring a craft to the code I write, the systems I build, and the experiences I assemble for my customers.

I’m heartened that it seems that I’m not alone in this. There are many like me that seem to have re-discovered what’s important and relevant to building sustainable systems – and discovered that, yes, it’s possible to keep that separate from the trendy, the immediate requirements, the moment’s conveniences. Computation, after all, doesn’t appear to change much. Lambdas and pointers are likely to be there, waiting for future generations just as they serve us today – modulo some (hopefully slight) packaging that helps with interacting with the broader world.

(It appears that this perspective may be necessary [though surely not sufficient!] in order to build successful, and not merely elegant systems that solve today’s and tomorrow’s pressing problems. No one will cheer much anymore when an application is delivered that is largely built while sitting on one hand [a Spolkyism there, I believe, referring to IDE wizards and such].)

Something said in that video really rung out to me, reminding me of the traditions of LISP that exist, and where I happen to stand in relation to them:

The old guys got it remarkably right.

Of course, the old vanguards have faded for the most part; many have turned to Clojure and other modern lisps. Thankfully though, that sense of craft and the original intent and spirit of “the old guys'” work is there and alive.

A quick search for ‘letterpress’ uncovers a host of shops, plying their craft, making beautiful things out of cloth and cotton and wood and steel. May that continue to be the case 100 years hence, too.

Why MIT now uses python instead of scheme for its undergraduate CS program

This week, I find myself lucky enough to be at the International Lisp Conferenceat MIT in Cambridge, MA.  I won’t get into why I’m here right now, for those of you who might be surprised.  The purpose of this post is simply to paraphrase what Gerald Jay Sussman, one of the original creators of Scheme, said yesterday in an a brief impromptu talk about why the computer science department at MIT had recently switched to using python in its undergraduate program.  This change is something that was widely panned when it was announced by many people all across the programming and computing world from various disciplines, so it seems worthwhile to try to document what Prof. Sussman said.

(The impromptu talk happened much after Monday’s formal talks and presentations, and I don’t think that anyone was recording Prof. Sussman’s remarks.  If anyone does have a recording, by all means, post it, and I’ll link to it here — and probably just drop my paraphrasing.)

This is all from memory, so I’ll just apologize ahead of time for any errors or misinterpretations I propagate. If anyone has any corrections, by all means, leave a comment (try to keep your debate reflex in check, though).  In a couple of places, I’ve added notes in italics.  Just to keep things simple and concise, the following is written in first-person perspective:

When we conceived of scheme in the 1970’s, programming was a very different exercise than it is now.  Then, what generaly happened was a programmer would think for a really long time, and then write just a little bit of code, and in practical terms, programming involved assembling many very small pieces into a larger whole that had aggregate (did he say ‘emergent’?) behaviour.  It was a much simpler time.

Critically, this is the world for which scheme was originally designed.  Building larger programs out of a group of very small, understandable pieces is what things like recursion and functional programming are built for.

The world isn’t like that anymore.  At some point along the way (he may have referred to the 1990’s specifically), the systems that were being built and the libraries and components that one had available to build systems were so large, that it was impossible for any one programmer to be aware of all of the individual pieces, never mind understand them.  For example, the engineer that designs a chip, which now have hundreds of pins generally doesn’t talk to the fellow who’s building a mobile phone user interface.

The fundamental difference is that programming today is all about doing science on the parts you have to work with.  That means looking at reams and reams of man pages and determining that POSIX does this thing, but Windows does this other thing, and patching together the disparate parts to make a usable whole.

Beyond that, the world is messier in general.  There’s massive amounts of data floating around, and the kinds of problems that we’re trying to solve are much sloppier, and the solutions a lot less discrete than they used to be.

Robotics is a primary example of the combination of these two factors.  Robots are magnificently complicated and messy, with physical parts in the physical world.  It doesn’t just move forward along the ground linearly and without interruption: the wheels will slip on the ground, the thing will get knocked over, etc.

This is a very different world, and we decided that we should adjust our curriculum to account for that.  So, a committee (here, Prof. Sussman peaked his hands over his head, which I interpreted to indicated pointy-headedness) got together and decided that python was the most appropriate choice for future undergraduate education.  Why did they choose python?  Who knows, it’s probably because python has a good standard library for interacting with the robot.

That is my best paraphrasing of Prof. Sussman’s remarks.  I spoke with him briefly earlier today, primarily to ask his permission for me to post this sort of first-person paraphrasing; he replied: “Sure, as long as you paraphrase me accurately.”  Hopefully I succeeded; I’ll mention again my solicitation for corrections in the comments.

As a short addendum, while I had Prof. Sussman’s ear, I asked him whether he thought that the shift in the nature of a typical programmer’s world minimizes the relevancy of the themes and principles embodied in scheme.  His response was an emphatic ‘no’; in the general case, those core ideas and principles that scheme and SICP have helped to spread for so many years are just as important as they ever were.  However, he did say that starting off with python makes an undergraduate’s initial experiences maximally productive in the current environment.  To that, I suggested that that dynamic makes it far easier to “hook” undergrads on “computer science” and programming, and retaining people’s interest and attracting people to the field(s) is a good thing in general; Prof. Sussman agreed with that tangential point.