This is my personal nearly-stream-of-consciousness scratchpad for what I’d like to see in a Clojure development environment. The comments are available below if you’d like to throw your 2¢ in – and I’m sure I’ll glom others’ ideas into this space from various sources. That said, I’m decidedly hoping to avoid beating the usual horse carcasses (fossils, really) that are often found scattered across this topic’s landscape.
The above is all a long way to say: these are my opinions, don’t get all twisted about them.
NOTE I have posted a Google Spreadsheet form to gather market data from those interested in paying for a polished Clojure development environment. If this describes your sentiments, do add your 2¢, and hopefully help convince some enterprising tool vendor to build the ideal Clojure environment. Petition / Market Research: For the development of a commercially-supported, polished Clojure development environment
The motivation for writing this up came started with a tweet of mine:
From there, Seth Schroeder asked if I was keeping “a wishlist of wants and needs for such an app”, and Michael Fogus said that such a list would be very welcome. And hell, it’s hard to argue with fogus. :-)
Current state of play
Check out the Getting Started with Clojure page, which AFAIK is a comprehensive list of available environments. My latest impressions of promising environments are near the bottom.
I think it’s impossible to really know what the breakdown of environments in use by Clojure programmers – there’s just not enough of us yet to avoid getting swamped with sample bias, regardless of your polling method.
Let’s get on with it (with not much order for now)
(In hindsight, I realize I should have made this a table, so I could check off which features are provided by each available environment; that’ll be in v2, I suppose.)
- Pervasive syntax highlighting
- More than just s-expressions and typical values: regex literals, interpolated strings, etc
- Brace, bracket, paren matching
- paredit equivalency
- Easily-togglable rainbow parens (I usually hate them, but then end up thinking they’d be handy for ~5 seconds)
- s-expression navigation
- automatic structural formatting
- don’t apply the same rules to docstrings as would be applied to other multiline string literals
- all of the above needs to deeply configurable – there are lots of different preferences out there, and I hold at least half of them from time to time!
- code completion
- Should have visibility to locally-defined vars/bindings, those that are :required or :used, as well as stuff in core
- Host (Java) interop completion should be key off of the type hints in the surrounding code, and do what it can with imports and such
- In all cases, provide associated docs as well as parameter hints (that are bolded, etc. as you go along adding parameters)
- in-place popup macroexpansion
- generation of deftype/defrecord/extend-type/gen-class/proxy method scaffolding
- Roughly the equivalent of the “implement method” quickfix that is available in Java IDEs when you add a superclass or interface to a class definition; i.e. be able to type (proxy [java.awt.event.MouseAdapter], and have shells of the various methods from the specified interface and/or abstract superclasses dropped into place. The same thing should be applicable to all forms that generate classes from clojure.
- absolutely top-notch Java development support
- this one item almost necessitates that I’ll always be in a major IDE, i.e. Eclipse, IntellJ, or NetBeans. A lot of the lower-level plumbing that’s necessary to pull off things like a complete debugger, profiler, etc. etc. is so complex, I can’t imagine anyone matching “the big three” without simply co-opting one of them.
- debugger and profiler that are able to effortlessly “focus” on clojure-level or java-level code/data structures depending on what I’m interested in at the time
- code coverage tools that hook into clojure.test and, again, can track clojure- and/or java-level coverage equally well
- maven (and eventually, clojure polyglot maven) support
- ensure that project settings, REPL configuration, etc. etc. are all driven from the POM
- insofar as lein becomes a mainstay of clojure development, support will be needed for it as well
- Refactoring (rarely needed, but when needed, it would be incrediblyhelpful)
- This would almost certainly be limited in static usage – local renames, and such.
- Given an open REPL that’s had my entire application loaded (and therefore should be able to use function/var metadata to its fullest), I should be able to do damn near anything you can do in a top-notch Java IDE (within the bounds of what’s relevant, anyway).
- the equivalent of Java IDEs’ “organize imports” in clojure ns use/import/require clauses
- Static analysis
- Symbol navigation
- Go to declaration
- A “Go to Var” option, corollary to “Go to File” or “Open Resource”; I should be able to type “redu” to find and go to reductions, or type “w-o” to find and to go with-open, in any namespace in any open project
- Find usages
- Making this work across projects would be absolutely stellar. i.e. If I’ve got two projects open, I should be able to find usages in project B of a var defined in a namespace in project A.
- Symbol navigation
- current-file code outline
- static namespace browser, with reasonable search capabilities
- Sane UI/UX
- progressive disclosure, discoverability of functionality, familiar help systems, etc
- Supporting, nay, embracing mousing. Yes, I use the mouse (actually, a trackball) in addition to the keyboard, get over it. Much of my time spent “programming” is actually spent doing things other than typing (thank goodness), and many of these tasks are not best done with the keyboard.
- All of this should go without saying IMO, but emacs seems to put a stake in the ground that much of the above is unnecessary or undesirable. The issue of progressive disclosure and discovery of functionality is a particular sore spot for me. Many of the most important features of a development environment (or perhaps any class of nontrivial software) are the most esoteric that might go untouched for weeks or months (debuggers, profilers, code coverage tools, configuration settings, etc), and one must be able to effectively stumble through them for the first time, and after having not touched them for a long time. I don’t want to have to learn, remember, or google for the M-x run-clojure-debugger command or somesuch along with 200, 500, 1000? others.
- REPL support (overall, I think it’d be hard to improve on enclojure 1 – and, AFAIK, swank-clojure, in this department)
- all REPLs should be separate processes (I don’t want a REPL session to hork my main environment)
- therefore, all REPLs are fundamentally remote REPLs
- ensure that the REPL server library is lean, available, stable, and licensed liberally, so I can roll it into my applications and therefore….
- connect to running REPL servers on any IP/port from the dev environment, making all REPL-enabled functionality available regardless of where the host REPL process is running
- REPL history persistent across IDE restarts
- Support for multiple REPLs (i.e. load code into the last focused REPL)
- Full editor capability in the REPL – highlighting, symbol completion, formatting, etc.
- Browsable/searchable REPL history
- Prior expressions accessible via standard command history (Ctrl-up/down, etc), as well as by clicking on expressions still shown in the REPL window
- Runtime namespace browser, with the same usage characteristics as the static one, but tied to the focused REPL
- Configurable pretty-printing of output
- This also means being able to “print” non-textual data, such as having images drawn inline into the REPL if so desired, etc.
- See Factor’s REPL for an example
- Ability to check status of and kill long-running REPL invocations
- Optional (likely default) separation of input, *out*, and *err* content
- Automatic generation and configuration of the classpath for local REPLs; this includes (almost certainly in this order!):
- all current-project source paths (this ensures that changes source files are loaded before same-named files or AOT’ed classfiles from the project’s artifact(s))
- all other projects’ source paths (this allows me to load changes I’ve made to code across my “main” project’s dependencies)
- all project dependencies (no-brainer)
- the project’s artifact(s) (or perhaps target/classes (in the case of maven projects) is enough – obviously necessary in order to have access to classfiles from other languages/tools that generate them)
- All clojure projects (whether maven- or lein-based) have clojure as a dependency, so requiring any special “clojure setup” or creation of a “clojure platform” is either pointless or dangerous (the latter if you are unwittingly using a statically-defined “clojure platform” while your project’s POM is explicitly declaring a different version of clojure as a dependency) 2
- The one area where having a default set of Clojure libraries available (as a “platform”, if it must be called that) is to start a REPL for a non-Clojure project, which is very convenient.
My evaluation process for a new or updated environment is to just try to do what I’d do in a normal day: open a project or two, open some files, do some editing, open a REPL, etc. When I hit some difficulty or serious unpleasantness, I walk away (sorry, there’s only so many hours in the day).
(FYI, these notes are current as of 10/26/2011; I don’t want people to get mistaken impressions after I’ve neglected this section/page for six months.)
I’ve been using Counterclockwise and Eclipse full time since fall of 2010. After struggling with significant difficulties with NetBeans itself for some months (segfaults suck; on the other hand, Enclojure itself was always fantastic), I had to find another home.
Counterclockwise’s REPL was previously not in good shape though. After deciding where I’d place my bets, I ended up designing and implementing nREPL (a tool-agnostic Clojure REPL server and client), and rebuilding the REPL support in Counterclockwise from scratch. The result made it possible for me to use it for real work, and I’ve enjoyed it ever since.
The recent v0.4.0 release has really brought Counterclockwise into its own, though. Its namespace browser is great, code completion is nearly as good as Enclojure’s (and approaching what I’ve seen in SLIME), and the REPL and editor continue to improve. FWIW, I’m currently working on some enhancements to nREPL and Counterclockwise to support richer REPL interactions than are currently possible in any mainline Clojure environment, and there are lots of things in the pipeline that make me confident that I’ll not have to switch my tooling for years to come.
(Especially with the DOM-like UI and css styling possible in the forthcoming e4, I can imagine a nouveau emacs-like environment (i.e. deeply-customizable UIs and behaviours) coming out of a combination of e4 and clojure.)
Enclojure, the plugin for NetBeans,
is my current tool of choice (NOTE: I started using Counterclockwise and Eclipse full time in fall of 2010), and has been since I started seriously programming in Clojure ~summer of 2008. I think it was the first Clojure IDE plugin, and IMO, it’s still the best. Why? It checks off more of the requirements listed above than any other option. :-) It definitely has a variety of issues, but no showstoppers; it gets the job done well, and the base NetBeans environment is sufficient.
La Clojure (IntelliJ)
La Clojure is notable in that its development is sponsored commercially by JetBrains.
The baseline editor functionality is probably the best of all of the IDE plugins, with fast and excellent clojure symbol completion – though only of core, it seems.
Things fell down pretty fast when I tried to get a REPL going though – La Clojure uses (what I consider to be) the fundamentally broken notion of a clojure platform (even though my project had a clojure and contrib dependency in its POM). Despite futzing with my project’s clojure framework setup quite a bit, I still wasn’t able to create a REPL – attempting to do so popped up a “No Clojure base path defined” error (or something to that effect).
Having professionally used IntelliJ in the past for full-time Java development, I know JetBrains can produce killer tools; hopefully we’ll see great things from them as La Clojure matures. Good on them for diving in, though.
The emacs world
It’d be hard to talk about lisp environments without talking about emacs. I received at least three or four replies to my original tweet suggesting it…including Dan Larkin’s, to whom I responded:
Though I’m a thoroughgoing pragmatist with regard to my tooling, I confess a degree of bitterness towards emacs for a variety of reasons, both technical and social. As a historical note, I’ve made two serious attempts at using emacs full time – one many years ago, the latest a two-month run in 2008. In both cases, I was foiled by UI issues, a lack of adequate JVM tooling (and no, the JDEE doesn’t cut it), and a complete mismatch between myself and what I’ll smugly call the “emacs culture”. On the last point, I’ll simply say that I’d always rather get real work done than futz with my text editor, and I can’t imagine ever being willing to spend as much time customizing and tweaking my development environment as really productive emacs users.
There’s no doubt that emacs provides a base for very capable lisp tooling – inferior-lisp, swank and slime, etc. etc. make it an easy base to build on. Where things seem to go pear-shaped are in the areas of UI/UX and any functionality that is JVM-centric.
Like I said, I’m a pragmatist, and will go wherever my capability and leverage will be maximized – so if emacs somehow were to change enough to meet the above criteria (or, meet more of the criteria than whatever my favored environment was at the time), I’ll use it with joy.
Actually, I think an ideal path would be if emacs (or a functionally-equivalent editor/scripting environment) could be hoisted into, say, Eclipse – that might come very, very close to satisfying the vast majority of my wish list items. Again, this is why counterclockwise (or some other eclipse plugin) may have a big leg up given what’s coming in e4.
OK, so Lispworks is a Common Lisp environment. It has no Clojure support, but I dare its authors to try. Why? My list above can probably be summarized by “Eclipse + LispWorks for Clojure”. It’s easily the best CL environment I’ve ever used, and I’ll bet they could put together a killer Clojure plugin for one of the major IDEs. I doubt that will ever happen, but it’s a nice thought experiment and shorthand for what I’m aiming for.
1 Fundamentally, I think people building clojure plugins and such should simply co-opt enclojure’s REPL components (Eric Thorsen suggested bringing them into clojure-contrib at some point (can’t find the link now, sorry), and I think that, or something similar should definitely happen).
2 This concept is either a valid and useful accommodation for new/less advanced clojure programmers so as to hide the necessary yet dull complexity of what versions of what libraries your project depends upon, or a misguided tooling holdover from other languages that are older and move less swiftly than clojure does. It’s certain that that rate of change is partially attributable to Clojure’s relative youth, but I suspect that Clojure’s rate of change will always be higher than most languages, given its relative simplicity. We shall see – maybe in another year, we’ll all switch to 1.3, and virtually everything will be in libraries for the next 5 years.