Cargo-culting to success and understanding

In doing my part to further the fortune-cookie bullshit cycle that is Twitter, I tossed out this nugget yesterday:

"Cargo-culting" isn't always bad. All things being equal, following the lead of smart, thoughtful people can be a good first approximation. — Chas Emerick (@cemerick) December 19, 2012

Little did I know that it would spur such conversation, debate, DMs, emails, and so on.  I was going to expound on the original tweet in eight parts, but thought better of it.

Cargo culting is an actual thing, of course.  Thanks to Justin Sheehy for this f'rinstince:

[caption id="attachment_926" align="aligncenter" width="500"](via http://tentpegs.patrickmead.net/?p=1780) (via http://tentpegs.patrickmead.net/?p=1780)[/caption]

He rightly pointed out that real cargo culting is all about mimicking the "trappings of others, not their essential behaviour".

Mimicking triviality certainly happens — witness the fads that drift in and out of tech culture around the most trivial things, e.g. the super-minimal desk/workspace, actually using a hammock for "hammock time", the revolutions in agile/xp/lean/scrum/etc. nomenclature, and so on — but I don't think that's what most people mean by "cargo culting", at least when it comes to software, programming, and related design.

By "cargo-culting", I was generally referring to doing something without properly knowing why it might be good or bad.  In most cases, doing this is a recipe for disaster, especially if the person doing it is of the incurious sort that is looking for permanent shortcuts.  "Keep adding indexes if your query is slow", "go look in the 'pattern' book when trying to design something", "use Mongo if you need to be web-scale".

(The dangerous bit is that we all occasionally cargo-cult things, implicitly, simply because we are social creatures, and we're inevitably influenced by certain patterns and design philosophies and technical approaches being baked into the fabric of our time and place in the world.  My sense is that many discontinuous innovations can be strongly correlated with someone becoming explicitly aware of these undercurrents, rationally re-evaluating their bases, and offering an alternative better suited to modern times and uses.)

What I was tweeting about is a different thing, though.

Especially when I'm groping into a new domain, I often ape others' work with abandon, and with only a dim view of the 'why' of the motivating design problems and decisions.  Doing so produces technical, design, and understanding debt, but I take it on willingly.  Making constant progress is often more important and more useful to me to than methodically building a formal understanding of the theory or practice related to my current task.  As I go along in the work, I continually look for ways to understand those decisions I previously adopted wholesale.  Because of the bias towards constant progress, I generally have the benefit of having a working system I built in front of me, so I have a tangible sense of the impact of those decisions.  I then can carry on having understood the original 'why' motivating them; and, if I'm feeling adventurous, I can use that understanding to usefully re-evaluate those decisions, and maybe make different ones to yield a better result.

Maybe I'm being too loose with the terminology, but the first part of this process certainly sounds like "cargo-culting" in software to me.  The difference is that:

  1. I explicitly acknowledge that I'm taking a shortcut, with the distinct intention of returning to the topic later.  (Not doing so would be an abject failure on my part.)  This is the "first approximation" part of the notion: the shortcut is a bootstrapping mechanism, not a final destination.
  2. I am extremely selective when it comes to whose work I'll choose to look at seriously.  Code pasted into a Stack Overflow answer doesn't qualify, nor does whatever is found in most popular "technical" books.  Libraries and systems built by people who have spent decades, careers working on problems similar to those I'm facing? Getting closer; but even given that sort of "population", care must be taken to match up the matrix of requirements as closely as possible.  e.g. if I'm in the neighborhood of distributed systems, taking hints from someone focused on embedded applications may be fraught with peril.

I've never been able to read a dissertation or book or three, and foom, produce out of whole cloth something well-designed, efficient, and extensible — but I've seen others do just that.  So, I know that what I've discussed here is an inefficient path, at least in certain domains and for certain people.  I think it is a natural response to attempting to build nontrivial things given fifty-ish years of software and engineering history to consider.

Finally, the terminology.  Perhaps given this notion of historical artifact, a better phrase than "cargo culting" might be "anthropological reconstructive software archaeology"?  Doesn't quite have the same ring to it though, eh?