May 25th 2014 Clojure: A Retrospective

Going into college, I (sort of) knew exactly one language: VBScript. Four years later, with a shiny BA in Comp Sci in hand, I had added five to my repertoire: Java, C, Objective-C, Python, and Ruby. A year out of college, I’d added another, Scala; I knew Erlang and Haskell well enough to muddle my way through a codebase. But then a funny thing happened: I stopped learning new languages. Since 2009, I’ve learned none.

Occasionally I’ve thought about why that happened. I suppose I was busy; I was living in New York, working at The New York Review of Books; all that was exciting enough that I wasn’t inclined to spend my time nose-deep in a programming book. On top of that, for several years, I strongly considered leaving the tech scene entirely, and aside from work, programming was the last thing on my mind. Being distanced from academia didn’t help, either. I lost my motivation to explore new programming languages. I lost my edge.

Learning new languages is both easy and hard. Easy, because once you know the general concepts and theories behind software construction, it’s easy to pick up new languages, just like it’s easy for a musician to learn new instruments once she has learned music theory; hard, because once you know a few languages, it’s faster to reach for the language you already know, rather than invest the time necessary to learn the minutiæ of an entirely new one. Think about the first language you learned. You not only had to learn how to write software, but you also had to learn how to write software in that language; you had to learn how the compiler or interpreter was invoked, how the build system worked, how to actually turn your source code into a working program. The former—simply writing software—is easy once you have learned the foundation of programming; the latter, how to turn your code into something—the language’s ecosystem—takes nearly as long each time you learn a new language.

And if you’re a perfectionist like me, you become hung up not only on the language’s syntax and semantics, not only on its build system and ecosystem, but on how to write good software in that language: how to properly structure your programs to maximize the benefits of the language, how to write idiomatic code in that language,1 and so on. These are things you simply didn’t worry about when you learned your first (and maybe even your second or third) language, because back then, you didn’t know any better.

So after learning a few languages, you just…stop. Or, at least, I did. When faced with the prospective of creating a new personal project, why take all the time to learn OCaml or Go or Eiffel or Prolog when I can write it right now in Ruby? Why take all the time to work through new language tutorials that are going to start off with simple tasks that were ingrained in me 10 years ago, when I can already write useful code in Python (and get paid a handsome sum to do it)?2

Enter Clojure.

My first exposure to Clojure was in 2007, thanks to Proggit. Around that time, I was introduced to Erlang and Scala, as well. Clojure seemed interesting, but I was a bit intimidated by the fact that, aside from having a superficial understand of its history, I knew nothing about Lisp.3 Needless to say, I gravitated towards Scala and Erlang instead.

I eventually grew tired of Scala; it’s an interesting language, to be sure, but it’s grown into a language that seems to have everything but the kitchen sink, and despite Scala aficionados’ claims to the contrary, I find the object-oriented and functional programming paradigms to be distinctly at odds with each other.4 On the other hand, I love Erlang’s programming paradigm, but nowadays most of my programming is web-oriented, and Erlang’s string handling frustrates me often enough that I have to be sufficiently motivated to use it; still, I find the language itself to be fascinating.

Clojure was always in the back of my mind, the language I dabbled in from time to time, but never threw myself into. I had four Clojure books (three on the language itself, one on web programming) sitting on my shelf, begging to be read. I just needed a project.

Tracking Karma Growth For Fun and Profit

I’m an avid user of Stack Overflow, but one thing that’s always frustrated me is that it doesn’t give you a lot of statistics about your reputation score; the best you get is a graph of the amount of reputation gained per day. I’ve often thought that it would be cool to see a graph of your total reputation over time, and maybe even the rate of change or average gain per day.5

For the past few months, I’ve been working at SocialCode, writing crawlers for Facebook’s and Twitter’s ad APIs. One day, it dawned on me: Wouldn’t it be cool if I started tracking my Stack Overflow rep, using the samples to calculate interesting statistics, like growth rate or average rep gained per day?

At first, I planned to use Erlang, since it’s one of a handful of languages we use at SocialCode. Once again, though, its obtuse string handling stymied me. I’ve always used small side projects to learn new things about programming, but I wasn’t feeling up to spending hours reading about strings in Erlang, for a project that could be written in an evening.

Why not use this opportunity to finally try Clojure on a real project?

So I did. And in two evenings, I produced Chameleon, a simple crawler that runs as a cron job, sampling a user’s Stack Overflow reputation at a specific point in time, and storing it all in a MongoDB database.

And the next night, using Chameleon as a basis, I wrote Karmanaut, which does essentially the same thing, but for Reddit link and comment karma.

Putting the Fun Back Into Programming

Originally, I’d planned this article to be a technical analysis on building a useful (albeit simple) computer program with Clojure and MongoDB; instead, in true Paul Graham fashion, I meandered into a discussion on how to put the spark back into one’s relationship with programming. While I hope to do a more technical write-up of my experiences with Clojure (and MongoDB), for now, I will leave with these parting words:

Clojure is fun. Writing code to connect with other web APIs was not only easy, but pleasurable. Incrementally testing my program was made easier with a combination of Clojure’s approach to program design (which is largely based around small functions that perform one task) and its excellent REPL. Leiningen, Clojure’s build system, is one of the best build systems I have used in any language, period.

It’s easy to get into a rut when you’re a professional programmer. You work with the same languages, frameworks, and tools for years on end, you start to get bored, and think that everything is terrible. Learning something new can help you remember why you started programming in the first place.

  1. Doubly important if you plan to share any of your code on the Internet, unless you want to be called an idiot and told to go die now please. 
  2. Or maybe get off the computer entirely, for once, and go outside and enjoy other things in life instead? 
  3. Clojure was also really young at the time; Erlang, on the other hand, was over twenty years old, and Scala, at the tender age of four, was relatively more mature. It was also more like Haskell, with which I had a love affair at the time. 
  4. That the language seems to radically change with every point release doesn’t help my love for Scala grow fonder, either. 
  5. I’m not particularly hung up on users’ reputation scores; I am particularly hung up on useless statistics.