Clojure 1.1 and Beyond

Useful presentation.
The most intriguing part, of course, is the Clojure-in-Clojure bit.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Is self-bootstrapping overrated?

There seems to be a general desire among programming language implementors to "write the language in itself".

But in pathological cases, this leads to languages that need a previous version of itself, or another implementation of the same language to bootstrap itself.

I think it is much more interesting to develop an ex-nihilo bootstrap, where the language can pull itself out of its implementation substrate as quickly and elegantly as possible.

For the record, I have implemented a Lisp->JavaScript compiler, CyberLisp, that does just that. I had to implement quasiquotation in the host language, which is less than ideal, but I still think it trumps having to require a previous or separate (i.e. interpreter) implementation of the same language.

(Update: This is not intended as a criticism of Clojure-in-Clojure, but rather as a question which I find interesting.)

Data structures

As I understand it, it's not the language that needs bootsrapping, as being a Lisp, it has grown out of its meta-circular evaluator.

It's Clojure's implementation of its persistent data structures that are the problem. Under Clojure 1.0, it simply was not possible to implement its core data structures in the language itself, which is why they were written in Java.

I think we can all see why this might be a tiny bit embarassing, so one of the big changes to Clojure 1.1 was the addition of deftype and defprotocol. Now, the machinery exists to write the persistent data structures in Clojure. There just remains the step of actually *doing* it.

Rich also wants to re-write Clojure's parallelism support in Clojure too, but he's waiting for JDK 7's fork/join framework.

Data structures with defprotocol and deftype

"Simply was not possible" might be a bit too strong. Clojure 1.0 had gen-interface and proxy, which would have been sufficient to write the persistent data structures in pure Clojure with semantics and algorithmic complexity that matched the Java versions. The actual performance, however, would not have matched and would have been unacceptable. I guess "unacceptable" is pretty close to "not possible".

But now the "master" branch of Clojure (which will probably become 1.2) has defprotocol and deftype which together form the required basis (or very nearly all of it anyway) to describe the persistent data structures in a host-independent way with performance at least as good as the current Java versions.

I think it is much more

I think it is much more interesting to develop an ex-nihilo bootstrap, where the language can pull itself out of its implementation substrate as quickly and elegantly as possible.

Sounds like IBM's Jalapeno/Jikes RVM. This is also a big focus of the smalltalk community.

No, self-bootstrapping is not overrated

Note that for any language that doesn't self-bootstrap, you'll need a language that does as a substrate. Also, you don't need an interpeter if you have a compiler written in a different language, as is the case for Clojure.

For the record, I've done quite some maintenance for Gwydion Dylan (which bootstraps from an interpreter written in C) and Open Dylan (which once was bootstrapped from Common Lisp, but these days just does what gcc is doing: assuming you already have a working compiler).

Smart-alec wisdom

I think it is much more interesting to develop an ex-nihilo bootstrap, where the language can pull itself out of its implementation substrate as quickly and elegantly as possible.

Self-hosting languages do this; the ex nihilo bootstrap environment is simply a special case where the binary compiler just happens to be a language compiler written in the language, for the language!

I'd recommend reading Friedman and Wand's Mystery of the Reflective Tower Revealed: a non-reflective description of the reflective tower for more thoughts on how to turn a philosophical problem (posed by Brian Cantwell Smith to F&W) into a mathematical one.

Free version of the above

Free version of the above paper.