Lambda the Ultimate

This is my first LtU post, so I need a quick introduction. I've been an LtU lurker for years. My career has been very applications-focused, and the last compiler I wrote was in high school, but nonetheless I've always been interested in programming languages and their design. In previous lives I worked with Mark Miller (at Xanadu and Electric Communities) and Doug Crockford (also at Electric Communities).

Recently I've been seeing a large amount of interesting work on gradual typing, partial typing, aggressive alias analysis, type inference, static metaprogramming, extensible parsing, type qualifiers, modular grammars and compilers, and all kinds of other techniques. It seems to me that a lot of these fields of research are actually very synergistic. And moreover, it seems to me there's room for a language design that is based on these concepts from the ground up.

What would it look like to create an extensible grammar and declarative compiler environment that, for example, defined primitive integers as a modular type extension on top of a very small dynamic core, and built the whole language up incrementally? Or that supported adding multiple varieties of type qualifiers as an integrated part of the compiler's basic type analysis? Or that allowed type declarations to be inferred wherever possible, while letting the programmer choose whether to view the inferred types and/or augment them with explicit declarations (at module boundaries, perhaps)? In other words, what if you rolled all these different techniques together? What would you get?

I've written a fairly weighty blog post, A Growable Language Manifesto, that tries to relate these various fields in a bit more depth. The manifesto itself may or may not be persuasive -- I'm sure you'll tell me :-) But hopefully, if nothing else, the references will be interesting; I've tried to cite a fairly wide variety of sources (many of which have appeared here, but not collected).

I'm interested in any and all pointers to similar work. I'm not claiming originality here -- obviously these ideas are gaining traction in many places. I'm just suggesting they may work together remarkably well.

I also know that there are huge potential pitfalls with this kind of extensible system -- including rampant forking of the language (everyone creates their own mini-DSL and no one can read anyone else's code), inadequate explicit typing (if strong typing is optional people may not bother to use it at all and the inferencer may fall behind), and the "declarative penalty" (sometimes easy imperative optimizations or analyses are next to impossible to state declaratively).

But even still, the vision is compelling. (To me, anyway!) As with all manifestos, the goal is to be thought-provoking. I look forward to wherever the discussion goes!

I was googling for deforestation haskell and came across the principal investigator's blog. From the paper:

Abstract. Haskell is a functional language, with features such as higher order functions and lazy evaluation, which allow succinct programs. These high-level features are difficult for fast execution, but GHC is a ma- ture and widely used optimising compiler. This paper presents a whole- program approach to optimisation, which produces speed improvements of between 10% and 60% when used with GHC, on eight benchmarks.

While nominally part of the YHC project, Haskell -011 is currently a Haskell to Haskell transformer that feeds GHC.