Lambda the Ultimate

What does this mean? I would find the paper far more compelling were they to include some example or datum to support this assertion. In my experience, peanut butter is a pretty wretched foodstuff. That doesn't make it so!

Granted, the topic of the paper is the authors' work to extend the C# language with constructs that natively support XML and SQL manipulation. But isn't that really just creating a DSL? I can envision a set of macros and data types added to CL or Dylan that would largely support the work described in the paper. Have I "grown the language", or did I create a DSL?

Should we be thinking again about Todd Proebsting's "disruptive technlogies"?

I think that if you combine lexical macros with static types you have the potential to create some /amazing/ DSLs. For example, using phantom types you can ensure that your HTML template language can *only* produced valid HTML, for example, or your SQL embedding can guarantee that the values you get out are statically typesafe. All the pieces we need to make amazing new programming languages already exist -- we just need to put them together.

Phantom types are not necessary, and AFAIK give no advantage here. I have a paper in the works where we translate (a sizable subset of) XML Schema into Haskell types and there are no phantom types, except in one place where they make sense, namely to distinguish element types with different names but identical structure.

Phantom types are, IMO, often used in situations where non-phantom types can serve even better.

This is extremely useful for being able to invent very terse, expressive sublanguages -- imagine trying to write a macro for printf-style format strings using either Scheme or Dylan's macro languages.

Terse, expressive, sublanguages might result in scalability issues on their own, especially when you have a large development group. In large groups is it sometimes essential that all (or most) of Dev and Test be able to identify/fix problems when they occur -- having only a few people fluent in your embedded language is a drawback in such cases.

A language is an interface, just like every other API, and you can and should manage it the same way.

It is really cumbersome, in Haskell. This is one of the situations where a language with iso-inference, as I described in another forum, would be a big win.

But the translation is mostly intended for use with Generic Haskell, to write typesafe `Schema-aware' applications, i.e., applications which work for any Schema, but still take advantage of the Schema. (For example, an XML compressor can take advantage of the Schema to improve performance.) In such generic programs, the overhead of isos is almost negligible, since you only treat each case once.

Do you perhaps use recursive types?

You mean equirecursive types? No, they are an abomination. :)

Neel,

I totally agree that no API is ever going to beat a language (or sub-language) that has been designed to carry out the same task. Consider regulat expressions in Perl vs. (say) in .NET. The latter's API is very reasonable, well-designed, and clean. It has sufficient "power" to carry out all the tasks I've ever needed. And yet, it doesn't come close to the accessibility of Perl's integrated support.

That said, every such language carries a price tag with it. First you have to notice that you're dealing with an embedded language, which might not do what you thought it would do. Then you have to learn its syntax. This is an overhead which is completely justified if you're dealing with the language on a daily basis. It might not otherwise.

Yes, that's what I meant.

The extensions to the conventional type systems that this paper proposes are fairly modest, which is why I think it's cool. Small ideas that yield big wins are the best and most elegant kind.

But that also means that it's their other idea -- of making the in-language representations match the documents and queries that your program is actually using -- the most fruitful direction for further investigation. There's surprisingly little work (for example) on building integrated scanner/parser systems, which you would need to mix sublanguages with different lexical conventions. (This is especially important if you want a template language that doesn't suck, for example.)

There's surprisingly little work (for example) on building integrated scanner/parser systems, which you would need to mix sublanguages with different lexical conventions.

You might be very interested in Scannerless Generalized LR parsing. Stratego uses SGLR to support Meta Programming with Concrete Object Syntax.

Actually Erik Meijer (the author of this paper) has a tip of the day on his homepage: Why did I ever mess around with parser combinators? Scannerless GLR (or Earley) is the way of the future!

For more on this and related issues in extensible language design, you might be interested in Donovan Kolbly's PhD dissertation. I'm not entirely sure I actually like the idea of langauges being quite that syntactically (and lexically) extensible, but it's certainly neat, and it definitely got me interested in Earley parsing...