Lambda the Ultimate

Thank you for pointing that out. Any other related languages anyone can think of?

Lisp? Ruby? Python?

I was referring to languages which can encompass a wide range of other languages with radically different syntaxes. Lisp, Ruby, and Python do not allow the same degree of dynamic modification and creation of syntactic structures to the extent that Katahdin and XMF do. Sorry I wasn't more clear.

I wonder how far you could push Common Lisp to do it. Reader macros?

You can definitely do similar things with reader macros in Common Lisp.

An example from INFIX:

#I(if x<y<=z then f(x)=x^^2+y^^2 else f(x)=x^^2-y^^2)

This still appears to be just a neat trick, than something that you could use to embed a real C-like DSEL in Lisp. If you were able to write the following code (note the function signature inclusion), I would consider it to be comparable to languages like Katahdin and XMF.

int f(int x, int, int z)
{
  if (x < y && y <= z) {
    return x*x + y*y;
  }
  else {
    return x * x - y * y;
  }
}

I'm not sure what you mean by "neat trick" or "fragile". The fact is that control is handed from the reader to a custom parser. There is basically no limit to what you can do with it at that point.

The actual implementation involves the use of a custom parser of the input stream which then renders this as sexps with various defined operations. The only difficulty in modifying this example to something that would work for C is in writing the parser for C.

In my opinion to really facilitate this sort of thing you'd want to have a nice way of describing parsers declaratively. DCGs or combinator parsers seem the nicest approaches in terms of ease of use that I've encountered, but anything will do in a pinch :).

Thank you for correcting me. Calling it a "neat trick" or "fragile" does not seem fair nor accurate

Perhaps we can express all of the things that we can in XMF just as easily in Lisp. However, I have skepticism that the reader macros approach can facilitate the kind of language weaving that we see at http://www.ceteva.com/XMF/Snippets/Mixed/Mixed.html. What do you think?

But from the lisp perspective this seems like a really odd thing to *want* to do, and so you very rarely see things like this done in practice.

If you swallow the lisp pill, the idea of then adding lots of syntax doesn't make any sense, since you are better off adding the domain language as new functions and macros which will then blend seamlessly with the rest of the application. I think this approach plays out quite well.

Sometimes, however, you really *do* want syntax. Try reading a bunch of implications and conjunctions in lisp....

(-> (-> A (and B C)) (A -> B))

Seems like it depends a lot on the audience of the DSL. Here's a question: even if you made new syntax, how much of the fact that it is a programming language would still show through? What does one gain by changing the syntax? And, if you have to pick a semantics to be easy to learn by non-programmers who don't have time to become programmers, what should it be? Declarative?

(Some of the XMF examples i.e. with every language under the sun all in the same file is like a ginsu knife cutting through solid steel - uh, great, but do I really need that? :)

I completely agree with you regarding the audience. Our experience with a number of useful languages is that they get no adoption in certain industries, even though the languages are ideal, because of the perceived difficulty of understanding. No amount of patient explanation, cajoling or downright shouting works - people won't represent information using things they feel uncomfortable with.

This is particularly true of Lisp (with reference to the previous post). Even though Lisp is one of the most perfect gems for representation and computation, the mainstream industry cannot accept it. Therefore, pointing out that you can neatly represent data and processing in Lisp is true but missing the point.

Regarding whether a programming language shows through: it depends whether it needs to or is useful. We are talking about all aspects of representation here - not just Lisp-in-C-in-Java (a somewhat unfortunate example which was chosen to prove a point more than anything else).

When developing a large enterprise system you use lots of representations: XML, HTML, scripting languages, programming languages, build scripts etc. Not all of these are 'programming languages' and all are domain specific in some sense.

DSLs are all about raising the abstraction level whether raising it above the level of program statements or above the level of pointer manipulation when creating data structures. If it needs to poke through then poke it through - otherwise hide it!

-- Tony

Getting the industry to use DSLs is indeed a problem. I personally tried to push for the use of some tailored DSLs for some specific problems, and had only little success doing so. In most cases my colleagues preferred using the traditional general purpose language, preferring redundant code over code in an unfamiliar language.
However, as you said it yourself, the newer industries, such as the web-application industry is a massive consumer of DSLs. In a typical web-application the development uses (at least) the following:

I was just writing up some stuff in Java and hating every minute of the obfuscation that is inevitable in that language. There's an algorithm in what I wrote, but sometimes I feel like "good luck finding it!" after all is said and done.

So I'm certainly motivated by experience (including having used other languages like SML, Lisp/Scheme, touches of Prolog, Scala, Haskell) to want to be able to express things better.

But I am not a language creator. I'm not well versed in the PLT needed to do a good job making a language. People are bad enough at writing code in languages designed by really smart experienced people, like those who made Java. How bad will it all be when I get a new job only to find that I have to program in some horrible language invented by the equally uneducated (except for how they thought it was a good idea to do it, while i am smart enough to be wary of such things ;-) person who worked there before me?

I mean, if I encounter crappy code in Java, at least I can google up a bunch of pages by smart people which talk about why XYZ is considered harmful and I can try to use that as leverage to show the code really is crappy. Or, I might learn that I was wrong, and it isn't actually crappy at all. But none of those can happen so easily when it is some scary home-grown DSL.

[edit: and, as the LtU meme goes, it isn't just the syntax, it is the semantics. how are people going to know enough to know that, and know enough to not make a hash of it?]

(I wonder if / hope that concept programming can will does help, too.)

Apart from Stratego/XT I found this interesting when I was trying to prototype a DSL:
ASF+SDF (very similar to Stratego).

I finally used BNF Converter because it gave me an OCaml AST and had a low learning curve.
But I guess these are just frameworks, unlike what you have in mind.
[edit:fixed links, sorry abt that]

There is no link for BNF Converter.