Lambda the Ultimate

Start from by making a good API, then transition into an eDSL, then progressively improve until you have what you want. This progressive model may be easier to implement (and to convince employers that it is a good idea), and it may provide more quality due to more time spent with the D before the DSL is designed.

A good DSL is a masterpiece of art and a labor of love.

It takes a lot of effort to identify desired use-cases, shake out the common principles, find the underlying formula. And even with all that effort, there is no guarantee that you found an optimal (or even near-optimal) arrangement of atoms and axioms with respect to reasoning, composition, extension, modularization, efficient interpretation or compilation.

It is easy to miss valuable symmetries or dualities. I've found that almost everything in computer science (models, advice, etc.) works just as well, often better, if you turn it on its head or run it backwards or do the opposite or switch the passive and active elements. (Consequently, I promote many ideas that seem to contradict decades of best practice, e.g. favoring global state instead of local state, rejecting events.) Even after developing a DSL, it is often profitable to try flipping things around a bit.

Haskellers develop many DSLs, leveraging their rich type systems. But, again and again, you'll find people developing variations - there must be at least five or six variations of pipes/iteratees/wires/etc. that have made it onto hackage, and many more that did not. There are at least as many reactive programming models, and again as many variations of diagrams/graphics-combinator packages.

How many DSLs do you think VPRI (the "STEPS people") have "done good"? Nile is clearly the flagship of their reports. And even Nile was a work of art and love by a man and his math-teacher mom, and even Nile leveraged a long history of existing successes and failures in the same problem domain.

Most people have little business developing a DSL. Often, people develop DSLs because they need an extension language, but they should instead be taking an extension language (e.g. Lua, Python, or JavaScript, or potentially even Haskell) and embedding an API or model (e.g. DOM). Leave "good DSLs" to the passionate experts in a field unless you are one of them - e.g. those who want to improve on SQL for the sake of improving relational database systems, or who want to improve graphics languages for the sake of performance, reuse, and extensibility.

A recommendation I've read for frameworks is that: if it won't be "great", don't bother - it will just get in the way. The same is true for DSLs. A language or framework is profitable only if you can work with it rather than working around it.

A good API is much less effort than a good DSL. There is much less risk of an API 'getting in the way' of whatever abstractions or models the user might entertain. But even good APIs aren't easy.

My 3 step process would be:

1) Have a problem which involves a class of objects which are not first class objects in any existing language (or at least good fitting existing language)

2) Create a language where they are first class objects and that fits the problem

3) Let the language evolve to handle related problems well

An internal DSL is done via some sort of meta programming mechanism, usually this means macros or in scripting languages that means base objects that implement the 'not implemented' method.
The result is usually hard to comprehend and even harder to debug; it is hard to understand what sequences are actually done as a result of the DSL code.

An external DSL depends on the quality of the implementation. Things like comprehensible error reporting, robustness and general usefulness often take a lot of time to implement; they can't be taken for granted that is.

So my message is beware of the new DSL. Any good tool must have been around for a couple of years, until it becomes good enough to be useful.

It almost seems as though that the creation of JavaScript spawned several DSLs. The underlying power of JavaScript resembling almost that of LISP gave programmers their chance to implement DSLs. Usually peculiar but aside from additives in syntax learning for users and meshing of synapses: I feel sometimes its needed. For example take databases based on Codd's laws -- without a DSL much more heavy lifting would be needed.

While implementing a DSL in a general-purpose library or libraries that are well under 20 000 lines of code is probably a no no, there come times where such added effort and learning curve(on the user's end) is required.