Lambda the Ultimate

But surely the point of interest here (and the reason it's relevant to this website) is not that it's a rule-based system but that there's an accompanying DSL for expressing the rules? Though maybe you were just commenting on the statement that the language is "based on a (relatively) uncommon model of computation"?

What you describe is exactly what a DSL is, so I am not sure I understand the title of your post. DSLs are all about providing a higher, domain specific, level of abstraction.

I think you and parent might be disagreeing on what domain means, which is understandable since there is no strong distinction of granularity. A DSL for angry birds, a DSL for physics castle invasion games, a DSL for games, a DSL for phone applications. ...

Maybe. Do people in gaming think of in-game languages (GSLs) as DSLs?

The lines are blurry. Personally, I don't find the terms GPL or DSL, or scripti language, to be very useful.

For the record, GSL was a joke.

I didn't get it :) Many games have scripting facilities, even in game ones (in the sense that they are open to users). I'm still waiting for a game designed around programming though.

Most DSLs I have seen wear the name pejoratively. They are (or are thought of) as tiny, incomplete, concrete and not reusable. Most are written to be used once and then thrown away. A few (like HTML and SQL) turn out to be very useful but instead of having a whole library of these, it seems the temptation to add stuff to them is irresistible. Cold Fusion anyone?

PuzzleScript defines an entire application. It has data and code. It has an (unfamiliar) execution model. It is compact and expressive. It has already been used to create hundreds of programs, and it looks as if it could grow into something very powerful. I found this unusual.

I don't feel that I'm doing this argument justice and this is not a good place for an extended debate. I am very keen to find other DSLs, or people who are working on them. What I really want is a Highly Productive Language, and restricting the domain may be the price to get it.

I still call them "Little Languages" (after Jon Bentley). I see this as a badge of honor, not in the least pejorative. LtU has of course always had a soft spot for DSLs. Just go back and see the archives of that category/department.

I almost replied to Ehud and Sean, but I'm unsure what distinction might be drawn between GSL, DSL, and scripting language. The lines do seem blurry. I tend to think of DSLs as tending to declarative form, sometimes lacking in Turing completeness, but nothing requires this be so. Basically general comes in a lot of shades of gray, so some languages fit in both general and scripting categories.

Why do we have so much difficulty doing this in other domains, and why are there so few languages like this one?

This is really close to a question I want to discuss. :-) But so far there's no good handle to start. Can you ask it again, in different words? Two issues come to mind: the problem, and the difficulty. Why don't folks try to do Y? And why is it hard to do X? Different conversations occur depending on choice of Y and X.

Almost any time you try to do something (a plugin for variable Y above), it reminds people of something else that already exists, and your effort is deemed pointless by virtue of meager novelty. Writing a script? Gosh I use Bash for that. (Maybe that sort of shell doesn't fit in this context, perhaps because I'm not really free to launch five thousand concurrent native processes where this happens.) New things are too little, too late, rip off so-and-so's work, not as easy to use as Z, dominated by company A, have better syntax in language B, provide no business model, won't virally infect social networks, and become legacy code after the singularity next Tuesday. Whatever your parade, it will get rained on. Constructive negative feedback sounds like: I don't like P, and can you add some Q? But usually folks just say: you suck. Not much fun.

It would be relatively easy to write scripts for responses to events in many domains. Do you have one in mind? But implementing the execution environment goes against "how things are done" more often than not. Naked imperative code in top-down style is popular.

Given an interesting domain such as database applications, are there any DSLs? If not, is it because people just prefer to keep writing Java, C# or Ruby, or is it just too hard? Rails, to take one example, contains a complete toolkit to do just that. Yes, there are bits of DSL-ish stuff like YAML and ERB, but most of what you will write is unashamedly Ruby, and pretty scary stuff it is too. Leaky abstractions abound!

Why should one not expect to be able to write a small database app with a couple of pages of high level code, and never see any of the underlying machinery? No answers yet, still trying to work out what are the right questions.

A few years ago I was fooling around with the Common Lisp systems ASDF, and QUICKLISP it does remarkable things with small amounts of code. Of course you probably have other issues with Common Lisp.

Thanks, that was interesting to think about. But the sheer number of things that gave me to consider is a hint "database" is not a very strong constraint. (I could make an argument anything that manages storage is a database application, and that therefore almost everything is, when reading and writing is involved, even if it's just in memory and not secondary storage.) You might want to pick a sub-domain of storage that narrows something a DSL can assume.

Language generally abstracts something away: more concrete details that vary. If the details matter, it's hard to abstract them away if a user wants to constrain semantics of those details and their interactions. I'm trying to resist itemizing examples, as it will consume remaining rough word count I had in mind. Format and interface issues abound, as do schema and performance factors. Timing of events can be very complex when it matters, especially when lifetimes of sessions can overlap in awful ways. For example, an earlier database session A can exist when new network flow F starts, which has outstanding async requests when the database restarts (assuming it's in another process or node), and new activity in later db session B engages. Then flow F resolves canceled requests from session A along with replies from session B, while someone else is trying to abort flow F because the earlier failed requests are unacceptable. This can be considered normal activity that must progress in an orderly and robust fashion.

When things happen can be as important as what happens, and this can interact with definitions of correctness too when you want consistent views observed by independent actors. This is just an example of something that might or might not matter, until relevant sorts of assumptions are stipulated as background for a DSL's semantic model.

Apple's OpenDoc in the mid-90's was not integrated in MacOS, so it actually functioned as a standalone user-space app. So information about what kind of content could be processed — when you dragged one thing onto another in the UI — was distributed among potentially many files, all of which had to be opened at the moment a user was waiting for UI feedback while dragging. All this info could have been gathered in one database file, but it wasn't. And nothing stopped users (early adopters who liked to install many things) from doing things with multi-second file system latency while a user was pressing and holding a mouse while waiting. This is an example of when performance matters, but it might be hard to abstract away in a DSL.

I have several longish anecdotes about latency behavior in storage problems that sound quite different, but it all amounts to more of the same. In one case, for a deployment expected to occur with physical RAM of N gigabytes, folks projected a database size of N*M gigabytes consisting of arbitrary RDF triples. And they said, "We want instant response time for arbitrary queries without indexes; just look at all the data—all N*M gigabytes. You can do that in a tenth of a second, right?" Even your technical users can make rather irrational demands.

If you want to start a new discussion topic about DSL design for a domain involving storage, I'll participate some. Topics about programming languages are generally okay, especially if it's about design and has a general quality. Threads of a "let's design a language" nature are disagreeable to folks here when so very concrete they lack any theoretical basis offering abstract value. Focus on syntax, comparison to existing languages, and analysis of what programmers are like, all tend to press too-concrete buttons. For example, I don't care about syntax at all.

A lot of LtU folks want to talk about math. :-) In fact, if you said you wanted to raise barriers to programming by requiring a better grasp of math when coding, this will appeal to many. But your idea sounds like the opposite: lowering barriers so folks who are not primarily programmers can specify part of what happens, somewhere along the fringe we associate with ideas of end-user-programming. It veers close to a marketing idea, which doesn't fit well in LtU's venue, but you can probably avoid that by assuming archetypical personas without dwelling on them too much. Moe is a business guy in a suit who pokes Curly in the eyes when angry, who wants to specify reports. (Is Nyuk taken as a programming language name yet? Edit: Just my idea of humor.)

I think you want to focus on messages in DSL design for things Moe will see, which would include reports, and meta messages about config, constraints, and internal events. I had to lookup LOB to learn this acronym means line-of-business, and thus that generally you're thinking about a problem space with distributed consistency issues, so there is clearly a message-oriented protocol aspect to this. You could run all the descriptions of message families through a compiler that targets a particular concrete backend infrastructure, which is what you want hidden by the DSL.

As always, naming things and invalidation are practical concerns with painful consequences when done poorly. If Moe writes descriptions, some parts will be names of entities referenced more than once, so relationships can be specified and elaborated. The naming scheme must make sense to Moe, and also be amenable to effective and clearly defined machine processing. Moe also needs an idea of what can be invalid, or else he'll have trouble reviewing his own output to prune away noise and mistakes.

Improving signal and reducing noise might be a main concern in good DSL design here. Computers make excellent amplifiers and will happily propagate both signal and noise. If it's easy to put nonsense in messages, and hard to scrub this, you'll end up spamming so much noise internally that it undermines the value of any signal. Making a language friendly to end-users can risk letting them say crazy things in the name of permissive design one hopes is agreeable. You want a tool like a personal assistant that catches mistakes politely instead of committing all your bad ideas without thinking.

Your comment's not that long. Most really short things aren't that interesting. There aren't that many places to talk about it. A higher volume venue is going to tell you Javascript is the answer to everything. :-)

Thanks for your feedback so far. I'm not really ready to tackle this seriously, I have a lot of research to do first and I'm really looking for leads and ideas. I'm also trying to find out if anyone cares about this stuff, or if everyone in academia is focussed on the latest type systems (or whatever).

I come from a commercial background, and I care about customers and users. If I write software and nobody uses it, I've failed. I'm looking for people who work on languages because there is a customer problem to solve, not because there is another theorem to prove and another paper to write.

I read things like http://c2.com/cgi/wiki?ExBase and http://c2.com/cgi/wiki?AreBusinessAppsBoring and I wonder if we can do better, and why so few people seem to be working on it.

REBOL seems to have been developed under similar constraints. REBOL programmers don't do much abstraction, but instead use a large number of DSLs to describe applications. Common requirements - http, e-mail, databases - tend to be supported directly by DSLs. A language 'Red' aims to address some deficiencies of REBOL (including its closed-source nature).

That said, I'm not especially fond of building discontinuities or 'walls' into the design. Cannot we have a smooth language, where users can graduate to reusable abstractions but also remain in touch with the concrete forms?

Yes, I need to follow up on Rebol (new official spelling). I don't know enough about it right now.

My experience has convinced me that, at least in larger systems, building in discontinuities and walls is vital. The main reason is to partition work for different skills. Secondarily it's for finding defects, by knowing they cannot cross certain boundaries. There are other reasons.

In this context larger means about 50K SLOC, bigger than a single person can build in any reasonable time. My reliable experience tops out under 1 million, although I've worked on a couple of bigger ones.

Do you really want to have to think about what's inside the OS or RTL whenever you write a line of code? That's the kind of partition I mean.

Do you really want to have to think about what's inside the OS or RTL whenever you write a line of code? That's the kind of partition I mean.

I find 'controlling what I have to think about' to be sensible. Our abstractions should support isolation of defects and confinement of side-effects. I'm fond of secure(able) language abstractions for this reason - cf. object capability model, value sealing, algebraic effects. I see these features as part of a continuous design, i.e. such that the inside of the language can be made to look just like the outside. I think it would be wonderful if users could extend their operating systems, runtimes, even favorite web-services in a secure way.

How is any of this related to hindering abstraction? That seems like an orthogonal issue.