Purpose-Built Languages

Mike Shapiro, Purpose-Built Languages, ACM Queue vol. 7, no. 1, February 2009.

Mike Shapiro from Sun Microsystems examines the evolution of what he calls "purpose-built languages" (essentially domain-specific languages). Shapiro discusses the way that such languages have often been a key part of the development of larger software systems, and describes how many of them have sprung into existence and evolved without formal design. In particular, he traces the evolution of the adb debugger language. As Shapiro says:

The debugger tale illustrates that a little purpose-built language can evolve essentially at random, have no clear design, no consistent grammar or parser, and no name, and yet endure and grow in shipping operating systems for more than 40 years. In the same time period, many mainstream languages came and went into the great beyond... For purpose-built languages, a deep connection to a task and the user community for that task is often worth more than clever design or elegant syntax.

The same article also appeared in the April 2009 issue of Communications of the ACM.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Languages also evolve by replacement

Mike writes that adb was added to SVR3 in 1980. By 1986, my group at Bell Labs was working on a replacement for both adb and sdb. There were several motivations for the new debugger:

  1. AT&T was then supporting three architectures (IA32, SPARC, WE32000), and needed a debugger that was both more portable/retargetable and also one that could debug cross platform. Our group, for example, was the origin of the DWARF debugging format. Some here may recognize a peculiar s-expression character to portions of DWARF...
  2. In parallel, the new ELF object file format was emerging for much the same reason. This almost forced us to rebuild, because neither adb nor sdb were maintainable.
  3. With the arrival of C++ and Ada, we needed to do a better job on multiple source language support.
  4. AT&T was slowly discovering the internet, and wanted to look at remote, multiprocess debugging.
  5. It was recognized that the command languages of both adb and sdb were a lost cause.
  6. We wanted a GUI front end for both X10 (no, that's not a typo) and the Blit. But also, we needed a command line interface that could cope with asynchrony (necessary for multiprocess).

Regrettably, the so-called "UNIX Debugger" never made it out as a product. More precisely, it emerged once with an sdb front end tacked on because that was the fastest way to bring up sdb for one target. Users mainly noticed that the sdb code size suddenly jumped to 3Mbytes (at the time that was big). A later, structurally similar debugger would ship from my group at SGI, and became the basis of the SGI "View" line of debugging and performance analysis tools (since renamed ProDev). In its original form, ProDev was capable of distributed debugging and shared sessions. Much of that was removed in the product version because the networks of the time weren't fast enough to support it well.

In designing the command line interface for debug, we borrowed heavily from the Korn shell (which was then new and not yet standard) with Dave's support and help. This gave us a more or less coherent and familiar command line system, and it additionally gave us the ability to use debug as a primary shell -- which some of us did. ksh already had a mechanism for "built-ins", and we simply extended this to cover debugging commands. This was somewhat similar to what John Ousterhout would later do with tcl, except that we retained the control structure and more coherent syntax of ksh. Along the way, we ended up creating a "shell library" so that we could kick off different scripting contexts, somewhat similar to what would later emerge in the scripting language world.

It is somewhat embarrassing to realize, looking back, how much value some of those components would have generated as separate tools/libraries which we utterly failed to recognize. All the more so because one of our team members (Mike Bianchi) was on a holy war to promote the importance of library reuse (dynamic libraries weren't yet widespread). The GUI world was just starting, and the time to recognize the importance of scripting languages wasn't quite ripe -- or at least we failed to see it.

The attention to multiprocess issues forced us to consider scripting and trigger issues fairly carefully, and this more or less forced us to adopt (or at least borrow) a coherent language structure. This was more or less reinforced by the embedding problem -- we needed to be able to embed expressions from multiple source languages in commands.

To me, one of the interesting lessons out of those two debuggers was that asynchrony and synchronous languages can often be welded together. In both debuggers, there was an asynchronous event and messaging layer, but trigger scripts were written in conventionally sequential form. When I later saw Mark Miller's work on E, where event-driven style is truly well integrated, I was struck that perhaps we might have done better had we stopped to think about it. But we weren't a research group, and that wasn't the time or the place.