The best way to bridge this gap---indeed, the only way---is to let programmers control how linkers, debuggers, and other tools handle generated code.
Plug-in APIs are nice, but the story needs an intermediate compiler language to really come together, especially on this debugging problem, which the author considers primary. An IL refactors the tool chain. It breaks down monolithic compilers and offers a natural insertion point for plug-ins (if not the only one). An IL provides natural scaffolding for the debugger and linker. Programmers could even hand-edit the IL now and then.
In the case of numerical languages like Mathematica and MATLAB, using XML for storage would allow programmers to put real mathematical notation directly into their source files.
His prior remarks had me thinking about Mathematica as a case in point even before this direct mention. It is a full programming language, not just a math toolkit. Actually, Mathematica does XML extremely well, and the company even drove development of MathML. It already puts "real mathematical notation" and cool vector graphics directly in source files. The XML argument is not what got me thinking, though.
Mathematica bears interesting resemblances to Lisp and Scheme. Code is data, everything is an S-expression. Mathematica's S-expressions are somewhat more rational, even if overall syntax has little else to commend it. I don't get hung up on syntax, though. That's why I seldom use Lisp or Scheme, in fact - I have Mathematica. Mathematica has a much stronger model/view paradigm. In fact it's strong enough to factor out two application programs, the "kernel" (model) and the "front end" (view). There are even commands to change the view, e.g. "StandardForm," "TraditionalForm," "FullForm." I very much like the model/view notion for source code.
Mathematica is a beautiful but proprietary system. The interesting aspect is how compiler control folds into the system. A "Compile" function turns S-expressions into faster IL (which is its own S-expression, and editable). However, the style of well written Mathematica code is highly functional (not imperative), so it relies on inference capabilities to select appropriate numerical algorithms and data layouts. Hence the compile function is rarely used. Some implementation details are allowed to peek through for fine-tuning, such as the packed array and sparse array directives and associated conversions.
Its "pluggability" comes from MathLink, which is just S-expressions over a wire. The kernel may be treated as a "component" of other programs, or may use other components (i.e. C/C++ functions). It is possible to write your own front end. Kernels can share computational burdens. There is a language subset for GUI development in the front end. The J/Link software that ships with Mathematica is a Java implementation of these concepts. I have used the C equivalents many times and find the development cycle highly productive.
Mathematica does a good job of meeting the author's criteria, at least for a proprietary offering that can only open itself to the extent that valid business considerations allow. Where Mathematica breaks down is debugging, which is often hard and involves tedious manual parsing of output. Still, it's one of my all-time favorite tools.
My main critique of this paper is that it falls a bit hard for the fallacy that XML syntax solves everything. As the author notes,
Language extensibility has been around for years, but is still largely an academic curiosity. Three things stand in the way of its adoption: programmers' ignorance, the absence of support in mainstream languages, and the cognitive gap between what programmers write, and what they have to debug.
The basic argument is that XML has finally moved the inertia of the unwashed masses, who are focused on syntax. So even though Lisp could do this stuff 20 years ago, now is the time to seize the bull by the horns. Well, OK: we have a new storage syntax standard. We still need semantic models by which the dev tools can interoperate. Encoding them in XML is a side issue. Is the story that we had to wait for the unwashed masses to agree on trivia like syntax before we could develop semantic models?
In the user interaction plane, Eclipse is the best hope for an extensible dev tools framework. The article gives only passing mention to Eclipse and implies falsely that it mandates a storage format. It is completely open.
The IL issue remains outstanding. An IL should incorporate type annotations anyway, and in so doing, enables clean debugging. I'm not sure .NET is an answer. That involves JITting and Microsoft anyway, so I would vote for some open-source, cross-platform alternative.