Lambda the Ultimate

XML has several problems. It is slow to parse, it can result in big files, it only describes what the data are and not how to handle them etc. But it is a step in the right direction: we need to have a way to make programs interact based on a common protocol. In other words, we have a problem (interaction of programs), the correct approach (a common protocol) with a wrong solution (xml).

Maybe if every computer came with LISP (or something similar), computers could use that as a protocol instead of xml or any other protocol. If you thing about it, there are lots of protocols invented to solve the same problems (or the same nature of problems).

Take html for example; any HTML document can be represented with LISP lists. In fact, if we had LISP data sent over the network, we would have less data to send over (as a LISP program can be sent as a list of tokens); we wouldn't have to invent javascript, dhtml or ajax, as LISP data are both code and data at the same time. We wouldn't have to use SQL, since database schemas could be represented by LISP lists. We wouldn't have to invent SOAP, since LISP data could be used for sending requests over the network; we wouldn't have to invent OpenDoc or document formats, as LISP lists are more than sufficient to describe any format. We wouldn't have postscript, as postscript is nothing more than a bunch of instructions ala xml/LISP. We wouldn't have command line interpreters, since LISP can also play that role. We wouldn't need COM or CORBA. And on top of that, development could be bottom-to-top instead of top-to-bottom, since LISP development is usually interactive.

And on top of all these things, we also have managers that are clueless in technical details, and they insist to use XML because it will look good on the company's resume...

Lisp also has a complicated syntax (quote, backquote, lots of lexical token types) and it not much less verbose than XML. It has the same lack of inherent semantics that XML has - in fact its data-representation semantics are weaker, since it doesn't have an element/attribute/content distinction. It lacks standards like namespaces and Xpath that allow you to express more complex structures than simple trees.

Lisp data is only code if there is a suitable set of function definitions, and these would have to be standardized for interoperability. The same is true for all of the other "just use lisp instead" things you list above. Without these standards there is no way lisp could support the kind of software egology that has grown up around XML. As a concrete example (going back to the origin of it all) how would your ideal lisp browser display a random bit of lisp as a web page?

"Lisp data is only code if there is a suitable set of function definitions, and these would have to be standardized for interoperability."

yes, but well, there is standardised functions. Just as in any programing language.

And i guess a browser could display random lisp a bit like this. But I dont know if its optimal as I havent looked at it much.

If you take Common Lisp or ISLISP, you have an attribute/value distinction (property lists and keyword arguments). Scheme also has association lists. Common Lisp has namespaces (its package system). I don't know enough about XPath to be able to comment on that.

Common Lisp is an ANSI standard, ISLISP is an ISO standard, Scheme has an IEEE standard.

I agree that Lisp has somewhat complicated syntax, but the fact that the syntax is programmable more than makes up for the lack of data representation semantics. As for standardization, it's a matter of decisions.

I do not undestand how your question of 'how to show random bit of Lisp as a web page' has actually any meaning. The point in Lisp is that it has a syntax that can be extended, which means that an XML-like DSL could be easily specified, as well as be extended with more features if the need arises. The real advantage here through would not be the DSL, but the ability to validate/interpret/process it without any other tools than the interpreter.

The general tone of the preceding post seems wave a lisp hammer in a world of nails, and some of the arguments, such as "We wouldn't have to use SQL, since database schemas could be represented by LISP lists" just can't inspire any kind of reasoned reply, but let's just stick to the central argument: This little chestnut about XML being some kind of brain-damaged version of sexps gets thrown around every time the subject of XML comes up, so it's worth seeing the opposing viewpoint:

http://www.prescod.net/xml/sexprs.html

Specifically, while sexps may do a great job at structuring hierarchical data, they're lousy at markup. I can't agree with Tim Bray's argument either, but he only seems to be misguided from a political viewpoint, not a technical one.

Philip Wadler has a paper where he briefly compares XML and s-expressions. From The Essence of XML: "XML is touted as an external format for representing data. This is not a hard problem. All we require are two properties:

• Self-describing: From the external representation one should be able to derive the corresponding internal representation.
• Round-tripping: If one converts from an internal representation to the external representation and back again, the new internal representation should equal the old.

Specifically, while sexps may do a great job at structuring hierarchical data, they're lousy at markup.

This is probably true and one reason why SGML was invented. SGML is great at markup and lousy at representing hierarchical data. It's quite complementary to sexprs.

Now look at XML: It sucks at both tasks. That makes it universal, at least that's what the industry is trying to tell us.

XML has that engineering quality of striking a balance between several opposite requirements. This gives it a huge edge over both Lisp and SGML in the markup world. Making good trade-offs is the soul of engineering.

Without wanting to start a language war, but can you back your claim up? SQL is a language for expressing database operations...A programming language like Lisp can easily be used to provide a custom DSL like SQL for database operations.

As for the article about s-exprs, let's analyse it a little:

"There is no standard syntax for attributes across Lisp-like languages"

That's not a criterion for language quality. It's a matter of circumstances and politics. If Lisp vendors decided they wanted a standard syntax for attributes, they could make one. The point is that the mechanisms of their language allow it.

"To me, the XML version looks easier to read"

That's subjective, as the author says, but if the XML document is 10 pages long, is just as unreadable as Lisp.

"The XML one defaults to treating random characters as text, not as markup"

So? big deal. How many times there are "random characters anyway? Lisp is a programming language. String literals are enclosed in double quotes. Where XML can stand for representing information, it can not do anything else.

"The XML one does not use standard human-punctuation characters as markup."

Well, the character are special. It is the symbols 'less than' and 'greater than' respectively. Show an xml document to a mathematician and he will have a hard time digesting things like "paragraph

"Redundancy Is Good"

No one stops you for using an extra word in your DSL at the end for this purpose.

"So in my opinion, XML's syntax is wildly better than S-expressions as a language to integrate the worlds of documentation and data."

I failed to see the 'wildly better' thing, and the author simply misses the point: the real power of Lisp is the ability to invent any DSL needed for a specific task, as well as code and data being intergrated.

"DTDs, RELAXand XML Schema define constraints on individual instances of XML documents. This is a necessary feature for web services."

A Lisp like environment is much more open to checking constraints, since it can be interactively programmed to do so. Extending a web server with new functionality is much easier with an enviroment like Lisp than with xml.

"XPointer allows change-resistent addressing of particular nodes in the parse tree ("infoset") in URIs.
"XQuery allows the expression of sophisticated queries on XML documents"
"XSLT allows the declarative, patternn-based transformation of XML documents"
"CSS and XSL allow XML to be presented to human readers with formatting"

All of these things can be done in a Lisp-like environment very easily.

"You cannot evaluate the hype around XML without incorporating all of these technologies into the evaluation. Cumulatively there are decades of person-effort embodied in those specifications!"

The effort would be much less if something like Lisp was used.

" The central idea of the XML family of standards is to separate code from data. "

Which it never actually happens with xml: most programs, if not all, use xml for communication with other environments, converting xml from/to a set of native objects. And two applications can share the same xml information, but if they need to manage it in the exact same way, they need to share the code too.

"The cental idea of Lisp is that code and data are the same and should be represented the same. The Lisp community's idea of "Schema" would likely be "Lisp program". The Lisp community's idea of "addressing language" would likely be "Lisp program." The Lisp community's idea of "query language" would likely be "Lisp program.""

Good, at least the author understands a few things.

"Unfortunately this response ignores the Principle of Least Power"

I don't see how that applies to a Lisp-like environment.

What the author cleverly sidesteps is just how powerful is a system like Lisp. It can be procedural, functional, object-oriented, purely declarative, mathematical, etc. I am not praising Lisp, I am praising the idea of macros, extendable syntax, data as code and vice versa, etc. Lisp has its flaws, and there are various implementations around, each one with its own problems, but there are incredible ideas behind it, not properly exploited by IT.

"The XML one defaults to treating random characters as text, not as markup"

So? big deal. How many times there are "random characters anyway? Lisp is a programming language. String literals are enclosed in double quotes. Where XML can stand for representing information, it can not do anything else.

Wouldn't you say that it is this property of XML (it's (dis)ability to slurp up everything that isn't explicitly markup as strings) that forces one to "define constraints on individual instances of XML documents" using things like "DTDs, RELAXand XML Schema?" Lisp has a type system; it doesn't treat everything as a string by default. You can declare types in Lisp without resorting to the use of external verifiers. This is good for storing and transmitting data. The fact that XML itself is so bad at defining and structuring data is evident in all of the external documents and tools needed to verify that the XML document is valid. However, I think that many in industry(especially those who don't have to deal with the XML Schema's directly, i.e. managers, etc.) like the idea of having a separate document that defines the types and structure of XML documents, even though such a system is less flexible than specifying types and structure in the document itself. I suppose it gives people a sense of security: we have this document that defines our XML right here, so we know our XML is valid.

In theory, no, but in pratice it matters a lot.
The reason why XML matter is that it has one unique syntax, not many which help synchronisation of course.
Of course, they dropped the ball for XML schema (and IMHO XSL suck, but..)

As for '<>' against (), if one use the XML view that everything is text then () have a disavdantage as they are quite common in text (just look at this webpage) so this increase the usage of escape character which lower readability..
Maybe LISP should use [] or {} instead :-)

That's even worse :-)

It means that inventing languages is hard, which justifies a dedicated weblog! He should have said: Don't invent languages, unless you read Lambda the Ultimate.

:-)

But Tim Bray's site is worth a visit all the same. Check out the flower photos.

Whenever you write a library/API, you are designing a new language, i.e., a family of operators that are about to be used together in a certain way. Are you suggesting that we shouldn't program if we don't have to? ;)

A family of operators is not a language, not even a dialect. It is at most a jargon ( or gibberish ;)

Kay

This is not just a bunch of operators thrown together, but they are to be used in a certain way - in a sense, they have to be obey a "grammar". I would already consider this to be closer towards a language.

I would claim that most APIs have such non-trivial interdependencies between operators.

There is nothing in the English language that enforces I use the following three conversational phrases in particular order:

• "Hello"
• "Nice weather we're having"
• "Goodbye"

Consider "We weather nice're having." You would get a bad mark for this in school, right?

Furthermore, I vaguely recall reading about an approach that makes it statically checkable that open, read and close are called in the right order, but I don't remember where I have read about it. As another example, design by contract is, though not static, at least an approach that claims to help to detect these problems earlier. Etc.

This is a big problem in Tim Bray's arguments! He says: don't invent new XML languages, but use microformats. But all his arguments apply to microformats as well.

I couldn't agree more with John Mitchell. Programming is language creation mostly. All these efforts (AOP, LOP, etc) and every API designed are mostly language creation tasks. And since language creation is the main task, the best way to "program" is to work with a tool that allows "language creation"...i.e. a system open like Lisp.

The link leads to an article about Language Oriented Programming. It's funny that it is mentioned as the "next programming paradigm", when it is over 50 years old. That's why I said "IT is on an acid trip."

XML is not a language, nor are its subsets like XHTML and Atom. They are data formats

XML isn't a "language" in the programming-language sense; it really isn't a "data format" either. Rather, it's a specification for creating text-based data representations, all which operate with a common substrate. Those text-based data representations may or may not be languages--XSLT is generally considered to be a programming language by most definitions. OTOH, an XML application for describing a purchase order, would not be. XHTML is complicated and interesting enough that I consider it to be a fine example of a DSL.

One final note, writing new programming languages isn’t that hard either.

Well, implementing a programming language might not be that hard, but designing one certainly is. Otherwise how do you explain the fact that many (most?) programming languages are severely broken in one respect or another? Designing a programming language typically means making tradeoff decisions here and there, and although a decision made may seem right at the time, it may turn out to be very wrong. A good example is C arrays. When C was created the tradeoff between speed and safety was typically biased in favor of speed, hence the unsafe, but very fast C array. Unfortunately, that tradeoff ended up being a very bad decision for two reasons:

1. Computers soon became fast enough (depending on your definition of soon and fast enough, of course) that such a decision was no longer warranted (except in the most critical of applications).
2. Although the cost of the decision is hard to estimate, the damage is very clear: many pieces of malicious code have taken advantage of the weakness of the buffer overflow, a direct result of unsafe C arrays. The cost of this damage could be in the millions, or even billions of US dollars.

Fortunately for the modern programming language designer, there is a lot of past research and experience regarding the design of programming languages. Also, there are forums such as LTU that have many bright individuals (excluding myself of course; maybe some day I'll be among those who understand this stuff ;) who are very knowledgeable of programming language theory. This makes the job of designing a programming language easier, as there are many people who know the tradeoffs and have experience as to what the better choice is.

Otherwise how do you explain the fact that many (most?)--snip--

I think the word you're looking for is "all".

Also... I dispute that there's a "better" choice for every dilemma in PL design. Choices seem to combine in a way that has non-linear effects on the value of the resulting language.

That's why it's better to have a language toolbox that allows you to create your own language for the task at hand.

Also... I dispute that there's a "better" choice for every dilemma in PL design. Choices seem to combine in a way that has non-linear effects on the value of the resulting language.

Yes that is true. In fact, I would argue one of the most difficult aspects of language design is deciding what tradeoff to make for your particular circumstances. I suppose that is why they are called tradeoff in the first place: your making some sort of compromise.

Key word in his post was "domain specific". Most programming languages suck because they have to balance a wide variety of tradeoffs; domain specific languages have a much narrower requirement, and hence don't need to make so many tradeoffs. I've done written a DSL in a day that works perfectly well for my purposes, cuts my job by 3/4th, and has already saved me significantly in maintenance work. I wouldn't think of using it for general programming (in fact, I can't - it's tied to the specific framework and task it was made for, and isn't really Turing-complete), but it gets the job done fine.

I think that's an important statement. You only gain experience in doing something by actually doing it.