Lambda the Ultimate

Notice that the problem with code manipulation is usually that the output code is not very readable. Of course, if the code is XML, no body expects it to be readable to begin with...

This is what Gerald Weinberg calls The Bolden Rule: If you can't fix it, feature it.

Well, you can manipulate code if you have a parser that produces an abstract syntax tree (AST). Using XML only helps if you are not able to find or build a parser for the language, so you prefer a standard XML parser.

Obtaining a parser is one issue. Another is creating the manipulation code. XSLT's syntax is nothing to write home about, but there is a reason why people use it instead of doing direct DOM manipulation in some other PL. Finally, you may need to bundle a compiler/interpreter with your app in order to run the resulting code.

If o:XML source is not intended to be written/read by human beings, great. Maybe it will make a good intermediate format for compilable languages to transform source into. That would be interesting.

Otherwise, why do so many XML projects expect us humans to deal with such a machine-centric syntax? Isn't this a step backwards from the idea that programs should be written for people to read? (let the compiler/interpreter do the hard part of translating it into something executable).

Indeed, it is.

Are you saying the XSLT is better in some sense than working on AST using modern programming languages, with pattern matching, algebraic datatypes etc. Or better than Scheme with cases or match?

If you are working with Scheme all the way then your assertions are certainly valid. In the context of the projects I mentioned this was not a valid choice because:
1. Scheme was not used to develop the applications.
2. The data was available as XML, not S-expressions.
3. Scheme was not installed on the target machines.

The point I was trying to make is that in some contexts it makes sense to have the data and code in the same format. This has been a historic feature of Lisp. It has also truned off quit a number of potential Lisp programmers.

A lot of useful data is now available as XML so it makes sense that code, especially code for manipulating XML, can also be represented as XML.

One method to overcome the obvious syntactic problems that arise is to have the code generated by some WYSIWYG environment. Another is to use XML as an intermediate format, as suggested by Todd.

It could probably qualify as a DSL, because it exists purely for HTML / XML templating purposes. It means to replace Zope's previous templating language in those areas by producing something with a core consistent feature set, a specific domain (while TAL can be used to generate plain text documents such as SQL statements and email, it's not built for that purpose), that could be used in modern WYSIWYG HTML editors without requiring custom libraries be built to understand the tags. Since it's native to XML and HTML, it's aware of its environment. You can not only replace attributes on a tag when the page is rendered, but you can access any attributes in the source document as well.

It's different from XSLT because it's not meant to transform XML to other XML - it's meant to be used for output from a web based application. Unlike o:XML, it doesn't provide any traditional imperitive/OO/procedural language features (to some programmers dismay, there's no concept of 'else'). And unlike Enhydra's XMLC, which aims to solve a similar problem, it allows a decent amount presentation logic to exist in the TAL code - but just enough to require any advanced logic to be written elsewhere (in constrast to full-on languages-in-a-big-string systems like PHP, classic ASP, and JSP; or custom SGML-ish string substitution languages like DTML and CFML which have traditional imperitive constructs like 'if/else', but were never intended to be full languages in the first place - even though they ended up being used as such). Overall, it's a pretty nice system to work in so long as you use it in its intended domain.

Regarding some of the technical details - in the Zope implementation, which is open for other systems to use (and some have), there are two parsers that compile into an intermediate language. One parser is specific to HTML (it still requires fairly well-formed HTML code), the other is for all things XML. Much of the implementation was done by core members of Python Labs. Those interested in the source for TAL can see it here. The execution steps are similar to o:XML - compile source into an intermediate language (Zope does this at object-write time, instead of first-run time), bind one or many contexts into the executable, and run it. Unlike o:XML, there is no need for DOM or XSLT since Page Templates are directed at producing output *only*, while leaving other programming constructs (like solving 8-Queens, or playing around with session settings) to things like Python Scripts.

Speaking of Page Templates - Hi Todd! (Todd Coram was responsible for a lot of the initial design specifications for TAL)

I'm even beginning to hear about what might be compared to "hygenic syntax extension", e.g. http://lambda-the-ultimate.org.

8^)

-Patrick

If you are working with Scheme all the way then your assertions are certainly valid. In the context of the projects I mentioned this was not a valid choice because:

• Scheme was not used to develop the applications.
• The data was available as XML, not S-expressions.
• Scheme was not installed on the target machines.

If the data is available in XML, we can easily parse it in S-expressions. We can even easier pretty-print S-expressions in XML or HTML (or LaTeX) afterwards. The issue of installing Scheme application on target machines isn't as difficult as it may seem: we merely need to compile our code. There are several Scheme-to-JVM compilers out there (see FAQ at www.schemers.org). There are several people working on Scheme-to-CLI compilers. If we distribute our application as a (statically compiled) ELF file or a .jar archive, we do not need to worry about installing Scheme on the target computers. I know of people who use SSAX and SXML tools in commercial companies with an explicit "only pure Java" policy.

A paper "XML, XPath, XSLT implementations as SXML, SXPath, and SXSLT" gives several examples of using Scheme to read, write and transform XML in commercial and government settings.
http://pobox.com/~oleg/ftp/papers/SXs.pdf
http://cvs.sf.net/cgi-bin/viewcvs.cgi/ssax/SSAX/docs/SXML-paper.tex?rev=2.6

A lot of useful data is now available as XML so it makes sense that code, especially code for manipulating XML, can also be represented as XML.

I'm afraid I don't see how that follows. I have seen many arguments that XML is a rather unsuitable notation for a programming language. For example, Adam Bosworth, vice president for engineering of the leading Web Services vendor BEA Systems Inc., says, "It turns out that there are things that XML isn't well suited for, and expression languages that humans can read and write easily is one of them. Nor, it turns out, are programming languages encoded as XML grammars."
http://www.fawcette.com/xmlmag/2002_02/magazine/departments/endtag/

Incidentally, that article says that a W3C working group in charge of XPath has looked into the XML notation for XPath. They quickly realized that such a notation makes XPath incomprehensible. They deliberately opted for a "more conventional" notation. I should also mention an alternative notation for RELAX-NG, a (better) XML Schema language. The alternative notation is not based on XML.
http://www.thaiopensource.com/relaxng/compact/

If you insist on the XML notation, we easily can pretty-print Scheme code in XML: http://pobox.com/~oleg/ftp/Scheme/xml.html#executable-XML

At the ICFP'02 banquet, Philip Wadler wore a T-shirt that read:

       2x                   (Descartes)


       \x.* 2 x            (Church)


       (lambda (x) (* 2 x)) (McCarthy)


       <abstraction>        (W3C)
        <var>x</var>
        <expression>
         <application>  
	  <const>*</const>
          <arguments>
	   <const>2</const>
	   <var>x</var>
	  </arguments>
         </application>  
        </expression>
       </abstraction>

A lot of useful data is now available as XML so it makes sense that code, especially code for manipulating XML, can also be represented as XML.

I'm afraid I don't see how that follows. I have seen many arguments that XML is a rather unsuitable notation for a programming language.

Exactly the point I was trying to make, if you carefully read the preceding paragraph. What I was trying to say is that because XML is a universally supported format, because of all the useful data that is available in this format, there are advantages for storing computational instructions (code) in the same format. This is why I like Todd's suggestion of XML as an intermediate format. I don't like to manually read or write XML any more than you do.

The issue of installing Scheme application on target machines isn't as difficult as it may seem

That depends. If the target machine is a server or belongs to a power user, you are probably correct. If OTOH your app is intended for download and automatic installation through the Web, its a different story.

Thanks to MS we can no longer count on the target machines having a JVM preinstalled. You can use Sun's JVM, but they tend to move it around a bit. Also, it's quit a hefty download. OTOH MS have made sure the most every target machine has XML support preinstalled.

Writing XSLT isn't fun, but it's not that bad either, especially using a tool like XMLSpy. Once written XSLTs don't tend to change a lot, and even if they do its easy to find a programmer to do it. A lot easier than finding a Scheme developer. And XSLT is easy to activate from most every environment/language.

There was another point I didn't make because I didn't want to make it. There is a marketing advantage in using XML. There is no such advantage in using S-expressions (though I can use them internally of course, without telling anyone).

to quickly present my case - i wrote o:XML as a response to what i feel are shortcomings of existing XML tools and because i felt i could really use a 'proper' XML programming language for XML manipulations and processing. sort of like xsl, but oo (because that's the programming paradigm i'm most comfortable with myself) and with a focus on generating XML rather than transforming it (this makes it a complementary technology to XSLT and not, as some may people think, a competetor).

firstly i just have point out - i'm amazed that no-one has! - that most computer languages are not 'human readable' to the majority of this worlds humans, and the ones that are are generally not that good. take the example of C - what makes C successful, and a good language, is not its instant readability but its condensed and expressive syntax. the strongest argument against formulating a computer language in XML i believe is one of verbosity - programmers are lazy and don't want to type more than necessary, and verbose code is harder to take in and make sense of. one of the main design criterias for o:XML was to minimise over-verbose constructs - compare for example an o:XML procedure call with a couple of parameters to the equivalent XSLT named template call.

oh, and i'm not posting here to push o:XML, i just came across the discussion and thought there were some interesting points made.

i'm currently in the process of extending o:XML with assertions and invariants to implement programming by contract, and i'm interested in developing the language and the ideas further. if anyone has suggestions or comments i'd be very interested to hear from you!

/m