Lambda the Ultimate

Because Lisp is relatively close to the untyped lambda calculus, the observation is almost a truism. The interesting question is whether the untyped lambda calculus is the most appropriate starting point for a programming language design today, and if not, how much the resulting language still resembles Lisp. if, as I tend to feel, "all languages eventually tend towards being Lisps" means little more than "all languages eventually tend towards having first-class functions," then I agree. If it means "all languages eventually tend towards being dynamically typed," then I disagree. If it means "all languages eventually tend towards having s-expression-based syntax," I laugh hysterically.

EVAL
Closures (Eiffel calles them Agents, C# has them, Java says Inner Classes)
Symbols (Ruby has it some how)
Macros
Late Binding
Garbage Collection
Runtime Dynamic Features (see Erlang)
Blend of imperative, functional and object-oriented features
Interactive Programming (REPL)
Runtime safety

Well, we might not have the prefix notation for programs, but we got lots of prefix notation for data and configurations. Called XML these days. It plays the same role. Where you had files with (person :name "foo" :age 40) you now have .... But it is not all data, it is also configuration. Where you had (define-window foo-window :title "foo" ...) you know have the configuration in XML as .... Same purpose.

OK, let's take this blow-by-blow, using MetaOCaml as our point of comparison:

• EVAL: I'll claim that brackets, escape, and run serve this purpose nicely.
• Closures: Check.
• Symbols: No.
• Macros: Check, via camlp4.
• Late Binding: No.
• Garbage collection: Check.
• Runtime Dynamic Features: No.
• Blend of imperative, functional and object-oriented features: Check, better than CL's
• Interactive Programming (REPL): Check.
• Runtime safety: Check.

And yes, we all understand that XML is often used to implement a non-Turing-complete language for configuration purposes, which s-expressions are also good for. But are you honestly going to tell me that MetaOCaml is a Lisp? Or are you going to tell me that you can't be a Lisp without symbols, late binding (which really needs defining), or dynamic runtime features? Then I think Smalltalk-80 would be a Lisp. It's interesting to note that, while searching around for good information on Ruby symbols, I came across a claim that Ruby is a Lisp-2, so the idea might not be as far-fetched as I think it is. But from my point of view, you're making my point for me: it isn't that other languages are becoming Lisps; it's that what it means to be a Lisp is becoming so watered-down as to be meaningless.

MetaOCaml got backquote and comma? Wow. ;-)

MetaOCaml is sure no Lisp.

Lisp without symbols? No way. No late binding? No dynamic features? No, not really.

Smalltalk 80 is no Lisp in a narrow sense, but it comes very close. But that is not surprising, since Kay and others learned from Lisp and Lisp developers also learned from Smalltalk. InterLisp-D and Smalltalk 80 were developed at the same time next to each other. If there is one system that is like Smalltalk 80, it is InterLisp-D.

I don't think the essence of Lisp is that fuzzy that it is meaningless. Some feature set has been found almost 50 years ago. Some facilities have been added over the years. If you look at trends, I think it is observable that some new languages have a large amount of similar features which Lisp had for most of the last 50 years. There are also crucial Lisp features like symbolic expressions, runtime safety (without that interactive programming is no fun), program generation, reflection, introspection, garbage collection, interactive development that have been (partly) added to language families that have a distinct non-Lisp look and feel.

I don't think that other languages will morph over time into Lisp, but there are sure more features to be added to some of those languages - features that are thought as common in Lisp. Also there is some trend to more usage of dynamic languages.

runtime safety (without that interactive programming is no fun)

The Forth people will beg to differ.

Good one. No rule without exception. ;-)

Of course, I didn't think you meant that MetaOCaml is a Lisp. Rather, my point is precisely that Lisp's influence is, indeed, felt far and wide without its successors necessarily themselves being Lisps, and that this isn't sufficient to rise to the level of failing to understand Lisp and, hence, reinvent it. On the contrary, I would say that many modern languages, including Ruby, OCaml, MetaOCaml, Haskell, Scala... understand Lisp extremely well and go beyond it in important ways. Consequently, I'm afraid that I'm having difficulty thinking of anything that, for example, any of the aforementioned languages would benefit from adding from, say, Common Lisp that they haven't already, albeit in the modified forms that makes those languages not-Lisps.

Macros?

Has been investigated in the context of OCaml. Haskell might not need it that much.

This is indeed what people most often say should be borrowed from Lisp, and it's a major reason that I maintain that the Lisp community is out of touch with the rest of the world. At the very least, OCaml has macros as the Lisp world understands the term: in compiling, the parsing phase is distinct from the other phases, and there is a tool, camlp4, with which you can modify, extend, or entirely replace OCaml's AST. OCaml even ships with two official syntaxes out of the box, camlp4o, "original" and camlp4r, "revised" syntax. There are some very useful syntax extensions out there, such as pa_monad, which adds monadic do-notation to OCaml. Graydon Hoare's excellent "One-Day Compilers" relies heavily on camlp4: his compiler basically transforms the AST of his language to the AST of an OCaml program that spits out the equivalent C, compiles the OCaml program, runs it, and compiles the resulting C program. Very slick.

So this is one of a few respects in which the appropriate response to Lispers saying "You should try X" is "We did. Good job, and thanks."

I doubt that a Lisp user will care whether OCaml has macros or not. I strong suspect that the pre-parser for OCaml has been developed NOT by Lisp users. So, I really don't understand your remark.

Lisp users are out of touch in that they belive that macros are a thing thats exclusive to lisp. They belive that they have to use lisp to get macros, and in so doing there choice of lisp as language of choice is based on false premises.

I don't think this is true. But it is important to note that Lispers deny the value of other macro systems. And macro systems are indeed not rated very high in other languages ( they are more nice-to-have than being an essential feature ). The justification for s-expr syntax depends on the believe that any language with non-homogenous syntax makes code transformations either hard or do it in a broken C-like fashion using textual substitutions. One has to provide more arguments than just stating that language X has a macro system / AST based preprocessor. Lisp macros are straightforward in Lisp. It is not surprising that they are present and they are not only used for a few language oriented tools like refactoring browsers or DSL implementations. But then one cannot really learn anything from Lisp macros in order to transfer them across languages but just use Lisp or its derivatives instead i.e. adopt Lisp for programming in a Lispy fashion. A similar situation exists with Haskell and monads. Some language features resist to get transfered ( or they are getting transfered as toys or for demonstration purposes ), not because it would be technically impossible or even particularly difficult but it's just hard for other language communities to see the point.

Yes, I didn't mean to say that all lispers are out of touch or something like that. But more like explain how it is out of touch for a lisper to not know about campl4, just you are out of touch to belive that repl is a feature unique to lisp. And I have myself made these mistakes, comming from a C, pascal and basic background, I found Common Lisp and it was an amazing life changing experience, but the lisp community let me to belive that it was just lisp, that lisp was the only language that let me do these wonderfull things. That all other languages was way behind. And that is most definitly out of touch.

Having said that, I feel that the lisp macro systems are way more user friendly than campl4 or any other non-lisp macro system I have seen. But still I think reficating ASTs as lists and symbols is not the ultimate way to do macros eighter.

You are making the same mistake like Paul Snively if you think Lisp users are out of touch. Especially the gross simplification about 'the Lisp community'. If you spoke to a few of them, I can tell you that there is a huge chance that you missed a lot of views. The 'Lisp community' for example consists of:

* implementors of Common Lisp, Scheme, ...
* implementors of commercial systems (Allegro CL, LispWorks, Chez Scheme, Corman Lisp, ...)
* implementors of free Lisp systems (GPL and PD)
* lovers of 'free software'
* programmers in companies who develop tools and applications
* users of embedded Lisps (AutoLisp, Emacs Lisp, Nyquist, ...)
* hobbyists
* lunatics
* people working on standards (like R6RS)
* scripting fans
* academics
* people working for government agencies
* ex-Lisp users
* Lisp-1 fans vs. Lisp-n fans
* people who never post to c.l.l

anybody saying 'the Lisp community' is this or that makes a statement about thousands of very diverse people and several groups without knowing them or having asked them.

Many Lisp users are well aware about macros and other facilities in other programming languages. The Lisp community has a long tradition of research into that topic (macros) and there is/was a lot of discussion of the problems and various approaches. Some Lisp people have earned money helping the development of other languages (for example the macro system in Dylan).

Same for the REPL. Ever used Smalltalk? Prolog? APL? Anybody with some clue knows that REPLs are widely used. Though I would say that the most advanced one was the listener on the Symbolics Lisp Machine. ;-) But even in SLIME/Emacs it is getting better. Did you see that you can click on the returned data in the SLIME listener and that it is mouse sensitive? SLIME has simple presentations in Emacs. People are working on similar facilities in McCLIM where you can mix textual and graphical output in the REPL with mouse-sensitive objects. Example. REPLs which present you forms for commands, can complete over various data types for input, provide context-sensitive help, have clickable menus for error handling and so on. If you think that a simple emacs mode for an impoverished interaction loop is the same thing, then I have to ask who is out of touch with what??? Some people like to develop in vi and a simple terminal. That's fine. But please don't tell me it is the same as a Smalltalk IDE or the IDE of LispWorks. Press help in a Lisp Machine listener, and you see that there are hundred or more commands. Go in a break loop and press help again, there are a multitude of things you can do there. For example you can show the contents of a stack frame (a function call), edit the data in-place in the REPL, redefine the function and and reinvoke it.

ASTs as lists is definitely not the ultimate way to do macros. Though in CL there is a bit more to it. But generally everybody knows that it is only a pragmatic choice. Scheme has other macro systems.

If you believe what some people tell you how advanced the language is, that's nonsense. What is advanced about CAR, CDR and CONS? But something else is quite advanced. The implementations of a language that has a lot of design choices and trade-offs. Implementations like LispWorks or Allegro CL are the result of work of twenty years. These implementations also have lots of improvements over what is in the language standard (Unicode, Environments, Hierarchical Packages, Threads, Networking, Portable GUI toolkits, one-click application delivery, generation of shared libraries, graphical IDEs, Interface Builder, Rule Systems, embedded Prolog, FFIs, OS interfaces, extensible streams, Meta-Object protocol, shell-access, style checkers, cross referencing, source locator, saving of images, ...) Lots of people demanded these improvements and paid for them. These implementations had a lot of progress in the last years.

But that there is progress even in the Lisp world, doesn't mean there aren't others. Microsoft, Borland, IBM (Eclipse), Cincom (VisualWorks), SUN, BEA, Apple, Jetbrains (IntelliJ)... all are working on development tools.

Please stop telling 'the Lisp community' is this or that... What ever you say, some subgroup will protest (like I did here). ;-)

I don't think you can define the Lisp Community as including ex-Lisp users.

I don't think that other languages will morph over time into Lisp, but there are sure more features to be added to some of those languages - features that are thought as common in Lisp.

When a language, despite not supporting s-expr style syntax, implements an AST based macro system ( MetaLua, Converge, ... ), Lisp lawyers will sue the language designer for violating the intellectual property of the Lisp community. As a consequence the accused and convicted programmer has to undergo a therapy where the beauty, poetry, art, historical relevance, superiority and what not of Lisp is explained to him. He has to publically admit that he was at least "inspired" by Lisp ( well, it doesn't cost much ) and respects the prior art of any modern language contruct by the Lisp community. There is no escape from Lisp since Lisp owns all your ideas in advance. Everything converges to Lisp in the end which is the ready-made of the end of the power hierarchy ( only quantum computing may change this but then we come up with quantum-Lisp ). The only possibility to esacpe the depressive fact of lack of originality is to convert to the one true programming language.

A serious nutcase who continues to dismiss the merrits of homogenous syntax after all is advised a second therapy where the evolutionary model of the Lisp community is explained to him: with macros you are, as a language designer on the same level as an application programmer. When we don't like your language=application design, we shoot your ideas down. That's how the whole computing science progresses and so it will ever be.

;-)

I'm not sure if this is a very popular way of talking about programming languages, but I'm pretty sympathetic to it, personally. This reminded me of something I wrote on the ll1 list some years ago:

If, on the other hand, the assertion is that any such basis must be Lisp, I disagree. In fact, a stronger statement is often made: any such basis _will_, de facto, be Lisp, whether the benighted OO people know it or not. This is the final major armament in the hands of position X, and it guarantees X's final unassailability (or so some believe): "Whatever you THINK you're doing, you're REALLY doing Lisp." Arguments of this form are enormously powerful and subversive, and at times they can serve to preserve power, and at times to undermine it.

On the note of smalltalk, while I don't know it myself in pretty much every other community I've seen people always seem convinced that all languages are converging to smalltalk.

I'd hoped that my comment was more along the lines of "the observation is tautological" than "the observation is wrong." After all, the phenomenon of Lispers hearing of some "new" invention and pointing back to its formulation in Lisp decades before is a very real one, and there have been many times when I've been the Lisper in question.

But it is nevertheless also true that I keep hearing what, to my ears, sounds like at least the insinuation that you can't do creative, exploratory development in statically-typed languages—that this remains an ineluctable advantage of dynamically-typed languages. This is false, and to the extent that it might be part of what people are driving at when they say things like "Whoever does not understand Lisp is doomed to reinvent it," it calls the latter observation into question. But an even larger point that I was trying to make is that if you are among those who believe that Ruby or Python are "reinvented Lisps," then I submit that your definition of "Lisp" is so broad as to be nearly meaningless.

Can you give some examples of widely adopted practices and tools? Not a horse with stripes sold as a Zebra.

How does it compare to, say, a typical Smalltalk 80 environment?

I think the crucial point is that environments such as OCaml, Haskell, Scala, etc. took a leaf from the Lisp playbook and offer REPLs. In the case of OCaml and Tuareg Mode, I can be entering code into a source file to be compiled later, but I can evaluate it in the toploop interactively while I'm developing. When things are gelling sufficiently, I can use omake -P to automatically keep my build up-to-date every time I save the file. So I have an incremental development experience from blank page and a prompt to full standalone binary with no discontinuities. That is, using SBCL + SLIME and using OCaml + Tuareg Mode don't feel appreciably different to me, apart from the greater correctness guarantees from OCaml and, incidentally in my case, the better, and easier-to-deliver, resulting code.

So, would you say that OCaml + Tuareg + Emacs is the state of the art w.r.t. to interactive software development with static functional languages?

First, I'm not even sure what "state of the art means" or why it matters. But even so, I would imagine that Cameleon, OcaIDE, Visual Haskell, etc. would be quite a bit closer.

My only point is that I have essentially the same interactive flow in EMACS + Tuareg as I do in SLIME, which seems to be the preferred Common Lisp environment these days.

It has a debugger in the REPL? Restarts? Break loops? Fix and continue?

OCaml has an excellent time-travel debugger for byte-compiled code that I highly recommend investigating. As much as I love to talk about OCaml, we're rather obviously OT at this point, so if you'd like, I recommend we continue via e-mail. You can reach me at psnively-at-mac-dot-com.

so, the debugger is a separate tool and not in the REPL. you have to set up debugging by compiling to byte code and run the debugging session separately. But that is not the interactive development you get from Lisp or Smalltalk. There it does not matter much if the code is compiled or interpreted. When you develop there, the tools are all available. The facility you describe is similar what you can get from other static environments also. But it is far away from what you get, when you start Squeak, LispWorks, Allegro CL, VisualWorks or similar implementations.

Btw., is the way language tools work and their facilities off-topic here?

Well, we're talking about one language and how it's used, and not informing the discussion much from the PLT point of view, except perhaps to the extent that OCaml is an existence proof that incremental, interactive development is quite possible in a modern, type-inferred, statically-typed language.

So as I've tried to make clear, I have both environments simultaneously: I have a toploop in which I can instantly, with a keystroke, evaluate an expression that I've just entered into a source file. Thanks to omake -P I can also keep a build that is literally as up-to-date as my last save of that buffer, which means that I can take advantage of ocamldebug at any time. The effect really is quite indistinguishable from that of an environment such as SLIME.

Incidentally, very few environments have time-travel debugging. Common Lisp and Smalltalk-80 certainly do not. Ironically, the only other such environments I've seen have either been even father-out (Prograph) or very forward-thinking C/C++ environments on extremely popular platforms (some versions of Borland's C/C++ tools, IIRC).

Really, at this point I don't know what else to say. Incremental, interactive development isn't a novel feature of Lisp or Smalltalk, and hasn't been for well over a decade now. The continued emphasis on it only serves to reinforce how dramatically out-of-touch the Lisp community is—and let me assure you, Rainer, as a technical reviewer for Peter Norvig's "Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp" and MacDTS' sole support engineer for Macintosh Common Lisp from 1989-1991, I take zero pleasure in the observation.

I'm not sure how much the Lisp community (is there one? or more? none?) is out-of-touch. I hear that sometimes. Though I have that feeling sometimes with the proponents of static functional languages (SFP), who develop ever more complicated languages with practical purpose for those complex facilities that gets ever smaller. The type systems are so complex and the machinery to work around them is even more complex. Little is done on making them useful for the application developer. MetaXYZ? Is that been used by anybody? All I get is an Emacs mode? What you describe as 'incremental, interactive programming' is certainly useful, but it is far from a developer experience you get with, say, VisualWorks or LispWorks.

It seems that Common Lisp is mostly driven by application development now and not adding even more complicated machinery is actually a good thing. Even an old implementation like GCL is being maintained, because it is used in applications for Computer Algebra (Axiom, Maxima) and Theorem Proving (ACL2).

As somebody with a Lisp background I can assure you that never in the last twenty years the applicability of that background has been greater than now. If you look around you would see lots of Ruby, Squeak, Python, Rebol, JavaScript and other stuff, where this background is directly applicable. Even though I'm personally interested in the dinosaurs (Lisp Machines), I don't feel that much out of touch.

He, even Apple adds garbage collection now and [[offset transformBezierPath:path] stroke] is an excerpt from Lisp code (a Cocoa interface for SBCL) I got from someone. ;-)

There was some time-travel debugging done in Lisp, but it never got very popular. See for example Lieberman's Zstep, developed with Macintosh Common Lisp.

...is the point flying by.

I'm not denying the advantage of having a Lisp background; I'm explicitly saying that having a Lisp background is a good thing, not only in the contexts that you enumerate (Ruby, Squeak, Python, Rebol, and JavaScript) but also in Standard ML, OCaml, Haskell, Scala...

It isn't the case that all you get with many statically-typed languages is an EMACS mode, although I don't quite see what would be wrong with that. It so happens that the most popular (I believe) EMACS mode for OCaml is quite good—good enough that a comparison with the most popular (I believe) EMACS mode for Common Lisp, SLIME, isn't ludicrous on its face. But even there, the only claim that I was making was that I have exactly as much interactive, incremental development "oomph" in EMACS + Tuareg Mode + OCaml as I do in EMACS + SLIME + SBCL, a claim that I haven't seen addressed here, let alone refuted.

I'm glad, too, that Common Lisp is relatively stable. But I would argue that the lack of a standard set of APIs for sockets and threads, among other things, hampers Common Lisp today. I'm also glad that Paul Graham and Peter Seibel have breathed new life into Common Lisp, but the pattern I see with Lisp investigation remains the same as it ever was: some folks are amazed by it and stay, a larger number are amazed by it and outgrow it, and far and away the largest number are repulsed by it. The Lisp community (yes, there are two, the Common Lisp community and the Scheme community; you're part of the CL one, as is my CTO, as am I) oscillates like a tiny crystal as it gains and sheds members—members who then take the concepts and ideals of Lisp with them, wherever they go.

Lisp has been around in one form or another since 1958, as you know. Scheme, arguably the cleanest, purest Lisp—the one that tends to be talked about in PLT circles—has been with us since 1975. Common Lisp was, of course, defined in 1984. Standard ML wasn't invented until 1975. The Curry-De Bruijn-Howard Isomorphism wasn't identified until 1980. Caml was first released in 1987, Caml Light in 1990, and Objective Caml in 1996. It should come as no surprise that there's more active research, and concomitantly less pragmatism (although, compared to Lisp, that's a highly debatable point) in the statically-typed functional language community. I'm ignoring Haskell here because its goals are explicitly research-oriented, although people are doing surprisingly practical things with it.

Finally, as I've done before, I'll refer to Scala and liftweb as an excellent intersection of theory and practice: a modern type-inferred statically-typed object-functional language targeting the JVM, complete with excellent Eclipse integration, and of course the requisite web application framework, which is being designed and implemented specifically in response to the shortcomings of Ruby on Rails, with which the author has a couple of years of experience under his belt. Update: There's a recent interview with Martin Odersky on Scala here.

So once again, I have to say that you're making my point for me: Lisp is incredibly influential, and that's a good thing. But that doesn't mean that newer languages are reinventing Lisp, nor—more importantly—that designers and users of newer languages have failed to understand Lisp. Many of us understand Lisp quite well—could even be considered experts in it—and have chosen against it both for good technical as well as good social reasons.

I'm not that far away from what you write in your last paragraph, though I'm more optimistic when it comes to Lisp (its community, its usefulness, its differences that justify its existence) and much much more pessimistic when it comes to Statically typed FP in in general and its outlook.

* Lisp is not 'relatively close to the untyped lambda calculus'. Which was your first point in this discussion. Which is especially true for Common Lisp, see the Pitman reference below.

* OCAML's toplevel does not give you an interactive experience like Lisp or Smalltalk (even SBCL has the debugger built in, allows you code patching and resuming, has dynamic updates for data changes and so on). Wasn't even old MCL much more advanced?

* Common Lisp with non-standard extensions is not worse than 10 incompatible static functional languages

* I don't see Scheme as the 'cleaner' Lisp. I see Scheme as a cleaner programming language in general. But as a Lisp it is less a Lisp than Common Lisp or other dialects. Lisp is dirty. Ugly. Pragmatic. Full of warts. Scheme has to little of them. It even defines a syntax for the language (see R5RS). ;-)

* most research in 'modern' type systems gives the application developer very little, the impact is not very high and the complication is enormous. I wonder when somebody will come up with a simplified FP language or if that is possible? The most interesting and innovative FP language that I have seen in the last years is Erlang. Erlang is not statically typed. Plus Erlang is driven by real applications.

* The Lisp background does not give me much when I look at Haskell. Central ideas of Haskell (Type Classes, Monads, ...) are just not present in Lisp.

* I don't think the 'Lisp community' is more out of touch now, compared to the past. There was always a bit 'crazyness'. It always attracted some people with unconventional thinking. That's also what I like. You can see a steady upswing in the last years. Comp.lang.lisp has more traffic than ever. Much more infrastructure is available for the community, several implementations are actively maintained, lot's of new libraries have appeared, ... and, no, it is not all because of Paul Graham (who, btw. actively hates CL ;-) ) and Peter Seibel. It is mostly because a young generation now takes over and seems to be quite active. If some of the prominent people are a bit silent the last months, I think it has to do that ITA has many of them working on their software (Airline Reservation System, ...).

If you say newer languages are not reinventing Lisp, then you are right (with some exceptions like Pico Lisp, newLISP, e7, Curl, L#, dotlisp, Arc, ...). Especially languages like Scala, Haskell, OCaml are completely different languages. Software written in those languages looks completely different, tools are different and the development process is also very different.

I think Kent Pitman's recent posting to comp.lang.lisp has nailed it down: CL is not secretly Scheme with bad syntax. ;-)

Side remark: If you read the Haskell community reports, you will find that they also struggle to be more practical. They also think about libraries, tools and applications. Plus Haskell got some traction with the Perl 6 'proof of concept' implementation.

Question: what are you doing with OCaml (if you are allowed to say that)?

(sounds like a good band name) but seriously folks,

I wonder when somebody will come up with a simplified FP language or if that is possible? The most interesting and innovative FP language that I have seen in the last years is Erlang.

is something I have to agree with (not that I'm anybody). I would love to get into Haskell and Scala but I'm frightened off by the patently (from the perspective of a Regular Joe, even tho I did SML in university) nutty over the top typing that goes on there. How am I really going to debug somebody else's library when they've gone crazy on the typing and I don't spend every waking moment thinking about those kinds of things? Maybe once you are into it it ain't so bad, but the learning curve seems pretty painful.

I was generally a static typing bigot, still probably mostly am, but reading about Dialyzer and Qi (and PLT has something, too, I think) has made me really start to appreciate the whole optional typing thing. I think. Haven't gotten to really use them yet, just toy around. Still stuck with Java and C++ at the moment. Have pity.

As long-time readers know, I look forward to being able to decide at what stage I wish to prove the absence of undesirable properties in my code, and under what circumstances. That would involve something in between traditional static typing, dynamic typing, or even "optional static typing," which is itself an all-or-nothing affair. In the meantime, even relatively weak static type systems such as SML's, OCaml's, Haskell's, or Scala's seem helpful to me.

I think what you describe, though, is consistent with a lot of people's experience, including mine for many, many years: this stuff is intimidating to read about, and when the syntax of the language in question serves up its own impediments (*cough*Haskell, OCaml classic*cough*) the burden can simply be too much. Honestly, the way that I've learned most of what I know about OCaml is by writing, frankly, a tremendous amount of broken code—sometimes broken in the sense that it wouldn't compile, sometimes in the sense that it would compile and do the wrong thing, sometimes in the sense that it would compile but not be any safer than it would be if I'd written it in Java or Lisp. What I've come to appreciate, over time, is that as I've learned how to address the first and third cases, the number of second cases has gotten smaller and smaller and smaller—and not once have I had to jump through hoops to satisfy a finicky type system for no good reason.

OCaml could still use some good tutorials, but I'm curious: have you looked at any of the Scala tutorials? They seem remarkably high-quality to me, and certainly don't seem to make the type system seem complex (to me)! Scala seems like an excellent choice for those looking at all this typeful PLT stuff with a pragmatic eye, and if you're an Erlang fan you might find Scala's actors particularly interesting.

Having said that, it's very nice to see Erlang getting the respect it deserves, and I would, as always, add Oz to the list of dynamically-typed languages well worth learning.

How am I really going to debug somebody else's library when they've gone crazy on the typing and I don't spend every waking moment thinking about those kinds of things?

Which is what people say about Lisp macros and dynamic types.

but what is the connection between those? None?

I'm frightened off by the patently (from the perspective of a Regular Joe, even tho I did SML in university) nutty over the top typing that goes on there.

I remember having a similar feeling several years ago (I know that I expressed it in the early days of LtU ;-) ). I remember getting a headache trying to sort out some complicated type, or type-inference failure, and wondering if this was worth it, or if it just got in the way, because after all I already knew how to program; what good was all this esoteric stuff?

Much like Paul's experience, sticking with the esoteric stuff and working through those "tricky cases" taught me that there was a LOT more to learn about programming and programs, and that understanding type systems deepened my understanding of how programs and programming really work, and which things are essential to them and which are just kruft that has accumulated based on the vagaries of slogging in the trenches.

Even if you end up using a language with less in-your-face typing, your understanding of it will be changed by the experience of understanding type systems.

Like a lot of things, including programming itself, when you start with it, it seems scary or difficult, but once you get the hang of it, it seems pretty straightforward, and you end up with an enriched arsenal of tools for thinking about problems and solutions.

A worthwhile investment, I'd say.

most research in 'modern' type systems gives the application developer very little, the impact is not very high and the complication is enormous.

I know I'd be interested in hearing more about any specific things/code-snippets which you've experienced that was complicated by types. It sounds like you gave Haskell a whirl. Was it the case that you couldn't write programs in the manner you liked? (either the algorithms, or because of some missing interactibility(?) of the REPL) Were the type error messages too daunting or obscure? Maybe it was harder to read other people's code? Problems using other's libraries/code? Too much academic literature, and not enough down to earth practical examples? Others points I'm missing? Were there other things (purity, laziness, type classes, etc.) about Haskell that you liked, if only you didn't have to contend with the static type system?

If you say newer languages are not reinventing Lisp, then you are right... languages like Scala, Haskell, OCaml are completely different languages.

Interestingly enough, some people think Haskell is a member of the Lisp family (the Preface, page v).

You make a lot of good points, and my sense has always been that we agree more than we disagree—at most, I think you see some things as essential to Lisp that I don't. That's fine; my desire is not to dissuade you, or anyone else, from using Lisp if that's what's appropriate and enjoyable.

My OCaml work isn't professional, and involves some fairly hairy noodling around with LablGL, the F-Rep approach to solid modeling, Oleg Kiselyov's delimited continuations and generic zipper, Chris King's FRP implementation, code reuse approaches inspired by solutions to the Expression Problem, and so on. My top-level goal has been to develop something akin to Tim Sweeney's UnrealEd. That's almost certainly too ambitious for a personal spare-time project, but it certainly has provided a large exploration space, and my day job being in Python these days provides a useful real-world contrast in the dynamic language space, although decades of experience with Lisp certainly provides that as well.

The most interesting and innovative FP language that I have seen in the last years is Erlang. Erlang is not statically typed. Plus Erlang is driven by real applications.

Armstrongs thesis (p. 3) talks about practical projects with Erlang in 1987:

Towards the end of 1987 the first major experiment with Erlang started—a group of Ericsson engineers at Bollmora, led by Kerstin Ödling, started work on a prototyping project. They chose Erlang to prototype something called “ACS/Dunder.” ACS was an architecture which was designed for implementing the Ericsson MD110 private automatic branch exchange (PABX).

I think we all can agree that type systems have made a lot of progress since then. Using Erlang which is 20+ years old as a motivation (?) for typelessness is not a very conclusive argument. People have even investigated Practical subtyping for Erlang.

Can anybody give me an indication what this 'lot of progress' is and what impact it has for software development?

I'd venture a guess and say that the main progress would be in terms of type inference, modules, type classes, and generic programming.

And concurrency.

if you have worked in sales, you would know that you need to present the arguments differently. A list of high-level buzzwords isn't something that convinces people. Start with a typical argumentation pattern:

1) a sentence making a claim
2) why is that?
3) an example that supports the claim
4) what advantage will it give me

If people here claim that so-called 'modern languages' (whatever that is) are 'better' (successors, more advanced, ...), I would think there should be some material that expands these claims and presents evidence.

Is there some material that descibes this progress in type inference, modules, type classes, and generic programming and presents advantages that are relevant for end-users (application developers)?

Yaron Minsk's Caml Trading: Experiences in Functional Programming on Wall Street in issue 7 of the Monad.Reader. (LTU discussion)

My claim was that research on *type systems* had made a lot of progress since the mid 80's. This basically means that quite a few programs we could not directly express under a Hindley-Milner type system in the mid 80's are expressible with HM+extensions that are known today.

My impression is that you seem to disagree that static typing is a good thing, which is perfectly fine. However, under the assumption that the language is typed, people usually want an expressive type system with nice properties.

An example of a typed language with commercial success is Java.

Actually I agree with most what you say here.

I think static typing is very useful, expressive type systems are fine - unfortunately it leads to lots of complications in projects and I know only few people capable of working with them. I can easily imagine software areas where I would use it. 'Unfortunately' my personal thinking is kind of incompatible with it. Atleast that's what I believe. I'm not the 'bondage & discipline' (see Hacker's dictionary) kind of person when it comes to software development. I feel most comfortable with evolutionary, flexible software development, ... where the software is more like a dynamic environment.

Btw., my own experience with Java says, that once you got the code compiled you only have left 90% of the work to make it function correctly. The type system of Java does not contribute much to get working software. At least that was my experience working on several mildly large Java projects (largest was around 20 developers over two years). Once you got the software compiled, you could be sure that a few hours later it was broken again. Some team member checked in software and claimed that it was working for him. Continuous Integration, running the test suites, showed a different result then.

The most funny quote about Java was lately from Steve Jobs when he was asked why there is no Java on the iPhone. He said something like: “Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain.”. ;-)

Do you know which languages are preferably used for iPhones application development?

A guess:

Apple uses C, Objective C and Javascript internally on the device.
The OS is C, the libraries/applications on top are Objective C, some of the UI probably is HTML/Javascript/CSS (this is a guess). If you look at the widgets, they look similar to what runs on the Mac, developed with Dashcode. Dashcode is a new fancy development environment for these kinds of applications. I would guess that there is a version of this development environment also available for the iPhone (also just a guess).

Externally people are currently left with the browser as a runtime platform. That means Javascript on the pages plus often some server developed in any language you like - since the server runs somewhere else. This follows the AJAX model. But I can imagine that a few selected companies are already allowed to work for the new platform using C/Objective C/C++.

Rainer Joswig: But I can imagine that a few selected companies are already allowed to work for the new platform using C/Objective C/C++.

I'd be willing to bet a not insignificant sum of money that this is false. One of the major lessons learned from the PalmOS-on-phones and BREW experiences has been that C-derived languages targeting devices that absolutely must be reliable is an unmitigated disaster. Living within WebKit really is the right thing on the iPhone.

You mean, I don't get Ms. PAC-MAN for the iPhone??? Don't be so cruel. No iPod Quiz? No BEJEWELED???

I don't buy that for a moment. There is TONs of software for phones.
Google for phone and software. ;-)

Symbian has lots of apps without problems. The are sites on the net with lots of software for phones. I had a Sony Ericsson p800 and a Blackberry. Downloading and running software was no problem.

EA develops games for the iPod. There is no official software kit for the iPod, but still a few companies are writing code. There are 16 games for the iPod. I bet that there will be games for the iPhone. Nintendo was mentioned (though that is unlikely, but THAT would be cool). But EA makes perfect sense.

I think it would be totally stupid if Apple would not try to sell software for the phone running on the phone. Plus I think the same platform that runs the iPhone will be running the next iPods. There will be software for those.

Absolutely reliable? But why does Apple write in C-derived languages? I read about lots of crashing software on the iPhone in the reviews. Webkit alone is a huge risk with bugs and holes detected every other week.

It seems to me that this thread is drifting fast from LtU related topics.

He, JavaScript on the phone has closures and eval. ;-)

of being a Lisp.

My former colleague Waldemar Horwat, who is responsible for the JavaScript 2 spec, implemented the semantics of JavaScript in Common Lisp, and is (in)famous for observing that he believed that JavaScript was just a different surface syntax for Lisp. Of course, it's true in the best possible way for it to be true: by implementation.

Apparently there will be a sense in which ECMAScript will be a "different surface syntax" for Standard ML. ;-) Of course, folks familiar with the embedding process being used will understand the limits of such a claim.

if you have worked in sales, you would know that you need to present the arguments differently. A list of high-level buzzwords isn't something that convinces people.

Is there some material that descibes this progress in type inference, modules, type classes, and generic programming and presents advantages that are relevant for end-users (application developers)?

Rainer Joswig: Is the way language tools work and their facilities off-topic here?

I like to hear about them, and there's a lot of latitude around here to discuss different aspects of programming language context, as long as the language part is in the picture. It could go off topic by emphasizing generic tool qualities not specific to any language per se, or by getting far from the original post. Tools and support specific to Lisp — emphasizing how Lisp works — seem on topic, especially when touching the original post's thesis of re-finding Lisp.

Language advocacy is off topic (unless you can be as stylish and funny as Kay Schluehr managed to be earlier :-). It's best to focus on what's technically interesting about a language. Or on general observations about what constraints aren't necessary that sometimes folks assume — like you mentioned compilation shouldn't be necessary for a debug support. We often don't talk about the software stack as a whole much, and that's both interesting and good place to mention upsides to a Lisp approach.

The original post had some style elements and punctuation that might normally get a thread like this tagged as bait of some kind, or just innocent zeal, but it's since gotten respectable enough. The clever, mature tack you take reads well, and I hope folks draw you out a little more. But since I'm pro Lisp, it would read like a staged promotion if I took part much myself.

Actually sorted lists do make a reasonably decent implementation of sets, but somehow I don't think this is what was intended...

XSLT is, in fact, a Turing-complete purely-functional language.

I'm aware of that; and while it is useful for a certain class of applications it doesn't make it nearly as flexible or fun to develop in as Lisp, though.

Of course—the occasions upon which one would want to take advantage of XSLT's Turing-completeness are minimal, and the results sufficiently byzantine to suggest reimplementation using another tool. I should have added a :-) above.

The Java project I was working had all that, too. Java application servers have the feature of deploying code in a running system. The software engineers didn't really get it working though - so there was some productivity advantage unrealized...