Lambda the Ultimate

Ain't that the truth. :-) I'm glad to report that I recently read his new book and I enjoyed it immensely. There's nothing to fear in this department.

I wonder if his claims about other languages are strategic? I'm betting that Arc will benefit from the most critical feedback of any language in history. :-)

Wait a minute! Have that many languages been designed by people who don't care about programming?

But yes, I generally agree with the "paradox." When looking at resumes, I always want to see evidence of interest in and knowledge of non-mainstream buzzword topics. I'll always choose to interview someone who both (a) knows a language they don't think they'll get paid to use, and (b) puts it on the resume anyway. So, yes, apart from the one gripe, I totally agree with this article. I hope I didn't imply otherwise.

Graham comes from the Lisp tradition; he knows he comes from the Lisp tradition, and most people who read or hear him know he comes from the Lisp tradition. So when Graham talks about "caring about programming," it's pretty safe to assume that he draws his set of priorities with respect to defining his terms from that community. Given that, I tend to transcribe "care about programming" to "care about abstraction" in my head, and that seems to work pretty well: his example languages all allow the programmer to build abstractions pretty well one way or another, whereas the obvious unmentioned ones (Java, C, C++) do not.

Yes, I know about C++ and template metaprogramming and Boost's preprocessor library and... C++ makes a lot of things possible, in some sense, but in a profoundly hostile way relative to the alternatives.

If I remember correctly Paul Graham says nice things about K&R. I don't know much about Gosling, but Guy Steele's work on Java seems to fit well with Paul Graham's world view. On ll1-discuss GLS said:

And you're right: we were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp. Aren't you happy?

I have a feeling he is thinking about languages like PL/I, Ada and COBOL not about C and ilk.

These persons may individually care about programming, but the committee they are part of only cares about consensus. In general, languages designed by one person are clean and focussed, while languages designed by committee are messy and convoluted.

The obvious counter example is Haskell; but perhaps Haskell's was a different sort of committee to Java's?

Certainly I don't think that collaboration necessarily dilutes the commitment and "vision" of language designers. Rather like Eric Raymond, Graham tends to lean rather hard on an individualistic mythos - the lone hacker, expressing his genius - which I find seductive and questionable in about equal measure. It's clear that Graham approves of renegades, but that doesn't preclude the possibility of successful partnerships - Butch and Sundance, say.

The obvious counter example is Haskell; but perhaps Haskell's was a different sort of committee to Java's?

The difference being that designing the ideal theoretical language is more suited for design by committee than practical languages? Certainly when for practical languages a design requirement might be that 80% of all programmers employed by the BigCos have to understand it.

Certainly when for practical languages a design requirement might be that 80% of all programmers employed by the BigCos have to understand it.

I don't think that the languages used by these programmers are simple to understand, or that they understand them. For example trying to come up with a correct copy constructor in C++ is really tricky (or so C++ programmers tell me), after all we have to deal with memory management, exceptions, identity, equality, object slicing, etc., and these issues aren't simple. Let's take a look at Java: we have issues with synchronized keyword, proper implementation of equals and hashCode, many library warts, final, finally, exceptions, the subtle issues created by the language and VM specs, etc. I can point to many articles, papers, anedocts, etc. on these "language for the masses" that make them don't be simple. IME as a Java programmer usually other Java programmers don't understand these issues and just follow expert advice as commandments.


I think the problem isn't with understanding but with the approach used to teach the language. My ideas are in this old LtU thread.

For Java and C, there's a lot you need to know. But most people can learn (memorize) all these facts.

Haskell otoh requires you to grasp some high level concepts. There is actually not that much to learn. The trouble is to understand it all.

In The Psychology of Computer Programming, Weinberg reports a study that showed that the more special cases and rules a programming language has, the LESS of those cases programmers actually use. You certainly see that in the Java community, where Java bloggers frequently post about discovering language features that are well described in the language spec. It will be interesting to see whether this effect is reproduced as people migrate to Java 1.5, sorry 5.0. The number of wierd interactions between autoboxing, coercion, inheritance and overloading might result in most Java code being written to a very restricted subset of the language.

Java and C# seem to be evolving by the designers adding features to simplify the job of programmers. If Weinberg is correct, this approach makes the job of programmers much harder. Perhaps a better approach would be to refactor the language: add features by designing a simpler, more expressive core language that can express existing and new language features as standard libraries.

The obvious counter example is Haskell; but perhaps Haskell's was a different sort of committee to Java's?

Also, wasn't Haskell originally quite closely based on Miranda, which was designed by one individual?

It's interesting that Haskell has been described as "the most Pythonic of all languages that have nothing in common with Python" (list comprehensions and indentation-based syntax being among the more obvious reasons why). The general flow of ideas I think is somewhat in the other direction.

I get the impression that using it would feel like reading a novel written by a literary critic.

First, let me say that, like many people, I enjoy your essays and find them well-written and thought-provoking.

In line with most of your writing, the quoted sentence is quite rhetorically compelling: you evoke almost poetically the wisdom of the common man in the face of a self-absorbed intellectual elite. Many of us know that feeling; it is one of cornerstone tropes of Western culture, and anyone who has seen more than a few Hollywood movies will certainly recognize it.

The catch is that you are using this feeling to defend a self-absorbed intellectual elite: the mythical hacker who is a smarter and better programmer than the plebs who use other languages, say Java, C, C++, etc.

So how does the hacker get to be "just right" on the scale between mouth-breather and pretentious twit? If he can use the "overly-academic" card to neutralize an elite he fears may be smarter than him (Haskell users), why can't a Java pleb do him in with the same ploy?

There are various kinds of elites. The kind of hacker I wrote about, though smart, is not (usually) a professor.

So there are (at least) two continuums: one related to writing research papers, and one related to writing software. And "just right", at least when it comes to solving demanding problems, is as far along the latter as you can get.

There are various kinds of elites. The kind of hacker I wrote about, though smart, is not (usually) a professor.

I'm not sure the two are as far apart as you suggest.

A key attribute of both the academic and the hacker (as you have presented him) is that both tend to choose their projects based on the inherent interest of the problem rather than the demands of the marketplace.

Haskell is almost exclusively used to produce software to solve new and interesting computational problems. Perl and Python are widely used to solve already solved problems (e.g. yet another e-commerce app), and sometimes very badly.

Which is more likely to have a concentrated group of hard-core problem solvers using it?

Haskell is almost exclusively used to produce software to solve new and interesting computational problems. Perl and Python are widely used to solve already solved problems...

Which is more likely to have a concentrated group of hard-core problem solvers using it?

Perl and Python are used by quite a few academics in computational biology, because they are the quickest way to create the software supporting their main research

Who's to say one could not do the job as well, or better, in Haskell?

I'm sure that Perl and Python have more libraries available for such tasks, but surely there was a time, back in the jurassic period perhaps, when Perl and Python had no such libraries, and one would have said the same about them and C as you now say about Haskell and them.

I don't see that there is anything intrinsic about Perl or Python, as programming languages, that makes them better suited for that domain than Haskell. (Not that I know anything about that domain, of course. :)

Who's to say one could not do the job as well, or better, in Haskell?

I'm sure that Perl and Python have more libraries available for such tasks, but surely there was a time, back in the jurassic period perhaps, when Perl and Python had no such libraries

As for intrinsic advantage, I think Haskell has at least one obvious handicap -- monadic I/O. Every minute I spend trying to understand how to program with it is a minute I am solving a problem I don't care about.

Perhaps the worst problem for a newcomer is that it makes printf-style debugging, a sort of lowest common denominator for program understanding, impossible. Say I write a beautiful quicksort in Haskell:

qsort [] = []
qsort (x:xs) = [y | y <- xs, y < x] ++ 
                  [x] ++ [y | y <- xs, y >= x]

A similar situation occurs with profiling, another common task that is, if not harder, at least completely different in Haskell. Perl and Python are easy to pick up because there is very little startup cost for solving simple problems. I'm not sure about eventual suitability for the domain, but the large conceptual overhead for solving simple problems makes Haskell IMHO a poor choice for someone whose primary interest is the application domain, not the software itself. Even Lisp, which lacks this conceptual overhead, has famously suffered even for its syntactic overhead; this doesn't make me optimistic about Haskell.

perhaps Perl's biggest advantage is in having libraries for the most obscure tasks.

Yes, that is my point. It's biggest advantage has nothing to do with the language itself, but rather the social/historical context. And that context could as easily have been/be established for Haskell, if people (had) want(ed) it to be that way.

(I'm sorry but I really need to get this off my chest, again:) Scripting languages thrive on hype because they have no other advantage; their advantage is the hype, and the popularity which accompanies it. Sure, Perl may be an improvement on sh (frankly, I am not so sure :) but it is hard to find a strawer man.

(Man, what the hell kind of marketing campaign is that? "Perl: Better than sh!"??? How about, "Bush: Better than Osama!" or "Spam: Better than starving!" Then again, spam is considered a delicacy in certain countries...)

Consider Scheme: Scheme has been—for, what, fifteen years now?—superior to extant scripting languages in nearly every way as a programming language, and continues to be so, even though it has changed only a little, but no one wants to use it, simply because it isn't popular. Scheme was a great language, even when Perl was still a glimmer in Larry Wall's eye. But, no, we can't use Scheme for scripting tasks, right, because it was already invented, and moreover by academics, and it has far too many parentheses. Instead, we need a new language like Perl, which is based on sound principles like DWIM, TMTOWTDI and Mr. Wall's personal theory of natural linguistics. Brilliant.

Most programmers don't want better tools as much as they want self-validation and a veneer of novelty. Inasmuch as library size is a measure of popularity, it doesn't reflect on the usefulness of the language itself.

Perhaps the worst problem for a newcomer is that it makes printf-style debugging, a sort of lowest common denominator for program understanding, impossible.

Here is the standard retort: It is a problem for newcomers who come from procedural languages, because they were miseducated, but it ought not to be a problem for non-programmers, which is who we expect domain experts to be. Haskell's computational model is far closer to the mathematics that we learn in high school than that of procedural languages, and it lets you reuse those principles. In arithmetic, there is no printf. Nor is there an assignment statement, any form of imperative loop, or array, malloc, free, pointers, setjmp or a hundred other of the things one needs to learn to use a procedural language. In arithmetic, referential transparency holds and a variable is just a name for a value and it doesn't matter what order you evaluate an expression in.

Anyway, we all know the problems that arise from relying on execution order, especially in the presence of state. It bites programmers every single day, but in procedural languages this problem is only dealt with haphazardly, one a case by case basis, or by OO-style objects, which are really only a convention for encapsulation of state. The need in procedural languages to always deal with state is a symptom of a poor separation of concerns.

But, I don't want to get into this debate...

I don't want to reopen the age-old debate, either (I'll try to steer clear of "human factors";), but just a couple of points:

Man, what the hell kind of marketing campaign is that? "Perl: Better than sh!"???

Actually, a pretty good one, with slightly different wording -- "Are you currently using X to perform some tasks? Here is Y, and here is how it can better perform them." It is addressed to someone who, like most people, has already found a tolerable way to perform various tasks, and who is therefore open to suggestions for improvement, but skeptical of suggestions that he completely discard that solution and try something radically different. This isn't "hype". Instead it is catering ("slavishly", I hear you saying) to users' expressed needs.

Most programmers don't want better tools as much as they want self-validation and a veneer of novelty

And I assume category theorists are above this.

Haskell's computational model is far closer to the mathematics that we learn in high school than that of procedural languages, and it lets you reuse those principles.

Agreed, but a lot of programming addresses problems that are not mathematically formalized, though they may be formalizable. If the problem description is informal, like a sequence of steps that one typically performs by hand, but wishes to automate, high-school math may not be the most "natural" (by which I mean most likely to occur first to someone with a typical non-programming education) way to express it. I know this is close to the dreaded "HF", and I don't have empirical studies to back it up, but it seems reasonable. When you decide to script something you have been doing by hand, do you (a) formalize it first, or do you (b) go brashly forth and write out the ugly, temporal, stateful steps, then debug the result? (note: if you answer "a", then maybe category theorists really are allowed to condescend to us mere humans)

When you decide to script something you have been doing by hand, do you (a) formalize it first, or do you (b) go brashly forth and write out the ugly, temporal, stateful steps, then debug the result?

I do not see (a) and (b) as either exclusive or exhaustive. Formalization can be ugly, or temporal, or stateful, or have any combination of these properties.
Also, I don't agree that sequential description is more natural (you are right, it's HF after all). When cooking, I use constraints and futures, and not sequences :) Then again, that can be unique to my cuisine...

Scripting languages thrive on hype because they have no other advantage; their advantage is the hype, and the popularity which accompanies it.

...

Most programmers don't want better tools as much as they want self-validation and a veneer of novelty.

I think that one point is missed here: the popularity provides support for peers who operate roughly at each other's level, e.g. providing explanations in terms the recipient can understand, even if those explanations would make an academic or a competent professional developer grind his teeth.

A language like Haskell acts as a selection filter for such communities: either sufficient people of a certain level just avoid the language, or features that are considered barriers would have to be modified to make the language more comfortable for the community. At the very least, in Haskell's case, you'd end up with a strict, mutating language instead of a lazy, pure one, not to mention the slew of imperative control structures that would be added. ;)

An argument one sees occasionally, e.g. on comp.lang.lisp, is that it's not a good idea to pitch language features to newbies, because soon they're not newbies and don't need them, and it's really the more advanced, general, powerful features that are important and help you do the hard stuff more easily. That all makes perfect sense, in the LFSP context, but if popularity is the goal, then this argument doesn't take the previous paragraph's point into account.

A significant proportion of the userbase of some of these languages are perma-newbies, who either don't write a lot of code or just aren't very good at it; plus, there's a constant stream of fresh newbies. If a language doesn't pitch features to newbies, that decision selects out a huge potential market, not just of the newbies themselves but the entire community that goes along with them - people who enjoy helping newbies, people who don't want to feel as though everyone else using the language is smarter than them, etc.

So, the lesson for language designers who want their languages to be popular is, concentrate on the marketing and feel-good messages - DWIM, TMTOWTDI, principle of least surprise - that appeal to newbies, because newbies are the plankton of the popular language ecosystem.

because newbies are the plankton of the popular language ecosystem

Setting aside the question of catering to plankton as a language design strategy ;-), I think this is beside the point in the overall context of the thread.

Paul Graham's original contention was that certain languages appeal to the best programmers. If they are really just following the lamest programmers, because that is where the libraries and support community are, then we have shown nothing that distinguishes good from bad programmers. (It would be a bit like saying good programmers all share the characteristic of breathing.)

If language choice is an indicator of a good programmer at all (and I think most of us here would dispute this in one way or another), we would have to show that such a "filter" language possessed some feature that only a good programmer could use effectively, or that had appeal to a good programmer but not to a bad one.

Unfortunately, the only obvious candidate for such a feature has already been discussed to death in another thread... ;-)

I think this is beside the point in the overall context of the thread.

Paul Graham's original contention was that certain languages appeal to the best programmers. If they are really just following the lamest programmers, because that is where the libraries and support community are,

Let me first restate my conjecture more generally, since I think it loses something of what I intended to convey when summarized as "following the lamest programmers" (although the way I stated it, that certainly seems to follow).

The general principle I was getting at, in response to Frank's post, is that barriers to adoption have a compounding effect on the growth of communities, and the relative unpopularity of some languages can be at least partly explained in those terms.

Based on this blander restatement, please now imagine my newbie-centric post rewritten in more politically correct terms. ;)

then we have shown nothing that distinguishes good from bad programmers.

Based on this blander restatement, please now imagine my newbie-centric post rewritten in more politically correct terms

Political correctness really isn't my issue. ;-)

I think the point Paul mentioned about Google looking for Python knowledge when hiring Java programmers is more significant for the greater breadth of knowledge it implies

You may think that, but that isn't the position I saw in the essays.

On the contrary, the argument seems to be:

1) There is some minority of programmers that are just way better than others.
2) These programmers gravitate to language X
3) Therefore, if you want to hire the good programmers, look for skill in language X.

As your newbie argument shows, the statement that "good programmers like X" is in no way equivalent to "all (or even most) X users are good programmers", which is what you would need to make the argument work.

If we accept premise 1, one winning strategy if you want your language to be popular is to cater to newbies and/or lamers, since there are more of them to draw from.

This still does nothing to explain what language the superior programmers prefer, if such a language exists. (Or even if these superior programmers exist. ;-) )

Here is the standard retort: It is a problem for newcomers who come from procedural languages, because they were miseducated, but it ought not to be a problem for non-programmers, which is who we expect domain experts to be.

So Haskell is only for domain experts?

Haskell's computational model is far closer to the mathematics that we learn in high school than that of procedural languages, and it lets you reuse those principles.

But a procedural model is closer to what I do in real life. We do things sequentially, and sometimes (when using a scripting language) I am programming the computer to do what I would do sequentially and by hand.

In arithmetic, there is no printf. Nor is there an assignment statement, any form of imperative loop, or array, malloc, free, pointers, setjmp or a hundred other of the things one needs to learn to use a procedural language. In arithmetic, referential transparency holds and a variable is just a name for a value and it doesn't matter what order you evaluate an expression in.

That's fine, but there's a tool problem in Haskell without printf. There is no debugger. Oh yes, there's Hood, and Buddha, and hat, but none of them support the whole range of extensions GHC supports. You could argue that we shouldn't use those extensions, but then you have to defend a Haskell without them, which gets much trickier.

Now, I would be happy with a good Haskell debugger. Even so, it's not very inviting to ask people to learn a language when they have to learn an additional tool (as opposed to printf) before being able to debug.

Anyway, we all know the problems that arise from relying on general recursion, especially in the absence of printf. It bites programmers every single day, but in purely functional languages this problem is only dealt with haphazardly, on a case by case basis. The need for purely functional languages to always deal with termination drops serious mathematical quandries in the lap of Suzy Q. Coder

Yes, I'm reaching. No, I don't actually think that's true, but I do think recursion and state are both valuable features. I think I could have taken that paragraph and substituded monads, or laziness, or just about anything; my point was that it contains claims, but no warrants or data. So, I would ask for some explanation of why the benefits of the simplicity of state do not outweight the disadvantages, but:

But, I don't want to get into this debate...

Another time, then.

But a procedural model is closer to what I do in real life. We do things sequentially, and sometimes (when using a scripting language) I am programming the computer to do what I would do sequentially and by hand.

• The point of programming is not to replicate how you have to do things in real life; the point of programming is to make things easier to do. In "real life", to get to my office, I take a certain path to work every day. In a program, I would rather not specify a particular path, but just specify the endpoints. In some cases, the particular path is relevant, and I can fix it if I please; but in most it is not, and being able to postpone that choice increases separation of concerns and decreases cognitive burdens.
• Sequential processes are a subset of parallel ones.
• Wrong, "real life" (whatever that is) is not sequential—it's massively parallel. Your life might be sequential, but there are other people in this world, and other agents, processes, particles, all interacting, and there is no linear order on them.
• Almost every form of I/O is non-sequential; as soon as you talk about networks, other processes and threads, even terminal I/O (which only presents parallel processes as sequential ones, by interleaving their output), you see parallelism. If you've ever tried programming a robot, or writing some kind of VR environment (say a first-person shooter), too, which certainly comes about as close to "modeling real life" as we can get, there is lots of parallelism, and trying to express it sequentially is hard. Ultimately, the reason is that there are many ways to sequentialize a parallel process, and all of them lose information and decrease locality.
• Well, let's look at how more knowledgeable people than us have tried to characterize this "real world" you speak of. We have physics, chemistry, mechanics, engineering. All of these are based on what we regard as standard mathematics, which is in form much closer to the formalism of Haskell than any of: Perl, Python, C, C++, Java, Pascal or even ML. If you are really serious about talking about the real world, it would behoove you to reuse that theory using a language which makes it as easy as possible to do so.

For the rest:

So Haskell is only for domain experts?

That's fine, but there's a tool problem in Haskell without printf. There is no debugger. Oh yes, there's Hood, and Buddha, and hat, but none of them support the whole range of extensions GHC supports. You could argue that we shouldn't use those extensions, but then you have to defend a Haskell without them, which gets much trickier.

Yes, I'm reaching. No, I don't actually think that's true, but I do think recursion and state are both valuable features. I think I could have taken that paragraph and substituded monads, or laziness, or just about anything

Haskell has state, (proper tail) recursion, referential transparency, monads and laziness. Pick almost any language X: it has a proper subset of these features. (BTW, monads are not really a feature; they are just part of the library, with some syntax support.)

But, as it happens, and since you bring it up, I think general recursion should be controlled, same as any other side effect, and similarly I don't like laziness-by-default.

my point was that it contains claims, but no warrants or data. So, I would ask for some explanation of why the benefits of the simplicity of state do not outweight the disadvantages

This debate is so old that the supporting arguments are not worth repeating. Just look at Wadler's paper, How to declare an imperative, for example.

I for one have tried using Scheme (Scsh) for my shell scripting, but to my surprise it turned out that I prefer using bourne shell.

There's a huge difference between theory and practice. I think it's your loss not to appreciate this.

There's a huge difference between theory and practice. I think it's your loss not to appreciate this.

If the theory doesn't fit the practice, you're using the wrong theory, or you're using the right theory wrongly. I think this is what you don't appreciate.

Insisting, like you, as a maxim on some ubiquitous gap between theory and practice isn't constructive. In the absence of theory there is no practice, only happy coincidence, from which grows superstition and mysticism.

I might be wrong in suggesting that Scheme is well-suited for scripting; if so, and I discover the reasons for it, I will improve or revise my theories. But someone who follows your philosophy will never even look for the reasons, putting it all down to the gap between theory and practice.

FWIW, I'm an old Schemer from Indiana (literally, born and raised there) and I prefer bash to scsh also. I take that to mean that familiarity counts, rather than that Scheme is bad for scripting or even that scsh is a bad implementation of scripting-in-Scheme. Besides, I also have DrScheme as a counterexample: it makes a very nice programming language implementation/web services/debugging/etc. scripting environment using Scheme.

So maybe it would be worth trying some other functional shells and seeing whether the issue really is about the "Scheme nature" or about accidents of the language that might be more about personal experience and taste than anything else. I found Es: A shell with higher-order functions, fsh—a functional UNIX command interpreter, and of course scsh itself. es looks pretty interesting to me.

Luke: There's a huge difference between theory and practice. I think it's your loss not to appreciate this.

As usual, I'll just ditto Frank's response to this. The assertion suggests replacing or refining the theory; positing that there's necessarily a huge difference between theory and practice is anti-scientific nonsense.

I've found that it's an interesting experience to mistake SICP for a book on software engineering, for example.

Also see:

Arjen van Weelden, Rinus Plasmeijer. A Functional Shell that Dynamically Combines Compiled Code. Proceedings 15th International Workshop on the Implementation of Functional Languages, IFL 2003, Selected Papers, Edinburgh, Scotland, September 8-10, 2003.

which is from the Concurrent Clean group.

inasmuch as function composition can work in a similar way to Unix pipes. Consider the following rather dull example, which prints a list of all the alphabet characters in an input string:

writeLine x = do putStr x; putStr "\n"
makeSentence x = x : " is a letter in the alphabet"

conv = mapM writeLine . map makeSentence . filter isAlpha

main = do chars <- getLine; conv chars

The function conv is defined by stringing a few other functions together: filter to select the inputs we want, map to transform them, and mapM to perform an action on each of them. It seems to me that quite a lot of "scripting language" programming - programming to automate some batch task or other - runs along similar lines.

main =
	do	wordList <- loopWhile continueq getNext []
		printf (show wordList)

loopWhile cond next init =
	let loop = loopWhile cond next in
	do	new <- next init
		continue <- cond
		if not continue then return new else loop new

continueq =
	do	continue <- input "Continue?"
		return (continue=="y")

getNext wordList =
	do	nextWord <- input "Enter a word:"
		let newWordList = nextWord : wordList
		printf ((show (length newWordList)) ++ " words collected")
		return newWordList

input str = do putStr str; putStr " "; getLine
printf val = do putStr val; putStr "\n"

Observe also the number of type annotations in the code (in actual fact, I put them in while I was writing it and then took them out again once I'd finished).

Admittedly this is a very trivial program. But it gets you over the main hurdle for writing interactive programs: it accepts inputs, processes them and writes out results. Many more complex programs can be built around the same (or a similar) skeleton.

The problem with monadic I/O w.r.t. debugging isn't so much writing programs at all, but modifying programs to do I/O. It's a sort of tar baby, in that you find yourself warping your entire program to add I/O to a leaf routine that didn't have it before, because the I/O changes the function's type. For example, say you have a complex comparison routine that may contain a bug, and the bug shows up when you sort a large pile of data. Before, we have:

Lisp:
    (sort things #'myfunc)
Haskell:
    sort myfunc things

After:

Lisp:
(sort things
      (lambda (a b)
        (let ((res (myfunc a b)))
          (if (not (member res '(t nil)))
              (format t "(cmp ~s ~s) ==> ~s~%"
                      a b res))
          res)))
Haskell:
    -- ???

You've just changed the type of your comparison function in Haskell, so you need to change sort, all its callers, etc., and that's just ugly.

There's at least one person using Haskell for bioinformatics.

But bioinformatics is a very bad choice of example in a discussion of "true hacking" because most writers of bioinformatics programs are motivated by the biology, not the programming (in fact they are often biologists) and do not care how ugly the code is as long as it gives them some results, so they gravitate towards whatever is popular (e.g., perl), as it is only a tool to them. Thus IMO bioinformatics, on the whole, falls into the ugly web app category, really.

This is not to belittle bioinformatics (or web apps, for that matter) but I am getting tired of hearing programmers from outside the field reflexively go "Bionformatics? Oooh, that must be so exciting..." Well, not necessarily.

From the paper of Amanda Clare:

The main disadvantage that was faced was dealing with unwanted laziness.
In a data mining environment, where all data needs to be read and counted,
and all calculations will be used, laziness provides few advantages, and usually
takes up huge amounts of heap space while delaying all computation until the
last minute. At every step of the coding process it was found that code was easy
to get right, but then difficult to get working in practice without running out of
memory.

So, in practice, it might be beneficial for a lazy PL to not only provide forcing primitives, but probably also support regions of default evaluation strategy switched to eager (though I do not know how to limit these regions).

In any case, didn't SPJ himself concede recently that laziness is overrated and that he was no longer sure it should be given such pride of place in Haskell?

If we compare Haskell and Clean we have two different strategies of dealing with laziness. In Clean you can add strictness annotations to function's arguments while in Haskell you use strict datatypes or sequencing operations (e.g. seq, deepSeq). AFAICT Clean programs are much simpler to optimize for strictness.

Who's to say one could not do the job as well, or better, in Haskell?

I think Haskell's lack of a standard, simple to use, cross-platform implementation hurts quite a bit here. As an aside, I'm always surprised at how many caveats are involved in this page. (And as another aside, I'd love to hear from anyone doing actual software development with Haskell under Windows.)

From that page:

The Glasgow Haskell compiler is a full implementation of Haskell, also offering many language extensions... It is available for a few Unix platforms (including Linux, Solaris, *BSD, and MacOS-X) and also for Windows.

Perhaps you're referring to IDE support. I was able to find hIDE, KDevelop, Vital, Haskell Support for Eclipse, Haskell support for JCreator, and of course Haskell modes for the popular extensible text editors, generally complete with interaction with either Hugs or GHCi.

Whatever Haskell's limitations, full implementation and platform availability doesn't seem to be among them.

Again, not really a discussion I want to pursue but, you cannot fairly compare the compiled implementations of Haskell with the interpreted implementations of scripting languages, which are far easier to port. Better to compare with Hugs, which, as the page says, indeed runs on just about any machine.

A tool (i.e., a programming language) can be useful both for writing papers and for writing software (emacs is :-). The fact that a language inspires many researchers shouldn't be held against it. You should examine the language and see how and when it can be useful.

Moreover, if the papers we are talking about are concerned with software development activities, than hopefully these papers -- if they are any good -- tell us something about real life programming.

I think the examples I gave of Haskell DSL design are a case in point. If one thinks DSLs can be useful in practice, one should look for good ways to design and build DSLs. I think the Haskell examples are inspiring.

Most important of all, I think curiosity is the key trait you are looking for. And curious folks learn esoteric languages. Why? Because they are curious to know what it feels like to program in them, of course...

For instance, Haskell is a very nice language to write papers *in* - it comes far closer than python to "executable pseudocode".

To me, QuickCheck (yay, extra credit! :) ) is the best example of an extremely cool and useful Haskell "hack" (in the best possible sense of the word) out there. The source code is sufficiently small to understand in one sitting, but it is still sufficiently clever, specially when it comes to generating random functions of arbitrary types. Everytime I look at QuickCheck, I think: "What a cool idea". Most of the advanced stuff in Hudak's "Haskell School of Expression" also left me with similar, if not as strong, feelings.

Functional Unparsing and Sorting Morphisms are two other papers that while not directly related to Haskell, would fall into this category, because of the interplay between research (in type theory, etc) and applications they show. The "Functional Unparsing" paper, specially, gives a quite cool way to alleviate the need for dependent types in some situations.

I was excited about QuickCheck when John Hughes presented it at last year's Erlang conference. Unfortunately I couldn't figure out how to apply it to the problem I had in mind, which was a small in-memory database. I had trouble formulating properties that would be easy to check -- they always seemed to require the checker to be a reimplementation of the checkee in order to know the right answer.

I've experienced this too.

I agree with you that it is sometimes hard to think of what properties you want to test, but I think this is something we should be doing anyways, as part of a "real" software engineering discipline: since doing proofs is hard, and arguably boring, we are helped by a systematic way of trying to find negative proofs.

With properties, the intent of the program becomes clearer: properties are excellent documentation. It's like having blueprints for a mechanical part, a formal description of the expected behavior of the part in terms of physical quantities (ie, a formal description), _and_ a way of testing whether those expectations hold (modulo test quality issues).

To me, this is the really cool part about QuickCheck: it brings "coding by specification" so much closer to practice, which by itself is of tremendous value. It feels closer to doing algebraic specifications than anything else I've used.

QuickCheck can also be used as a smart assert: instead of trying to prove stronger properties of the program, you can use it for sanity checks of data structures, etc. that don't incur in run-time overhead (it may feel weird to talk about performance in Haskell, but some of the checks I wrote are pretty slow :).

My most "advanced" quickcheck uses were probably when I was hacking some geometrical data structures in Haskell, and I needed all sorts of sanity checks. For example: when implementing a triangulator for polygons, I checked whether the sum of the areas was equal to the area of the polygon, (yes, there are floating point issues), whether all triangles are disjoint, and whether they cover the whole polygon. My BSP tree tester would check the code to generate the polygon of a leaf node, so that it didn't overlap with any other leaf polygon in the tree; that the sum the the areas of two children have to be equal to the area of the parent; that a parent polygon contains both children; that all of the polygons were convex, etc.

Another trick that may help to avoid the reimplementation issues is to have the tests use different parts of the same program. For example, you have two procedures, A and B, and then state the properties of A with regards to B and the properties of B with regards to A. If the checks fail, you won't know which procedure is wrong, but usually the counterexample QuickCheck gives helps.

I have very little experience implementing databases, but I'd try to figure out the properties from relational algebra (was it a relational DB?). For example:

- the size of 'A join B' is at most |A||B|;
- if you do a 'A join B' and then select back all the keys and columns from A, you should get exactly A;
- A union A = A;
- for every 'A union B', a select of a single item from either A or B should return exactly one row;

etc. I'm probably showing my DB ineptitude, but I hope you get the idea: although these don't give you everything you need to describe a relational DB, they are all valid properties that the DB operations should hold. The properties don't have to be sufficient to describe the program algebrically. If you can pull it off, great, but this doesn't stop you from using QuickCheck.

QuickCheck is not quite there when it comes to dealing with state, though, and that is probably the largest setback right now. (shameless plug: This is the reason my Scheme port of QuickCheck is little more than an exercise in getting typeclasses in Scheme: pure code can be easily tested, but stateful code is a whole different story which I honestly don't know how to deal with. Obviously the same problem would happen if you want to test a program inside the IO monad, but since programmers are discouraged to do anything not strictly necessary in it, this is not as big of a deal)

I'm not sure this little database has such nice properties as relational algebra though. It's designed to exactly match the semantics of the command-line configuration interface on an existing line of ethernet switches.

I think DARCS, a revision control system implemented in Haskell, qualifies as a cool hack (and gets me my extra credit).

Darcs, Unison, and the ICFP contest winners are great ambassadors for Haskell and ML. Because of them (and The Evolution of a Haskell Programmer) I actually do think that those languages are important and I do write a small program in them every so often in the hope of enlightenment. (So far it just feels like writing Erlang with a slightly different syntax. I suspect I'm not writing the right sort of programs.)

I'm also pleased that although Darcs does work very well it also has bugs (see the mailing list), and they're exactly the sort of bugs that I expect real programs to have. Somehow it's reassuring to see Haskell connect with reality in this way.

Saying that, here's my list:

Many examples of DSL design. On the top of the cool scale I'd put Pan/Fran and Haskore. So easy, yet so much fun.

I am also a fan of the origami school, and would recommend the maximum segment sum paper (from O(n^3) to O(n) by program transformation) and the radix sort paper. Both by Gibbons.

"The Fun of Programming" is a good choice, if you are looking for a book on Haskell hacks.

The problem is that something that looks superficially similar to what you would do in another language means something completely different in Haskell. When you run into the difference in a case like this, the solution to what appears to be a "local" problem is to either globally and significantly change the way you reason about and write your programs, or thread a "world" argument through the whole thing for a single printf. Unless there is an automatic transformation like you describe, this just looks ugly.

I'm sure you already know this, but for the record, Haskell does provide the "trace" function, which is not in the IO monad, and is intended for just the same sort of debugging that printf is used for.

I was about to be somewhat ashamed -- I'm not a Haskell programmer, and I have never heard of trace. But are you sure it's standard?

Prelude> :t trace

Variable not in scope: `trace'
Prelude> Leaving GHCi
rh220-50:src% ghci --version
The Glorious Glasgow Haskell Compilation System, version 6.0.1

Not sure if it's standard, but it's well supported. It's in a different module, though, so you need to import it.

Prelude> :module + Debug.Trace

Prelude Debug.Trace> :t trace

trace :: forall a. String -> a -> a

I'm pretty sure there must be better ways of locating a bug in a comparison function than relying on a sort algorithm to come up with the particular test case for which it fails. I can see that it would be frustrating, in the event that you had detected a failure that way, if you couldn't then find out what particular value the function had failed for.

I mentioned the possibility of "out-of-band" side-effects in another discussion a while back: side effects that take place in what from the program's point of view is a parallel universe (a separate value of "world"). The programmer, as overseer of all possible worlds, gets to observe these side-effects, but referential transparency is preserved for the program in its own world (since leakage between dimensions is one-way). Maybe you could do this in Haskell using various shady operators that pretended to be id, but actually funnelled information into the parallel world. Of course, you'd have to teach compilers not to optimise the fake ids away (in debug mode, at least). The fact that I know very little about compilers is probably the reason why I don't instantly see why this is a stupid idea.

See http://www-users.cs.york.ac.uk/~olaf/PUBLICATIONS/transHaskellTrace.pdf for a paper on Hat, which performs more or less the kind of program transformation I was describing.

(edited) - actually slightly different, as it uses side-effecting FFI calls outside of the I/O monad rather than threading the state of a debugger through the program.