Lambda the Ultimate

A very interesting read indeed, although most of the interviews are, at least at first sight, biography focused (How did you get involved in programming? Studies and work experience? etc.).

The talk with Ronald Knuth, which closes the volume, is the first I read and, as an historian, the one I liked the most. I will content myself with quoting rather then commenting.

How one makes himself a programmer (on the IBM 650):

[...] I decided I would write a little program to calculate the prime factors of a number. It was about 100 lines long. I would come at night when nobody else was using the machine, and debug it. And I found more than 100 bugs in my 100-line program. But 2 weeks later I had a program that would find prime factors of any 10-digit number that you dialed into the console switches.
That was how I learned programming — basically taking one program that I made up myself and sitting at a machine over a period of some weeks, and kept getting it to work a little better and a little better.
My second program was converting between binary and decimal. But my third program was a program to play tic-tac-toe and that was what really made me a programmer.(p. 567)

On the “pretty mathematically intense stuff” in The Art of computer programming:

Like in the parts that I’m writing now, I’m starting out with stuff that’s in math journals that is written in jargon that I wouldn’t expect very many programmers to ever learn, and I’m trying to dejargonize it to the point where I can at least understand it. I try to give the key ideas and I try to simplify them the best I can, but then what happens is every five pages of my book is somebody’s career.
In other words, there’s still so much more beyond any five pages of my book that you can make a lifetime’s worth of study, because there’s just that much in computer science. Computer science doesn’t all boil down to a bunch of simple things. If it turned out that computer science was very simple, that all you needed to do was find the right 50 things and then learn them really well, then I would say, “OK, everybody in the world should know those 50 things and know them thoroughly.” (p. 568-569)

And last but not least, as Ehud Lamm already mentioned, the importance of the history of the discipline for its current and future development:

Seibel: Do you feel like programmers and computer scientists are aware enough of the history of our field? It is, after all, a pretty short history.
Knuth: There aren’t too many that are scholars. Even when I started writing my books in 1963, I didn’t think people knew what had happened in 1959. I was reading in American Scientist last week about people who had rediscovered an algorithm that Boyer and Moore had discovered in 1980. It happens all the time that people don’t realize the glorious history that we have. The idea that people knew a thing or two in the ’70s is strange to a lot of young programmers.
It’s inevitable that in such a complicated field that people will be missing stuff. Hopefully with things like Wikipedia, achievements don’t get forgotten the way they were before. But I wish I could also instill in more people the love that I have for reading original sources. Not just knowing that so-and-so gets credit for doing something, but looking back and seeing what that person said in his own words. I think it’s a tremendous way to improve
your own skills.
[...]
For example I spent a good deal of time trying to look at Babylonian manuscripts of how they described algorithms 4,000 years ago, and what did they think about? Did they have while loops and stuff like this? How would they describe it? And to me this was very worthwhile for understanding about how the brain works, but also about how they discovered things.
A couple of years ago I found an old Sanskrit document from the 13th century that was about combinatorial math. Almost nobody the author knew would have had the foggiest idea what he was talking about. But I found a translation of this document and it was speaking to me. I had done similar kinds of thinking when I was beginning in computer programming.(p. 599)

I hope this extensive quoting will appeal the reader to pick a copy of the book. It is full of valuable insights, not only for historians.

I happen to be an alumnus of the same chapter of the same fraternity as Don Knuth, and the composite portrait that contained his photo used to hang in the hall directly across from my door. So with all due respect to Don Knuth, I think his taste in language design sucks. :-)

But I do appreciate that his goals are often a bit different than my own. I must admit that doing algorithmic analysis in Haskell, for example, is a far more challenging a task than C or Assembly.

The Sieve of Eratosthenes is one of my favorite examples. Even the inimitable Rob Pike has explicitly asserted that the unfaithful sieve is the same algorithm with the same asymptotic running time as the traditional sieve. This assertion appears at at 27:30 in his Google Tech Talk about Newsqueak.

Lately, I've also been toying with lucky number sieves a bit, in both C and Haskell. I don't have that much to report so far, but I've learned and relearned two things:

1. What's important to the prime sieve is unimportant to the lucky sieve, and what's unimportant to the prime sieve is critical to the lucky sieve.

2. It's quite easy to write programs in Haskell that appear to implement the same algorithm, but have clearly different asymptotic run times, empirically speaking. In some cases I still don't understand what the difference is.

Of course, if your main interest was algorithms and their analysis, due to the latter observation, Haskell would probably not be among your favourite languages. You'd definitely be better off with Scheme, C, or Assembly.

TeX macros are a scourge that either should have been properly designed or never unleashed upon the world at all. However, literate programming is very cool idea. I'm not sure it is suitable for general programming, but once you start writing executable papers and books, you never go back.

Ironically, the unholy union between TeX and Haskell is probably the best literate programming environment currently available. :-)

This is a really great book.

I found Fran Allen's interview (quoted above by Ehud) provocative. To write sensible programs, I think that programmers need things like predictable performance models, well-defined order of evaluation, consistent behaviour between compilers, and so on. (Just look at the popularity of single-implementation languages.) But these are exactly the features that language and compiler researchers are least inclined to offer us.

I was tickled by exchanges like this:

Seibel: What has changed the most in the way you think about programming now, versus then?

Steele: I guess to me the biggest change is that nowadays you can't possibly know everything that's going on in the computer. There are things that are absolutely out of your control because it's impossible to know everything about the software. Back in the 70s a computer had only four thousand words of memory. It was possible to do a core dump and inspect every word to see if it was what you expected. It was reasonable to read the source listings of the operating system and see how that worked. And I did that - I studied the disk routines and the card-reader routines and wrote variants of my own. I felt as if I understood how the entire IBM 1130 worked. Or at least as much as I cared to know. You just can't do that anymore.

Oh yes you can! I read this while programming in Openfirmware, a self-sufficient sub-megabyte Forth operating system with its own compiler and highly readable source code. I too have studied the (SD) card-reader routines and written my own variants - just last month!

We have it a lot better today than people seem to think :-)

No Andy Gavin, Jason Rubin from Naughtydog Software.

Those two guys are some of the most inspirational programmers I know of. [They designed the predecessor or early prototype of LLVM-like concepts, GOAL, except in many ways it was more robust than LLVM even is today]

Knuth has a challenge to programming language designers. He claims that every time a new language comes out it cleans up what's already understood, and then adds something new and experimental. How about "setting our sights lower" and aiming for stability. "It might be a good idea," he says.

Dylan probably comes pretty close to a language that didn't invent a lot of new stuff, and instead cleaned up everything they had learned. Dave Moon's Programming Language for Old Timers is similar.

Both languages have somewhat "experimental" syntaxes, though.