Coders at Work

Peter Seibel's book Coders at Work is apparently available for purchase, so this is a good time to say a few words about it here.

The book consists of interviews with several illustrious programmers about their personal histories, programming style, likes and dislikes and so on. Among the interviewees are several who are well known in programming language circles and are mentioned regularly on LtU, for example Brendan Eich, Joe Armstrong, Simon Peyton Jones, Peter Norvig, Guy Steele, Dan Ingalls, and Ken Thompson. The interviews go into more detail and depth than I dared hoped for or expected, though as is inevitable you end up annoyed that a question you really wanted answered did not come up.

I am sure LtU readers will want to read these interviews for themselves and revel in the technical miscellanea (I read them on a long flight from the US to Australia...), so I am not going to post a detailed review with spoilers. It would be more fun to hear the questions you guys would have asked had you conducted the interviews (and to know which answers you want to quibble with!) So in lieu of a long and tedious review, here are a few LtU-worthy things that caught my attention in a couple of the interviews that I think will interest LtU members.

For some reason I started by jumping to the interview with Dan Ingalls. It turned out to contain many nice morsels to chew on. Dan emphasizes an attribute that might be called programmability all the way down (he is a Smalltalk guy, so that's not such a surprise, I guess): "You should be able, in a computing environment, to zero in on music and musical synthesis and sound and just understand how the whole thing works. It should be accessible. The same thing with graphics." Not surprisingly, Ingalls admits to having an exploratory programing style (in contrast to Knuth who wrote TeX in a notebook...), which probably influenced the types of language he found himself working on. This seems to be the case for several other interviewees as well. Interestingly, Ingalls recalls being influenced by APL. The interactive environment was part of it, but significantly he also mentions the influence on him of the fact that it is expression oriented and not statement oriented like Fortran.

And oh, Ingalls also opines on the age old question: should programmer education begin with assembly. His answer: No. As you would expect, other interviewees probably feel differently.

The interview with Knuth is also very interesting, as you might expect. Here are a few of the things I picked up on in his interview. From his description, it would seem that Knuth was doing his own style of test driven development, though for some reason this angle is not elaborated on in the interview. While Ingalls ponders how to expose kids (and adults) to programming, and Norvig reflects on the failures of end-user programming, Knuth recites his observation that 2% of people are natural born programmers (my words) since they "really resonate with the machine." Perhaps surprisingly Knuth is here concerned with being attuned to the way the machine "really works," not to algorithmic thinking in a general sense. Perhaps, I wonder, what really unites programmers is a compulsion to program: Knuth admits to having the need to program even before having breakfast.

Knuth has a challenge to programming language designers. He claims that every time a new language comes out it cleans up what's already understood, and then adds something new and experimental. How about "setting our sights lower" and aiming for stability. "It might be a good idea," he says. I am pretty sure some here will argue that we have too much lowering of expectations already...

What really resonated with me, and with the LtU ethos, was Knuth lament about people not going back to the original papers and source materials. He puts it simply and powerfully: "I wish I could... instill in more people the love that I have for reading original sources... I was unable to pass that on to any of my students." LtU always had a history department, and going back to historical papers is something I personally love doing. Maybe Knuth should guest blog on LtU... Really! He talks about having collections of source code, compilers in particular - we want to know more!

Finally, let me note the nice contrasts you find among the interviewees. Naturally, these manifest themselves in differing opinions about C. While Knuth sees the C pointer as one of the great advances in computer science, Fran Allen argues that "C has destroyed our ability to advance the state of the art in automatic optimization, automatic parallelization, automatic mapping of a high-level language to the machine."

And if that's not a call to action to all you programming language fanatics, what is?

Disclosure: I was asked to provide a blurb for the cover of the book, and so read the interviews before the book was published. Other than that I had no involvement with the creation of the book, and I have no stake in its success.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

A very interesting read

A very interesting read indeed, although most of the interviews are, at least at first sight, biography focused (How did you get involved in programming? Studies and work experience? etc.).

The talk with Ronald Knuth, which closes the volume, is the first I read and, as an historian, the one I liked the most. I will content myself with quoting rather then commenting.

How one makes himself a programmer (on the IBM 650):

[...] I decided I would write a little program to calculate the prime factors of a number. It was about 100 lines long. I would come at night when nobody else was using the machine, and debug it. And I found more than 100 bugs in my 100-line program. But 2 weeks later I had a program that would find prime factors of any 10-digit number that you dialed into the console switches.
That was how I learned programming — basically taking one program that I made up myself and sitting at a machine over a period of some weeks, and kept getting it to work a little better and a little better.
My second program was converting between binary and decimal. But my third program was a program to play tic-tac-toe and that was what really made me a programmer.
(p. 567)

On the “pretty mathematically intense stuff” in The Art of computer programming:

Like in the parts that I’m writing now, I’m starting out with stuff that’s in math journals that is written in jargon that I wouldn’t expect very many programmers to ever learn, and I’m trying to dejargonize it to the point where I can at least understand it. I try to give the key ideas and I try to simplify them the best I can, but then what happens is every five pages of my book is somebody’s career.
In other words, there’s still so much more beyond any five pages of my book that you can make a lifetime’s worth of study, because there’s just that much in computer science. Computer science doesn’t all boil down to a bunch of simple things. If it turned out that computer science was very simple, that all you needed to do was find the right 50 things and then learn them really well, then I would say, “OK, everybody in the world should know those 50 things and know them thoroughly.”
(p. 568-569)

And last but not least, as Ehud Lamm already mentioned, the importance of the history of the discipline for its current and future development:

Seibel: Do you feel like programmers and computer scientists are aware enough of the history of our field? It is, after all, a pretty short history.
Knuth: There aren’t too many that are scholars. Even when I started writing my books in 1963, I didn’t think people knew what had happened in 1959. I was reading in American Scientist last week about people who had rediscovered an algorithm that Boyer and Moore had discovered in 1980. It happens all the time that people don’t realize the glorious history that we have. The idea that people knew a thing or two in the ’70s is strange to a lot of young programmers.
It’s inevitable that in such a complicated field that people will be missing stuff. Hopefully with things like Wikipedia, achievements don’t get forgotten the way they were before. But I wish I could also instill in more people the love that I have for reading original sources. Not just knowing that so-and-so gets credit for doing something, but looking back and seeing what that person said in his own words. I think it’s a tremendous way to improve
your own skills.
For example I spent a good deal of time trying to look at Babylonian manuscripts of how they described algorithms 4,000 years ago, and what did they think about? Did they have while loops and stuff like this? How would they describe it? And to me this was very worthwhile for understanding about how the brain works, but also about how they discovered things.
A couple of years ago I found an old Sanskrit document from the 13th century that was about combinatorial math. Almost nobody the author knew would have had the foggiest idea what he was talking about. But I found a translation of this document and it was speaking to me. I had done similar kinds of thinking when I was beginning in computer programming.
(p. 599)

I hope this extensive quoting will appeal the reader to pick a copy of the book. It is full of valuable insights, not only for historians.

Knuth's programming language

I happen to be an alumnus of the same chapter of the same fraternity as Don Knuth, and the composite portrait that contained his photo used to hang in the hall directly across from my door. So with all due respect to Don Knuth, I think his taste in language design sucks. :-)

But I do appreciate that his goals are often a bit different than my own. I must admit that doing algorithmic analysis in Haskell, for example, is a far more challenging a task than C or Assembly.

The Sieve of Eratosthenes is one of my favorite examples. Even the inimitable Rob Pike has explicitly asserted that the unfaithful sieve is the same algorithm with the same asymptotic running time as the traditional sieve. This assertion appears at at 27:30 in his Google Tech Talk about Newsqueak.

Lately, I've also been toying with lucky number sieves a bit, in both C and Haskell. I don't have that much to report so far, but I've learned and relearned two things:

1. What's important to the prime sieve is unimportant to the lucky sieve, and what's unimportant to the prime sieve is critical to the lucky sieve.

2. It's quite easy to write programs in Haskell that appear to implement the same algorithm, but have clearly different asymptotic run times, empirically speaking. In some cases I still don't understand what the difference is.

Of course, if your main interest was algorithms and their analysis, due to the latter observation, Haskell would probably not be among your favourite languages. You'd definitely be better off with Scheme, C, or Assembly.

TeX macros are a scourge that either should have been properly designed or never unleashed upon the world at all. However, literate programming is very cool idea. I'm not sure it is suitable for general programming, but once you start writing executable papers and books, you never go back.

Ironically, the unholy union between TeX and Haskell is probably the best literate programming environment currently available. :-)

Donald Knuth and programming language design

Well, he says it himself: "Mostly I don’t consider that I have great talent for language design." (p. 597)

Great book!

This is a really great book.

I found Fran Allen's interview (quoted above by Ehud) provocative. To write sensible programs, I think that programmers need things like predictable performance models, well-defined order of evaluation, consistent behaviour between compilers, and so on. (Just look at the popularity of single-implementation languages.) But these are exactly the features that language and compiler researchers are least inclined to offer us.

I was tickled by exchanges like this:

Seibel: What has changed the most in the way you think about programming now, versus then?

Steele: I guess to me the biggest change is that nowadays you can't possibly know everything that's going on in the computer. There are things that are absolutely out of your control because it's impossible to know everything about the software. Back in the 70s a computer had only four thousand words of memory. It was possible to do a core dump and inspect every word to see if it was what you expected. It was reasonable to read the source listings of the operating system and see how that worked. And I did that - I studied the disk routines and the card-reader routines and wrote variants of my own. I felt as if I understood how the entire IBM 1130 worked. Or at least as much as I cared to know. You just can't do that anymore.

Oh yes you can! I read this while programming in Openfirmware, a self-sufficient sub-megabyte Forth operating system with its own compiler and highly readable source code. I too have studied the (SD) card-reader routines and written my own variants - just last month!

We have it a lot better today than people seem to think :-)

Steele: I guess to me the

Steele: I guess to me the biggest change is that nowadays you can't possibly know everything that's going on in the computer. There are things that are absolutely out of your control because it's impossible to know everything about the software. [...] I felt as if I understood how the entire IBM 1130 worked. Or at least as much as I cared to know. You just can't do that anymore.

What's the point? I know, only I would question the wisdom of this, but seriously, what's the point?

How has the design of systems changed since Dijkstra described his layering approach to the design of the THE operationg system? Isn't it more important these days to design a system using a consistent set of techniques that are easier for computers to understand and verify for us?

In other words, isn't it much easier these days to look at forty years or so of system design theory and say, "that won't work". When I was 11, back in 1995, and the book Design Patterns first came out, this was the biggest question on my mind. I didn't want to know what the coredump of my computer was. I want to build huge software systems all by myself that did impossibly amazing things no one else could do, because they were simply doing it wrong. I wanted an edge in system design expertise they didn't, and do much less work and achieve far greater results.

Sure, when you look at

Sure, when you look at closely confined areas, you are right. But when you work with, say, a popular operating system on your PC, with a couple of open applications, you are not right anymore. An application might misbehave, and you restart it and the effect is gone. Or you restart all applications, open the same documents, and the effect is gone. Or you reboot the computer, restart the same applications, open the same documents, and the effect is gone. And even the highliest skilled and involved engineers would have a hard time figuring out how that misbehaving came to be. Did it ever occur to you that we have silently crossed a line: we are now building from deterministic components non-deterministic systems.


No Andy Gavin, Jason Rubin from Naughtydog Software.

Those two guys are some of the most inspirational programmers I know of. [They designed the predecessor or early prototype of LLVM-like concepts, GOAL, except in many ways it was more robust than LLVM even is today]


Well, Jason Rubin isn't a programmer as far as I know :)

As much as I admire GOAL, I don't see the LLVM connection. (Not trying to be snarky.)

Snarkiness is okay, but

Snarkiness is okay, but could you explain why you don't see the LLVM connection? The Streaming world editors feature is sort of ahead of its time.

As for Rubin being a programmer or not, I'm not sure it matters. My CEO, who plays a role in the software design of what I code, hasn't written a line of code since the early 80s. A lot of programming is conceptual design, which is why I dislike Knuth's dumb comment about 2% of people being natural born programmers. I'd rather be born with a better set of natural gifts than just "born to program".

EDIT: Also, let me clarify, that I think while household names sell O'Reilly books, I am the sort of reader who adores Michael Lewis's NY Times sports column about some athlete you never heard about, doing something remarkable. In the same vein, you can't read a book written by Andy Gavin, so why not interview him in detail and see how a quiet but great programmer thinks? I've read interviews by Knuth and some of the others before, and unlike amazingly wittingly scientists like Alan Kay, they tend to repeat the same lessons learned over and over. This isn't to slight Knuth, who I adore, either. I did order Coders at Work today, but mainly due to people I've never heard of like this guy.

they're just both compilers

GOAL is a homebrew language and compiler. And it's very impressive that the language is used "all the way down" so to speak. It's not a DSL that's part of a larger C runtime.

LLVM, to the best of my knowledge, is a low-level JIT code generation system that can be used as a compilation target.

So I don't really see the connection, other than that both systems fit into the spectrum of compilers somewhere and are very cool :)

(BTW, the only reason I mentioned that Rubin isn't a programmer, is because I think the language design and tech work was mostly driven by Gavin. But Rubin was clearly integral to the vision of Naughty Dog's games. And truth be told I don't work as a programmer either, so I don't mean to belittle him!)

LLVM, to the best of my

LLVM, to the best of my knowledge, is a low-level JIT code generation system that can be used as a compilation target.

LLVM IR is also a low-level language, and the in-memory representation and the IR source and binary representations are semantically identical. The LLVM C++ source is compiled to this IR, so you could argue that the LLVM IR is also used "all the way down".

What GOAL are you talking about?


I did not heard of GOAL before, and as a programming language I found GOAL by Hindriks et al in the Nederlands.

Are you talking of the same GOAL? I don't see any obvious connexion between an agent programming language and LLVM!


Different GOAL

It's this GOAL.

The LLVM connection is obscure to me too. GOAL is certainly not a generic language back end.

Re-reading what I said, I

Re-reading what I said, I see why it was a little baffling. I wasn't intending to make a direction comparison between GOAL and LLVM.

GOAL compared to "core LLVM" would not make much sense, since as you point out it has many features that make it more robust than something developed by mostly one programmer at a game studio (generic language back end, etc.).

If somebody were to attempt to build GOAL today, they would very likely use LLVM to bootstrap the process. As I understand it, GOAL had some clever mechanisms that allowed integration of lisp, high-level assembly language and C that all worked against the same ABI. This doesn't sound particularly amazing, but when you consider that it supported the ability to replace pieces of code anywhere from a variable to an entire object, it is a good height of engineering. The real feature tht stands out about GOAL is the ability to send a sequence of assembly -- like a pretty print statement -- to the running executable and have it run it inline, without screwing up registers.

LLVM itself doesn't have to be used in a GOAL-like fashion. LLVM is a robust toolkit for building all kinds of programmable abstractions, including compile-time-only ones. LLVM's support for anything more than x86-based processors is also immature, as I understand it.

All I can really say is that nobody really engineers systems "the GOAL way" without building a robust code generator like LLVM. So LLVM is sort of a step in the direction of the sort of Ph.D.-engineered compiler you'd need to build GOAL systems. So, when I said LLVM-like concepts, that includes a lot of stuff not in the compile-time-only core LLVM module.

No, the one Anton linked to

No, the one Anton linked to is the right one.

You should read the Wikipedia article. I've got some other web articles about it lying around here. I've also managed to talk to some former NDS employees about it, to better understand it.

As a compiler junkie like me, you'd probably like to hear about it. It's neat stuff.

Obviously, you're more of a raw performance guy, but stuff like GOAL takes performance in a different direction: rather than pouring man-hours into the compiler, the exploratory nature of the environment allows pouring man-hours into the performance bottlenecks of the specific application, ripping out object code and replacing it with high-level tuned assembly code.

Knuth is my homeboy

Knuth has a challenge to programming language designers. He claims that every time a new language comes out it cleans up what's already understood, and then adds something new and experimental. How about "setting our sights lower" and aiming for stability. "It might be a good idea," he says.

Dylan probably comes pretty close to a language that didn't invent a lot of new stuff, and instead cleaned up everything they had learned. Dave Moon's Programming Language for Old Timers is similar.

Both languages have somewhat "experimental" syntaxes, though.


What did Java add that was new and experimental? It could qualify as "aiming for stability". Especially in its early incarnations.

It seemed very much about consolidating and popularizing features that were already quite well-understood, but not yet ubiquitous at the time.


The difference is that Dylan went a bit farther than Java (restartable conditions, first-class escape procedures, numerical tower, ...).

Update: removed macros from the list, because they were a new design.