Lambda the Ultimate

Following this thread, here is my review of Practical API Design.

Jaroslav Tulach is the founder and original architect of the NetBeans project.

Practical API Design is a book with apparently one bold aim: to be the book that gets people thinking seriously about the topic of API design -- in particular the maintenance of backwards compatibility in APIs -- and kick-starts an API design revolution, with more books and university courses to follow in its wake. Incidentally, it can also be a highly readable introduction to API design, a thought-provoking set of warnings to experienced engineers embarking on managing a growing API and an entertaining odyssey through the development of the NetBeans API. With regard to the main aim again, we must hope that more books and papers are indeed written developing this theme, as this book cannot do it all by itself; more on that topic later.

The book is curiously subtitled "Confessions of a Java API Designer", but due to the individual and highly digressive style (at least in part one) a more apt subtitle may be ""The Life and Opinions of Jaroslav Tulach, Gentleman". It may seem odd at first, but later in the book this style does show its strengths. My experience of trying to find good material on the subject of the maintenance of backwards compatibility is the same as the author's: I can find very little worth reading, and nothing that covers the subject in depth. Therefore the book must take a personal point of view, setting down beliefs and opinions and setting them down strongly in the hope that a reaction is provoked. The areas in which this book lacks are advanced as areas into which you, the reader, might contribute. This is the book as a call to arms rather than an attempt at solid theory. I think the approach works rather well.

The question uppermost on my mind when picking up the book for the first time was, "how Java-specific is this?". Tulach has attempted to push Java specifics into the second of the three parts (the largest), and has mostly succeeded, although some Java-centricity is noticable in part three. That is not to say that the second part is useless to those not using Java; far from it! This section is the practical part-- where the theory was built up and strengthened on the proving-grounds of NetBeans development; as such it is integral to the book whichever language you are using. But if you are using a different language, how much extra work is required? Quite a bit, unfortunately. What you can and cannot do in a compatible way is dependent on the details of the binary interface (which may well be compiler-specific if not laid down in the language or other specification), so there is no way at all that one book can attempt to do this in a generic way. My own point of view is Symbian OS APIs, which are affected by the C++ language in which they are written, the EABI binary interface specification and the decision to use link-by-ordinal -- this combination needs its own careful consideration and results in a different set of emphases to those Tulach advances; for example, the specific technical dos and don'ts require much more effort and explaination in C++, and the idea of a checking tool would warrant more than the four lines it gets in Practical API Design. This means that, if you are using a language other than Java, you must think about each piece of advice with respect to the peculiarities of the languages you are using. Worse than that, of course, you must work out for yourself what sorts of compatibility difficulties you might have that are entirely unmentioned by the book; C++ has many such pitfalls.

All of the code samples are in Java; most are clear and easy to understand, although often too fond of generics to be easily parsed by a newcomer to Java. Tulach also denies us any warm fuzzy feeling when one of his very first examples is a two-page horror involving some subtle implicit effects and highly Java-specific calls that will be totally opaque to the Java neophyte, as well as an apparent instance of double-checked locking and a curious piece of at-first-sight useless code that must be for multi-thread access (p76). Having studied it with some Java expert colleagues, we have convinced ourselves that the code is, in fact, correct. But we have not convinced ourselves that it is "good design". Tulach will retort, as he does many times in the text, that beauty should be sacrificed every time in favour of keeping compatibility, and he is right. So now working out if a nicer solution would have the same benefits is an exercise for the reader; this is not out of keeping with the book as a whole, as much is for the reader to work out. This is not because Tulach has failed to put together a compelling book; he has. It is because he has deliberately overreached in order to present to us a vision of what can be achieved with care, hard work and the right attitude. However, the low-level detailed answers must come from the reader.

As a reader embarking on the beginnings of their own project, the book takes the form of a stern warning of the pitfalls that lies ahead, and sketches some potential solutions. For a project using only Java, the book seems to me (I am no Java expert) a thorough sketch of the territory ahead. Colourful, passionately argued and totally convincing.

As a reader already scarred by the trials of compatibility maintenance, there will be many nods of recognition and some pointers of where the compatibility armour could do with some hardening. The book is a valuable read.

As a reader looking to translate NetBeans' lessons into another (machine) language, there is much more work to do, as I have already mentioned. There is no technical scaffolding against which to build a compatibility policy; no guide or checklist of things to look out for. For example, if one were to attempt to come up with compatibility rules for Haskell, one would have to delve deeply into how each element affects the binary to which others link, and how this linking occurs. Explaining this is not the purpose of the book. It will definately give a head-start as some issues will overlap between your target language and Java, at the expense of having to learn why certain things work and other things do not in Java. This is good education anyway-- I even found a new (somewhat grudging) respect for the language.

As a reader looking for cookbook techniques for the design and maintainance of Java APIs one is likely to be disappointed. The correct way to read Practical API Design seems to be with a notebook in hand, jotting down interesting topics and random thoughts that occur (some of my questions that occurred took many, many chapters to be answered), then re-reading and fleshing out how each topic applies to your situation. If you get that far, maybe you could publish it too as a cookbook! But Practical API Design is not a cookbook.

I was slightly disappointed by a particular omission; there is no discussion on the circumstances under which one might consider a break acceptable. For example, if you need a new system-wide feature (a multi-module search, for example, whose index would be invalidated by anybody having continuous access to the data), or a new architecture for performance purposes that makes an old API impossible, or an API is found to embody a security problem. Perhaps you just know these issues when you see them and it is acceptable to have little discussion. Tulach's opinion is firm, even with bug-compatibility. He will only admit that fixing a null pointer exception is OK; I can't help feeling that some discussion is warranted here.

There is only one place where I actually found myself outright disagreeing with the author, though. This is in the issue of whether maintaining compatibility of an API by making users enable the new API with an environment variable or other global switch is acceptable. I always argue that this is not OK in Symbian, too, and am routinely ignored, so perhaps Tulach is in good company. I feel that (for some APIs) pushing this decision on to customers (especially as our customers have their own customers to look after) rather than solving it more optimally ourselves is very rude.

At 383 non-index non-contents pages with 20 chapters covering testability, modularity, run-time issues, trust, teamwork and so on divided into 3 parts (basically philosophy, practice and process) it covers the ground very well. Some might find the tone too chatty, although I liked the Jekyll-and-Hyde split between the zealous hardliner of the main text and the practical consensus-builder of the (mostly NetBeans-specific) sidebars.

I enjoyed Practical API Design; I have not come across any book treating the subject of API maintenance in nearly such depth and breadth. Recommended for architects who care about their APIs, especially those using Java. Those using other languages will find much of value also.