Lambda the Ultimate

[Side note on McDonalds: surely popularity is different from quality! But it is nevertheless interesting to study why something is popular, and to see if quality might be one of the factors.]

My essay is primarily about objects as a design construct, not a language construct--so if you use ADT language constructs to encode objects, I would say you are essentially using objects, and you will benefit from the interoperability advantages they bring.

Of course, it is much easier to program according to an object-oriented design in a language with direct support for objects. Most people who want to write a framework (and gain the resulting reuse benefits) or set up a software ecosystem, will likely prefer to do it in an object-oriented language (there are exceptions, e.g. see GTK+). The pervasiveness and economic importance of frameworks and ecosystems argues that general purpose languages should provide good support for objects.

On the other hand I am not arguing against ADTs! In my opinion there is little reason not to support both well.

Hi Jonathan,

If you didn't read my post before the edit, don't let my disclaimer make you think I said anything too harsh. :) I realize your essay doesn't advocate again ADTs and I noticed that you are in favor of supporting both. I'm certainly in favor of having good support for objects.

My argument was mainly against the selected quote. I don't think that ADTs vs. an object encoding for a Set container is a case of pros and cons -- I'm arguing you want the ADT, full stop. The object encoding can be viewed as a specialization of the Set ADT but not vice versa (if you start with a Set ADT that requires private details to perform binary operations, you can't squeeze them through the object interface).

Here we go, in code:

module SetADT
   type Set a
   empty: Set a
   contains: Set a -> a -> Bool
   isSubsetOf: Set a -> Set a -> Bool

module SetObject extends SetADT
   type Set a = object
       contains: a -> Bool
       isSubsetOf: Set a -> Bool
   end
   contains o x = o.contains x
   isSubsetOf o o' = o.isSubsetOf o'

Hi Matt,

You are certainly right that you can define a SetADT interface and implement it using objects. But this does not mean there is not a tradeoff between ADTs and objects.

In your example, SetObject is only interoperably extensible (i.e. new modules can provide different implementations of the Set object, and these will all interoperate) if type Set a is exposed, as an object type, in the signature of SetObject. But then Set a is a manifest type, not an abstract data type. It hides nothing, and holds nothing abstract from a type perspective.

If "abstract data type" means anything as a design technique, it ought to mean you are hiding at least some part of the definition of a type. This is common in all the historical literature on ADTs that I know of. So we still have a tradeoff. You can define a SetADT signature, which is an ADT, but different implementations of it will not interoperate. Alternatively, you can define a SetObject signature, which is not an ADT because it defines the type Set a to be manifestly an object type, and then it is extensible and implementations do interoperate.

Jonathan,

I'm comparing two alternatives:

1) Only define SetObject and have clients bind to that.

2) Define Set, the ADT, and have most clients bind to that. Then, provide a SetObject implementation that clients only bind to if they need to know about the interoperability requirement (for example, because they're building new sets).

I'm arguing that as an architecture 2) has advantages and no real drawbacks over 1). If you go down the path of 1), then all client code will be dependent upon the assumption of a particular object implementation of Set, without any good reason. Under approach 2), only the code that needs to know about this interoperability requirement needs to know about the object encoding. But having a bunch of code using the ADT doesn't compromise interoperability unless there is a second implementation of the ADT used.

So yes, you would have to decide to give up this interoperability (locally) later on with approach 2) if you decided to use client code with another Set implementation, but having the option is better than not having it.

It sounds like cheating.

Clients cannot know that an ADT has an alternate implementation, by the very definition of an ADT having only one.

Clients who wish to use object-oriented Set cannot seamlessly integrate with ADT Set. Furthermore, in practice, the interfaces should ideally be different, so deriving one from the other and having a "magic" operator to cast between the two isn't really possible.

The idea is just that some clients only consume sets and don't need to know that you can construct them with objects. These clients can be used with other types of sets.

Sorry for the delayed reply--I was traveling all of yesterday.

Set is not a great example for objects; I started with it to set up the argument, and to explain why people like ADTs, but I did not use it to make my core case. It is plausable that Liskov was right, and many programs can get away with only one implementation of a datatype such as Set (I have some quibbles about this but we can assume it for now). In that case extensibility is moot, and the ADT is all you need.

In the case of frameworks, which according to my argument is one place where objects really matter, the abstractions only exist to be extended, so defining a datatype first is pointless.

But let's consider your argument about alternative (2) anyway. We have a system S that is made up of a bunch of components C1...Cn. A subset of these components use the Set ADT. A couple of these components also provide their own implementations of Set and create Set ADT instances from their implementations--perhaps because they want to use sets that are particularly efficient for their mode of use. Sets are exposed in the interface of several of the components, but we set up type sharing constraints* appropriately so that everything works.

Now we extend the system with component Cn+1. Component Cn+1 needs to get sets from several of the other components and take their unions, etc. So component Cn+1 defines SetObject and provides an implementation. We re-link all the components that only depend on Set to use Cn+1's implementation of SetObject. But we are in trouble because of the components that defined and instantiated their own Sets which are not SetObjects, and thus cannot interoperate, breaking Cn+1's needs. Since these components depend on the use of their own implementations, we cannot make the system work without drastic surgery.

In contrast, with alternative (1), adding component Cn+1 is easy, because everyone who defined a set before, defined it as a SetObject. Only alternative (1) ensures that the world is safe for extension.

So we are back to the original tradeoff. We can define Set as an ADT just in case we need to optimize for performance later. Or, we can define Set as an object, just in case we need extension and interoperation later. There are certainly problems and domains where you would pick performance, but for most application programming, design guidance suggests planning for extension is the better choice.

*As a side note, type sharing constraints don't scale well--see Pierce and Harper's ATAPL discussion of "fully functorized code" and why it is a bad idea. An implication is that you can't realistically make a statically typed module parametric in all the ADTs it uses. Notably, this issue doesn't come up with OO types, because they are not abstract.

First let me respond to your footnote: making fully functorized style work seamlessly is one of the goals of my own language project. My copy of ATAPL is in storage, but, as I recall, I didn't find the case as compelling as you make it sound. It was of the form "we've tried it and hit some annoying issues" rather than a systematic analysis of the problems involved with a careful consideration of possible solutions. This may be an important part of our disagreement.

As for your hypothetical development, you lose me right out of the gate:

A couple of these components also provide their own implementations of Set and create Set ADT instances from their implementations.

If you have multiple implementations of the Set ADT then you've already decided to forgo interoperability. If you want to keep interoperability, then don't do that. Perhaps your concern is that having the ADT around will lead to accidental abandonment of interoperability when someone adds an alternative Set implementation for expedience? This is related to issues with fully functorial code and I think this can be mostly addressed.

The scenario I'd want to focus on is that you have a system S that is made up of a bunch of components C1...Cn, where some subset only use the SetADT interface and the rest know about the SetObject interface. Now we want to use the components that only rely on the SetADT interface in some other situation entirely with another implementation of Set that doesn't need to be interoperable with the SetObject-using components.

I'd be excited to see what happens with your project on making the fully functorized style work. I'm quite interested in this problem too, although I think using object types solves most of the problem, making it much more tractable in the relatively few situations where you really want ADTs ;-).

Yes, my concern is that having the ADT around will lead to accidental abandonment of interoperability when someone adds an alternative Set implementation (not for expedience, but because they need one).

In the scenario you want me to focus on, your solution is fine, even beneficial in cases where there is genuine doubt about whether interoperability is needed. I just think that scenario is highly unrealistic, because if you publish a SetADT interface, people definitely *will* write and use their own implementations of it. If you can come up with a way for that to not be a problem, that would be very cool.

Unlike the OOP idea "everything is an object", the idea "everything is an ADT" is actually a good one. ;) That's really why I'm arguing that ADTs are more fundamental. I agree that this Set example is contrived (as an argument for either objects or ADTs) and also that most typically when you have an object type you won't need to replace that type.

In what cases are ADTs superior to objects, or is it just a matter of being less expressive (so more simple semantically) but expressive enough...i.e. eating your vegetables?

Right, it's kind of like the difference between "everything is a function" and "everything is a hashmap". But my approach is also based on a philosophy of only defining things up to their logical relationships and trying to minimize the number of assumptions you make about those other things. I think theories (ADTs/modules) are the way to this systematically. Objects can help do this, too, by hiding implementations behind interfaces (this is what many people like about objects), but they aren't good at modeling some relationships (binary+) and they add too many moving parts to logical contexts that should be simple.

I've been able to get objects to model full on graph relationships, but that involved getting rid of set theoretic types. This design space is kind of big for objects. ADTs on the other hand, sound quite simple and limited. If I were going to describe ADTs to a normal programmer, I'd say they were like classes, but less expressive.

Then you'd clearly be spreading misinformation to that programmer. Just observe that you can wrap every object into a respective ADT value and back, but the inverse does not generally work.

I don't buy that you can lift fundamental limitations of the object abstraction (which are independent from typing) by magic changes to the type system. (Also, note that most type theory is not "set-theoretic".)

Show me examples and don't talk about convertibility: what are actual use cases where using ADTs be better than using objects?

What about branding? Objects are not expressible as ADTs because existential types are not expressive enough to deal with branded class relationships (at least, this is what I remember from the Moby papers). You could augment ADTs with brands, but then you are basically turning them into classes.

Most type systems model subtyping as basically a compatibility/unification/subset relationship, which is overly restrictive and quite unnecessary. Getting rid of subtyping drastically opens up the design space for objects. I was going to mention something here, but its not fair to intrude on Jonathan's fairly good essay with my own fringe work.

What about branding? Objects are not expressible as ADTs because existential types are not expressive enough to deal with branded class relationships (at least, this is what I remember from the Moby papers). You could augment ADTs with brands, but then you are basically turning them into classes.

I could not google this concept of "branding". Can you please help.

(What follows is a questionably educated guess.)

I've heard the word `branding` used in only a couple scenarios with regards to OOP. One of them was money in E language, and another was some language with first-class classes (was it Newspeak? I can't recall). In both cases, the `type` of an object was essentially augmented with a first-class brand, which basically tracks where objects of that type were constructed. So if we had two different brands of integer - e.g. if I had `Int[dave]` and you had `Int[zbo]` - then they would effectively be different types. To add two Integers, we would need to access the `add` method from the correct object of `Math[dave]` vs. `Math[zbo]` type. I imagine this is what Sean means by "branded class relationship".

Now, brands aren't well motivated for integers (though I suppose using them as a unit or something), but there are some useful applications for branding - e.g. for allowing multiple versions of a module to coexist in a single application, or for objects that maintain some global relationships through their constructors. They are also useful if you basically want 'first-class' types, which I've occasionally thought would be useful for typing individual client-server connections. (cf. first-class type tags).

The brands don't actually need to be statically typed, since they could be checked dynamically. They could be cryptographically enforced, using signatures or similar.

Anyhow, as Sean mentions, existential types have a tough time dealing with branded class relationships. The existential concept (there exists some kind of integer and math) has no name for the particular existence once the elements are separated (let's hold the math over here and send the integer to my friend), no way to reconnect them when they come back together.

Conversely, if you augment ADTs with brands, you're basically turning their modules into first-class objects because you can have many different brands/instances of each module and their corresponding ADTs.

Then you'd clearly be spreading misinformation to that programmer. Just observe that you can wrap every object into a respective ADT value and back, but the inverse does not generally work.

I'm not equipped to discuss this myself, but it sounds like there's some violent agreement here. I think Sean McDirmid was saying ADTs are like classes but make a library less expressive for the library user, and here you're saying the presence of ADTs makes a language more expressive for the library writer. These claims can both be true.

Matt,

[edited to tone down the language--I should have seen your smiley ;) regarding the absolute claim about everything being an ADT]

We're deeply in opinion territory now. But despite the risk of going in more deeply, I'll respond with a few reasons why I think we can't just make everything an ADT:

• First of all, ADTs require static types. While I like static types in many situations, and maybe you do too, not everyone does. Whether static types is a good thing (which I would equate to a mix of programmer productivity and software quality) is not something one can prove mathematically; it is inherently an empirical question. And, the empirical evidence is not yet in (although it is coming in--we are doing some work in this area, as are others). My best guess is that the empirical evidence is going to say that static types are good in some situations and not in others. Until we find out, though, it's probably premature to say everything should be an ADT.

  Note, by the way, that objects work quite well without static types! They can even provide strong encapsulation without static types.

• As my essay argues (elaborating on prior work by Reynolds, Cook, and others), there's a tradeoff between the extensibility that objects provide, and the stronger reasoning about binary relationships that ADTs provide. I think you are trying to avoid that tradeoff by saying that objects can be ADTs, but as we already discussed (and even agreed, I think) this requires solving the problem of some component implementers implementing the ADT not using objects, and then discovering later on that we need them. I understand you are working on a solution to that problem (making the fully functorized style working seamlessly) but again, until you have a solution that the community can evaluate, this is a premature claim. Based on current technology, and for the reasons in my earlier post in this thread, I think defining objects as refinements of ADTs is a bad idea in cases where you may eventually need the interoperability of objects.
• The problem with the philosophical argument that one should try to minimize the assumptions one makes, is that ADTs make the fundamental assumption that there is one representation for your datatype. That is a stronger (more constraining) assumption than the object-oriented assumption, which allows multiple representations. Again, maybe you can fix this with a fully functorized style, but that remains to be seen. But for now, the "number of assumptions" made by ADTs and objects is at least incomparable.
• I do not agree that objects are fundamentally more complex than ADTs (in terms of "adding too many moving parts"). I would suggest that one can give the semantics of a dynamically typed object calculus with encapsulation with fewer and simpler rules than the semantics of a statically typed lambda calculus with ADTs.

Now, I am not one of those people who claim everything should be an object. I am quite sure, however, that not everything should be an ADT.

Jonathan,

You're right that we've strayed into opinions (at the very least unsubstantiated claims). I won't disagree with the points you've made, except to suggest that you should keep an open mind about everything being an ADT. What I mean by that is that all values are only defined up to their logical properties/relationships and are otherwise "abstract". I'm pretty confident this will work well as a programming style. Of course I'm not going to claim that it's the best style for everyone... or at least not today. :)

Matt

P.S. Sorry I missed the flame version :)

ADTs require static types.

That's not correct. The direct untyped equivalent of ADTs is tagging/wrapping with private names. You can also achieve similar effects by other means, e.g. indirection of a public index through a private map. Even encryption is closely related.

Hi Ray, thanks for your comment on my essay!

ADTs are indeed good for some things, but different implementations of an ADT cannot easily interoperate in the sense I mean. Two implementations of a Set ADT interface can coexist, but you cannot test whether an instance of one Set implementation is a subset of an instance of the other Set implementation, because the types are different and incomparable. As Cook's essay points out (and Reynolds before him), this allows ADTs to provide optimized implementations of binary methods, but it precludes interoperability of different implementations of an abstraction.

I would argue (following Cook) that Go's interface types, as well as records of function pointers, express the essence of objects. So these examples support the thesis of my essay. Of course, objects are much easier to define and use if the language supports them natively!

I'm not sure I know exactly what you mean by template programming, but generally type parameterization (without adding either existential types or subtyping) is not enough to support the rich forms of interoperability that are needed by framework code. A framework, for example, may need to have a dynamically constructed list of plugins, each one a different implementation of a common abstraction. Either subtyping or existential types seem to be required to support this.

In Ada (from about 93 forward) they use type in the traditional "Pascal" sense and type_name'class to means what a class name usually means in object-oriented languages. I wouldn't say they completely succeeded in unifying ADTs and objects but They clearly demonstrated that the two concepts are not as complementary as some people claim. I don't think this undermines your main points in any way and I generally agree with you paper. In fact it might actually strengthen your argument as the designers for the Ada revision clearly identified the interoperability as the key missing feature required to support the object paradigm. The interaction with genericity is particularly interesting to understand.

Thanks, I'll have a look.

"Objects" is a very loosely and ambiguously defined term, and exactly which features of objects are essential to success vs. existing for historical reasons is debatable.

And heavily debated!

I had thought of including that addition but I was already exhausted from having internally debated qualifying the statement with "It goes without saying", "As we all know", "Everyone would agree", "All right-minded people are aware", or "It hardly needs to be pointed out".

Lol. It works better coming from a second person anyway. :p

Definitions are important; my essay used Cook's definition (essay, Lambda the Ultimate discussion) which stated (in the essay) that "an object is a value exporting a procedural interface to data or behavior." This is equivalent to Reynolds's 1975 definition of "procedural data structures." Another way to describe the definition might be "an object is a first-class structure that carries its own behavior with it."

People may of course continue to debate whether this definition is the right one for objects. In my opinion, however, Cook makes a compelling case--one that is supported by OO history, quotes from pioneers such as Alan Kay, etc. Everyone I have talked to in the OO community feels that objects should provide at least what Cook's definition implies. Some would include inheritance or state, but most people I have talked to agree these are less fundamental than the property that Cook uses.

I think Cook's definition is at least a reasonable one, if not a consensus, and it has the virtue of being unambiguous. In my essay I argue that anything we know of which provides the notion of interoperable extension that I defined, meets this definition of objects. I'd be very interested, of course, to see a counterexample!

In my essay I argue that anything we know of which provides the notion of interoperable extension that I defined, meets this definition of objects.

The problem with this is that using the heavily-overloaded word "object" to describe this will inevitably lead to confusion. You may be excluding inheritance or mutable state from your definition, but the statement "Objects are inevitable", without qualification, does not automatically imply that exclusion, and seems a misleading way to characterize what you're actually saying.

It also doesn't say anything about the appropriate level at which such abstraction should be defined - e.g. if the concern is interoperability, then (as you observe in the essay), modules that provide something like COM-style exports fit the bill without necessarily requiring an internal language that resembles an OO language in any traditional sense.

However, someone seeing the general term "objects" used to describe this could be forgiven for thinking that this implies that some traditionally OO language should be the preferred vehicle for this "inevitable" technology, when that may not be the case at all.

This perception is reinforced because the essay is also explicitly defending "object-oriented programming" and "object-orientation", which terms generally imply much more than the restricted definition of objects that you're using.

Here's a representative quote from Section 5 that suffers from these issues:

This interoperability cannot be duplicated in other programming paradigms without essentially simulating objects.

The problem with this is that OO "paradigms" tend to involve much more than just the definition and use of objects in the sense you've defined, so it's misleading to use your conclusion to justify those paradigms, as you seem to be doing here.

The flip side of all this is that these obvious issues give those opposed to objects an excuse to dismiss what you're saying.

You'll note that these criticisms are not aimed at your core technical claims, which actually sound fairly reasonable to me although I haven't properly evaluated them yet. I do think it's important to identify and examine features, like interoperability mechanisms, that are critical to the success of production systems.

But the value I would hope to get out of such work is to find ways of moving beyond the ad-hoc systems that plague industry, towards languages and systems with a more rationally constructed and justified foundation. It would be a pity if instead the work was merely used to justify the very mistakes we should be trying to correct.

(It has also not escaped me that framing your contribution in the context of a never-ending flamewar du jour will certainly help stimulate discussion, as this thread demonstrates. But to help advance the state of the art may require a more careful separation of the technical concepts from the socially-based fluff that surrounds them.)

It also doesn't say anything about the appropriate level at which such abstraction should be defined - e.g. if the concern is interoperability, then (as you observe in the essay), modules that provide something like COM-style exports fit the bill without necessarily requiring an internal language that resembles an OO language in any traditional sense.

What does traditional sense mean here? Jonathan's argument is that you are essentially recreating objects, whether your internal language directly supports that or you have to do some extra work to get there.

The flip side of all this is that these obvious issues give those opposed to objects an excuse to dismiss what you're saying.

Audience is important. Such an essay written with respect to an ICFP rather than OOPSLA audience would be very interesting, though I think the for/against camps essentially come down to ideology and reasonable arguments are not effective either way.

It would be a pity if instead the work was merely used to justify the very mistakes we should be trying to correct.

I'm not sure the essay is accessible enough (e.g. it is not a B. Victor essay made for mass consumption) to reach such an audience that would jump to such conclusions.

What does traditional sense mean here?

Things like objects as the core abstraction mechanism, methods as the only kind of procedural abstraction, pervasive mutable state, a rather specific model of class-based implementation inheritance, etc.

Jonathan's argument is that you are essentially recreating objects, whether your internal language directly supports that or you have to do some extra work to get there.

Neither recreating nor extra work is necessarily implied. A language could have a facility for producing Cook-definition objects without necessarily having all of the baggage mentioned above as core features that can't be avoided.

The Clojure example discussed elsewhere in the thread could be considered an example along these lines. It's all very well to say that something like Clojure is object-oriented in the Cook/Aldrich sense, but in that case the definition of object-oriented is being radically expanded as a result of constraining what "object" means. This seems unhelpful to me.

I think the for/against camps essentially come down to ideology and reasonable arguments are not effective either way.
...
I'm not sure the essay is accessible enough (e.g. it is not a B. Victor essay made for mass consumption) to reach such an audience that would jump to such conclusions

The essay itself seems a bit ideological to me, but I don't think it needs to be to get the technical message across. It also seems to be equivocating somewhat along the lines I described, so it's not just the reaction of a broader audience that I'm thinking of.

Funny, my only complaint about the essay is that it's not ideological or provacative enough. To each his own, I guess.

Ever since I read Cook's essay, the term "object" has acquired a precise technical meaning in my vocabulary. I recognize that not everyone agrees with Cook. But the term "object" ought to mean something, and Cook's definition is arguably the best candidate out there.

Should I shy away from the term object in my essay just because some will misunderstand it when they read my title (and no further)? The definition I use is not hard to find in my essay--even the abstract has a pointer to the definition. I am interested in the question of why object-oriented programming has been successful. I do not pretend to have the full answer to that, but I believe my essay gives an important part of the answer. The answer in my essay does not rely on all the characteristics found in OO languages, but it does rely on the most fundamental of those characteristics.

What term would you use instead of "object"? Reynolds's term, "procedural data abstraction," is less ambiguous, but it is in a sense even more misleading in my context, because my argument relies critically on objects being used to abstract things other than data. I thought of using the term "service abstraction," following Kay's description of objects as servers offering services to their clients through a message-based interface (see The Early History of Smalltalk). But this may be misleading too, in the world of service-oriented architectures.

As for defending object-oriented programming--well, it is true that most object-oriented programming languages includes more features then my definition of objects. But it is also true that non-OO languages do not provide good support for my definition of objects--it can be done with a record of functions, but I know few OO people who would be satisfied with that. In that sense, I think my defense is on target.

In the end, I stuck with object, because it is the success of objects that I want to explain, and because I hope that my essay will help move the world towards a more rigorous, technical definition of the term object (following Cook's lead here).

But the term "object" ought to mean something, and Cook's definition is arguably the best candidate out there.

The term already means many things. Do you really want to base your message on an attempt to squeeze toothpaste back into the tube?

Should I shy away from the term object

I think it's important to distinguish carefully between your/Cook's restricted definition of "object", and the various and ubiquitous broader definitions, to avoid unintentional equivocation that may even affect your own thinking on the subject. The easiest way to do that may indeed be to use some other term.

I am interested in the question of why object-oriented programming has been successful. I do not pretend to have the full answer to that, but I believe my essay gives an important part of the answer.
...
But it is also true that non-OO languages do not provide good support for my definition of objects--it can be done with a record of functions, but I know few OO people who would be satisfied with that. In that sense, I think my defense is on target.

One important distinction that I want to preserve is that even if the features of your definition of "object" were an important part of the success of OO languages, and even if current alternatives do a poor job of satisfying this requirement, the kind of results that I would find useful would help us to move forward to better languages. Achieving that may very well entail rethinking much of what is generally accepted to constitute object-orientation and object-oriented programming today.

It's not clear to me from your essay whether you would agree with that, and it could easily be interpreted the other way, as a defense of the full OO status quo.

I thought of using the term "service abstraction," following Kay's description of objects as servers offering services to their clients through a message-based interface (see The Early History of Smalltalk). But this may be misleading too, in the world of service-oriented architectures.

"Service abstraction" seems less encumbered, at least. Perhaps you could qualify "service" in some way to distinguish from services that don't qualify as objects in your sense. You could also consider qualifying "object" somehow, e.g. "service object".

I hope that my essay will help move the world towards a more rigorous, technical definition of the term object (following Cook's lead here).

Is that an important or useful goal? More to the point, is it achievable?

Anton,

Thanks again for the constructive criticism. I have tentatively decided I like the idea of using a different term to denote exactly the element of objects that I am studying, and I revised the paper to use service abstraction for that (the new version is now at the original URL; you can find the old version as objects-essay-v1.pf). I'm still trying to make some general conclusions about objects, but I'm trying make the case through claims about service abstractions and then discuss the consequences for objects.

I'm not certain this will stay--I need to check with the Onward! PC to make sure they don't mind the change--but if it does I hope it will make the case more clearly and steer away from irrelevant objections.

and it all made plenty of sense to me. :-)

I think the argument based on how language shapes thought. To me, that we see objects as animate beings is key to understanding why objects are so accessible compared other core constructs of other paradigms. An object has behavior, it has an identity, it has encapsulated state, which are properties we attribute to physical things.

But the technical argument in this essay is very clear and easily testable. Are you claiming that no large frameworks easily exist without objects? In that sense, wouldn't a study of the Haskell ecosystem be useful in figuring what the alternative is and measuring its complexity somehow?

A small note for camera copy, but I think the DLL Hell comment might merit a Szyperski cite.

I'm not sure I can claim that no large frameworks would exist without objects. I'm trying to limit the number of absolute claims in my essay, which already has a few ;-). FoxNet is an (admittedly small) counterexample, as I discuss. There may be a few large, but highly static, frameworks which don't need objects. However, I am arguing that most frameworks rely critically on plugin interoperability, and therefore do require objects or their equivalent. That claim ought to in principle be amenable to empirical validation.

Regarding language shaping thought--surely it does, to some extent! I think it is likely that psychological effects have played a role in the success of objects. However, such effects are out of of scope in my essay, both because I don't consider myself qualified to make that argument, and to keep the essay more focused.

Thanks for the Szyperski pointer--"Greetings from DLL Hell" I assume.

It would be very interesting to analyze large frameworks and see (a) what they have in common and (b) if that commonness resembles objects. I would think that (b) might actually be true, but I have little Haskell experience to know even where to start looking. Unfortunately, confirmation bias is a huge danger here, and even if one does the study objectively, or even empirically, they'll be accused of bias by those unhappy with the results anyways. There are also problems with programmer bias, in that, even though they might be using Haskell, they could be tainted by OO experience in the design of the framework.

At the end of the day, I feel like we are losing our field. "Objects may have won" but the moral high-ground has gone to the FP community who constantly deride OOP as being the wrong, attacking a Java straw men to prove their point. There is demand for a stronger more provocative essay on the topic.

And yet, Rich Hickey supports and promotes objects in Clojure:

Clojure models its data structures as immutable objects represented by interfaces, and otherwise does not offer its own class system.

There is nothing in Jonathan's essay that goes against what Hickey says, where his attack is primarily on Java classes and not objects in general.

If you like it so much, print it out and stick it on the fridge... This discussion shouldn't be focused on what we like, but how we can apply abstraction to build better software. How would you build a better Widget API using Clojure's "simple associative model for information"? Also, would you apply Rich Hickey's comments that "objects thwart reuse" to the Widget API given in the paper?

While I dislike Jonathan's essay (probably because I am not the target audience), and may one day respond to it with constructive criticism, he at least carefully defined his argument. It is simply noise to juxtaposition a paragraph in an interview with an essay, because Rich doesn't define what a "typical OO application" is or examples of "explosion of code". Also, he uses the phrase "anti-reuse" ambiguously, as if there is only one kind of re-use.

That said, Jonathan's paper basically does explicitly define the re-use that comes from "interoperability" and "uniform treatment": The Open-Closed Principle. But this is also a weakness in his paper, since people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques. See footnote 13 on page 5:

The definition of modular extension captures Meyer’s open-closed principle [20], which states that abstractions “should be open for extension, but closed for modification.” It is particularly important when an abstraction is designed by one organization, but extended in an unanticipated way by another; in this case, the second organization may not be able to modify the original abstraction. 13

________________________
13 Such modifications may be literally impossible if the abstraction is distributed in binary form; or they may merely be infeasible in practice because the second organization wishes to avoid modifying (and then maintaining) a component it did not develop.

As an aside, I do not like this section of the paper. I hate the idea that designers can't anticipate how others will use their abstractions. It's not correct. Designers learn how problem domain experts think, and try to predict as many different variations as possible, and then try to create a stable interface that can satisfy all those variations. In other words, the author doesn't clearly explain what is unanticipated, and gives the sense that ghosts can emerge from the machine. How goals will be achieved is what is unanticipated.

At the same time, I am not convinced Widget goals need to be encapsulated in the Widget objects. For the Widget API, an alternative model is emerging, called Responsive Design, which essentially uses logic programming.

Rather than have a StackPanel Widget container which can only achieve its goal of laying out its children in a fixed way, HTML5 web developers are essentially using logic programs to query the system (browser media) and apply different layouts and sizes based on the media query results. In fact, not only does the position of a Widget change, but its display contents as well. The paint() goal is completely removed, such that the logic program instead computes a scene graph. Procedural abstraction is only used to hot-swap out logic programs, usually using $.ready. For example, applying a Google Wave-like ShortScroll bar.

Why, then, does Jonathan say the paint() goal is a good example of using objects to achieve high-level goals? The assumption is that a Layout Manager should automatically resolve the size of every Widget. To do this, the problem is decomposed such that every Widget reports how much size it would like to have, and also how much size it actually took.

What's also funny is that Jonathan thinks his Widget API provides two services, when most real-world developers I know would think of it as a single service:

the abstraction captures the negotiation between a UI element and its container concerning how large the UI element should be, as well as the goal of painting itself on a display device

...and that the paint() goal should be split into two separate scene graph services / sub-goals: measure and arrange. Then rendering can be encapsulated in a separate component which can make smarter decisions and optimizations, including computing efficient "damage repair" and "hit detection" when composing scene graphs. Most common "layout Widgets" implement constraint-based layout algorithms. A common variant is called linear constraints, where Widget positions are expressed as a system of linear equations. Fixed, relative, stacked and grid layouts are all specializations of linear constraints. Ergo, rather than make "open" a layout Panel which measures and arranges its children, it would be more general to simply expose a linear constraint solver. For layouts that cannot be expressed in terms of linear constraints, you simply glue the two together. And it is easy enough to name constraints like StackWithVerticalOrientation, StackWithHorizontalOrientation... or vbox and hbox for short.

Also, I would argue the Smalltalk-80 GUI API was not a huge success relative to the success of bitblt, which enabled easy portability. For example, the way Smalltalk-80 decided which Controller "seized" control of the event loop and could intercept events was insecure and not modular, in my humble opinion. Many GUI libraries, like Swing, also have perpetual inconsistent event handling, because of poor event loop handling. In turn, this caused IBM to build SWT as a Swing replacement, simply to fix problems caused by developer's inability to do interoperable event handling!!! How hard is it to program OnMouseEnter and OnMouseLeave such that there is a consistent ordering to those events?

Puff, puff, puff. I am out of steam for now.

Hi Z-Bo,

Thanks for your feedback. I'm sorry you didn't like the essay, but perhaps I can make some improvements to it that you and others would appreciate.

Regarding the term 'unanticipated,' I use it in what I think is the usual sense, but perhaps I should clarify:

By *unanticipated extension* I mean an extension for which neither the extension's code, nor the particular feature(s) being added by the extension, were explicitly anticipated by the designer of the abstraction being extended. Of course, in order to design an extension point at all, the designer must have some idea of the class of features the extension point is intended to support, but it is typically impractical to enumerate all features that fall into that class.

I think I will add a paragraph like the above to the essay; does it clarify my meaning for you?

Also, can I ask what you mean by saying people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques?

Regarding Widgets and Smalltalk--my goal is to talk about the history and what is in widespread use, not to claim that the designs I discuss are the best. My Widget example is taken more or less directly from Apache Pivot, but is very similar to what you see in Swing or SWT. I like the idea of specifying constraints via logic programming, but I think to explore this would be a diversion in the essay. Smalltalk-80 is not the most advanced GUI framework, but it was the first, I believe; although it was not commercially successful, it had a huge influence on later frameworks that were. But--any suggestions you may have about how to get the central points of the essay across more effectively would be appreciated.

There are not slight objections.

any suggestions you may have about how to get the central points of the essay across more effectively would be appreciated.

First, regarding logic programming GUI framework versus an object-oriented GUI framework. There is a reason WHY object-oriented GUI frameworks have been successful, and you completely missed it in your paper. It is exactly because of interoperability and extensibility, but the example you give doesn't illustrate that. Think of the evolution of Microsoft Office, which typically had many plug-ins written in Visual Basic and not Win32. These plug-ins used COM (component object model) OLE (object linking and embedding). Typically a Visual Basic plug-in would even use a different API to paint to the screen. Also, since Office plug-ins do more than just paint, and also need to implement a Caretaker pattern to serialize their state when Office tells it that it wants its memory back, this is a pretty fancy example using the most widely deployed application in history.

There is a pretty clear reason why a logic programming approach would not work here. I explicitly say the renderer is taken over by the system in this scenario, which precludes delegating to different drawing APIs. This ability to glue different models together allows people to create different approaches to solving problems in the same domain. (As a philosophical point not worth putting in the paper, I believe this has significant evolutionary meaning, as groups need the ability to diverge and go down separate paths before converging on an optimal solution.)

can I ask what you mean by saying people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques?

In terms of language mechanisms for open recursion, such as super and inner, How many ways can you name that OO programmers use OCP? Likewise, how many ways can you name that "ADT programmers" use OCP? I can list it for you, but I would want to create a fairly exhaustive list, and I can't do that for you right now. But I just know objects are far more complicated when applying OCP.

By *unanticipated extension* I mean an extension for which neither the extension's code, nor the particular feature(s) being added by the extension, were explicitly anticipated by the designer of the abstraction being extended. Of course, in order to design an extension point at all, the designer must have some idea of the class of features the extension point is intended to support, but it is typically impractical to enumerate all features that fall into that class.
I think I will add a paragraph like the above to the essay; does it clarify my meaning for you?

It just adds words! I already said what you should say. How goals are achieved cannot be anticipated. That's pithy. Maybe you had something else in mind, though.

[Edit: I suppose a totally unanticipated usage of Visual Basic would be third party vendors using VBX and then OCX to create non-visual libraries for things like file compression.]

You're right, these weren't quibbles.

Thanks for the Microsoft Office example, it is a great illustration!

Regarding the Open-Closed Principle, you're definitely right that its application to objects is complicated--but I am just looking at the dynamic dispatch nature of objects (inheritance is out of scope), so I think the issues are simpler in that case. Either you use an abstraction, or you reimplement it.

I did have slightly more in mind than "how goals are achieved" with respect to anticipation--often the designer of a framework knows the kind of goals to be supported, but has not enumerated all the possible specific goals that plugins might have. But in any case, you were right that a clarification was needed.