The Power of Interoperability: Why Objects Are Inevitable

Essay by J. Aldrich to appear at onward! Abstract:

Three years ago in this venue, Cook argued that the essence of objects is procedural data abstraction. His observation raises a natural question: if procedural data abstraction is the essence of objects, has it contributed to the empirical success of objects, and if so, how?

This essay attempts to answer that question. It begins by reviewing Cook’s definition and then, following Kay, broadens the scope of inquiry to consider objects that abstract not just data, but higher-level goals. Using examples taken from object-oriented frameworks, I illustrate the unique design leverage that objects provide: the ability to define abstractions that can be extended, and whose extensions are interoperable. The essay argues that the form of interoperable extension supported by objects is essential to modern software: many modern frameworks and ecosystems could not have been built without objects or their equivalent. In this sense, the success of objects was not a coincidence: it was inevitable.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Why have McDonalds hamburgers been so successful?

Edit: Since we're in holy war territory here, I've tried to sand down some of the sharp corners that were in the original post. The subject line is just a reminder that popularity is different from quality.

The disadvantage of ADTs, relative to objects, is the
lack of interoperability

Is this really a disadvantage? The object pattern can be used as the implementation of the ADT! So we can have this relationship between module signatures:

SetInterface :> SetObjectInterface :> SetImplementationX 

The SetInterface provides the standard set ADT. The SetObjectInterface provides a refinement of that ADT that specifies that the abstract type is an object encoding and gives implementations of all of the operations by having each operation delegate to the abstract type through the object encoding. Finally specific implementations can be implemented of the various operations.

This allows most code to be written to the SetInterface so that it can work with any Set -- object or not. Meanwhile, code that needs to be aware of multiple implementations of Set can bind to the more specific SetObjectInterface to make use of that interoperability.

This is in accord with what I think is a fundamental design principle: code should depend upon the minimum number of assumptions as possible. The OOP paradigm breaks this principle by requiring all values to adhere to the object pattern.

It is object *designs* that provide interoperability

[Side note on McDonalds: surely popularity is different from quality! But it is nevertheless interesting to study why something is popular, and to see if quality might be one of the factors.]

My essay is primarily about objects as a design construct, not a language construct--so if you use ADT language constructs to encode objects, I would say you are essentially using objects, and you will benefit from the interoperability advantages they bring.

Of course, it is much easier to program according to an object-oriented design in a language with direct support for objects. Most people who want to write a framework (and gain the resulting reuse benefits) or set up a software ecosystem, will likely prefer to do it in an object-oriented language (there are exceptions, e.g. see GTK+). The pervasiveness and economic importance of frameworks and ecosystems argues that general purpose languages should provide good support for objects.

On the other hand I am not arguing against ADTs! In my opinion there is little reason not to support both well.

Smaller argument

Hi Jonathan,

If you didn't read my post before the edit, don't let my disclaimer make you think I said anything too harsh. :) I realize your essay doesn't advocate again ADTs and I noticed that you are in favor of supporting both. I'm certainly in favor of having good support for objects.

My argument was mainly against the selected quote. I don't think that ADTs vs. an object encoding for a Set container is a case of pros and cons -- I'm arguing you want the ADT, full stop. The object encoding can be viewed as a specialization of the Set ADT but not vice versa (if you start with a Set ADT that requires private details to perform binary operations, you can't squeeze them through the object interface).

Here we go, in code:

module SetADT
   type Set a
   empty: Set a
   contains: Set a -> a -> Bool
   isSubsetOf: Set a -> Set a -> Bool

module SetObject extends SetADT
   type Set a = object
       contains: a -> Bool
       isSubsetOf: Set a -> Bool
   end
   contains o x = o.contains x
   isSubsetOf o o' = o.isSubsetOf o'

If SetObject is interoperably extensible, it is not an ADT

Hi Matt,

You are certainly right that you can define a SetADT interface and implement it using objects. But this does not mean there is not a tradeoff between ADTs and objects.

In your example, SetObject is only interoperably extensible (i.e. new modules can provide different implementations of the Set object, and these will all interoperate) if type Set a is exposed, as an object type, in the signature of SetObject. But then Set a is a manifest type, not an abstract data type. It hides nothing, and holds nothing abstract from a type perspective.

If "abstract data type" means anything as a design technique, it ought to mean you are hiding at least some part of the definition of a type. This is common in all the historical literature on ADTs that I know of. So we still have a tradeoff. You can define a SetADT signature, which is an ADT, but different implementations of it will not interoperate. Alternatively, you can define a SetObject signature, which is not an ADT because it defines the type Set a to be manifestly an object type, and then it is extensible and implementations do interoperate.

SetObject isn't the ADT. Set is.

Jonathan,

I'm comparing two alternatives:

1) Only define SetObject and have clients bind to that.

2) Define Set, the ADT, and have most clients bind to that. Then, provide a SetObject implementation that clients only bind to if they need to know about the interoperability requirement (for example, because they're building new sets).

I'm arguing that as an architecture 2) has advantages and no real drawbacks over 1). If you go down the path of 1), then all client code will be dependent upon the assumption of a particular object implementation of Set, without any good reason. Under approach 2), only the code that needs to know about this interoperability requirement needs to know about the object encoding. But having a bunch of code using the ADT doesn't compromise interoperability unless there is a second implementation of the ADT used.

So yes, you would have to decide to give up this interoperability (locally) later on with approach 2) if you decided to use client code with another Set implementation, but having the option is better than not having it.

I don't get what you are doing

It sounds like cheating.

Clients cannot know that an ADT has an alternate implementation, by the very definition of an ADT having only one.

Clients who wish to use object-oriented Set cannot seamlessly integrate with ADT Set. Furthermore, in practice, the interfaces should ideally be different, so deriving one from the other and having a "magic" operator to cast between the two isn't really possible.

The idea is just that some

The idea is just that some clients only consume sets and don't need to know that you can construct them with objects. These clients can be used with other types of sets.

You still need to decide up front

Sorry for the delayed reply--I was traveling all of yesterday.

Set is not a great example for objects; I started with it to set up the argument, and to explain why people like ADTs, but I did not use it to make my core case. It is plausable that Liskov was right, and many programs can get away with only one implementation of a datatype such as Set (I have some quibbles about this but we can assume it for now). In that case extensibility is moot, and the ADT is all you need.

In the case of frameworks, which according to my argument is one place where objects really matter, the abstractions only exist to be extended, so defining a datatype first is pointless.

But let's consider your argument about alternative (2) anyway. We have a system S that is made up of a bunch of components C1...Cn. A subset of these components use the Set ADT. A couple of these components also provide their own implementations of Set and create Set ADT instances from their implementations--perhaps because they want to use sets that are particularly efficient for their mode of use. Sets are exposed in the interface of several of the components, but we set up type sharing constraints* appropriately so that everything works.

Now we extend the system with component Cn+1. Component Cn+1 needs to get sets from several of the other components and take their unions, etc. So component Cn+1 defines SetObject and provides an implementation. We re-link all the components that only depend on Set to use Cn+1's implementation of SetObject. But we are in trouble because of the components that defined and instantiated their own Sets which are not SetObjects, and thus cannot interoperate, breaking Cn+1's needs. Since these components depend on the use of their own implementations, we cannot make the system work without drastic surgery.

In contrast, with alternative (1), adding component Cn+1 is easy, because everyone who defined a set before, defined it as a SetObject. Only alternative (1) ensures that the world is safe for extension.

So we are back to the original tradeoff. We can define Set as an ADT just in case we need to optimize for performance later. Or, we can define Set as an object, just in case we need extension and interoperation later. There are certainly problems and domains where you would pick performance, but for most application programming, design guidance suggests planning for extension is the better choice.

*As a side note, type sharing constraints don't scale well--see Pierce and Harper's ATAPL discussion of "fully functorized code" and why it is a bad idea. An implication is that you can't realistically make a statically typed module parametric in all the ADTs it uses. Notably, this issue doesn't come up with OO types, because they are not abstract.

If that is painful then don't do that

First let me respond to your footnote: making fully functorized style work seamlessly is one of the goals of my own language project. My copy of ATAPL is in storage, but, as I recall, I didn't find the case as compelling as you make it sound. It was of the form "we've tried it and hit some annoying issues" rather than a systematic analysis of the problems involved with a careful consideration of possible solutions. This may be an important part of our disagreement.

As for your hypothetical development, you lose me right out of the gate:

A couple of these components also provide their own implementations of Set and create Set ADT instances from their implementations.

If you have multiple implementations of the Set ADT then you've already decided to forgo interoperability. If you want to keep interoperability, then don't do that. Perhaps your concern is that having the ADT around will lead to accidental abandonment of interoperability when someone adds an alternative Set implementation for expedience? This is related to issues with fully functorial code and I think this can be mostly addressed.

The scenario I'd want to focus on is that you have a system S that is made up of a bunch of components C1...Cn, where some subset only use the SetADT interface and the rest know about the SetObject interface. Now we want to use the components that only rely on the SetADT interface in some other situation entirely with another implementation of Set that doesn't need to be interoperable with the SetObject-using components.

if you publish the interface, they will implement it

I'd be excited to see what happens with your project on making the fully functorized style work. I'm quite interested in this problem too, although I think using object types solves most of the problem, making it much more tractable in the relatively few situations where you really want ADTs ;-).

Yes, my concern is that having the ADT around will lead to accidental abandonment of interoperability when someone adds an alternative Set implementation (not for expedience, but because they need one).

In the scenario you want me to focus on, your solution is fine, even beneficial in cases where there is genuine doubt about whether interoperability is needed. I just think that scenario is highly unrealistic, because if you publish a SetADT interface, people definitely *will* write and use their own implementations of it. If you can come up with a way for that to not be a problem, that would be very cool.

Few situations for ADTs!?

Unlike the OOP idea "everything is an object", the idea "everything is an ADT" is actually a good one. ;) That's really why I'm arguing that ADTs are more fundamental. I agree that this Set example is contrived (as an argument for either objects or ADTs) and also that most typically when you have an object type you won't need to replace that type.

In what cases are ADTs

In what cases are ADTs superior to objects, or is it just a matter of being less expressive (so more simple semantically) but expressive enough...i.e. eating your vegetables?

Right, it's kind of like the

Right, it's kind of like the difference between "everything is a function" and "everything is a hashmap". But my approach is also based on a philosophy of only defining things up to their logical relationships and trying to minimize the number of assumptions you make about those other things. I think theories (ADTs/modules) are the way to this systematically. Objects can help do this, too, by hiding implementations behind interfaces (this is what many people like about objects), but they aren't good at modeling some relationships (binary+) and they add too many moving parts to logical contexts that should be simple.

I've been able to get

I've been able to get objects to model full on graph relationships, but that involved getting rid of set theoretic types. This design space is kind of big for objects. ADTs on the other hand, sound quite simple and limited. If I were going to describe ADTs to a normal programmer, I'd say they were like classes, but less expressive.

Misinformed

Then you'd clearly be spreading misinformation to that programmer. Just observe that you can wrap every object into a respective ADT value and back, but the inverse does not generally work.

I don't buy that you can lift fundamental limitations of the object abstraction (which are independent from typing) by magic changes to the type system. (Also, note that most type theory is not "set-theoretic".)

Show me examples and don't

Show me examples and don't talk about convertibility: what are actual use cases where using ADTs be better than using objects?

What about branding? Objects are not expressible as ADTs because existential types are not expressive enough to deal with branded class relationships (at least, this is what I remember from the Moby papers). You could augment ADTs with brands, but then you are basically turning them into classes.

Most type systems model subtyping as basically a compatibility/unification/subset relationship, which is overly restrictive and quite unnecessary. Getting rid of subtyping drastically opens up the design space for objects. I was going to mention something here, but its not fair to intrude on Jonathan's fairly good essay with my own fringe work.

Please help.

What about branding? Objects are not expressible as ADTs because existential types are not expressive enough to deal with branded class relationships (at least, this is what I remember from the Moby papers). You could augment ADTs with brands, but then you are basically turning them into classes.

I could not google this concept of "branding". Can you please help.

Branded class

(What follows is a questionably educated guess.)

I've heard the word `branding` used in only a couple scenarios with regards to OOP. One of them was money in E language, and another was some language with first-class classes (was it Newspeak? I can't recall). In both cases, the `type` of an object was essentially augmented with a first-class brand, which basically tracks where objects of that type were constructed. So if we had two different brands of integer - e.g. if I had `Int[dave]` and you had `Int[zbo]` - then they would effectively be different types. To add two Integers, we would need to access the `add` method from the correct object of `Math[dave]` vs. `Math[zbo]` type. I imagine this is what Sean means by "branded class relationship".

Now, brands aren't well motivated for integers (though I suppose using them as a unit or something), but there are some useful applications for branding - e.g. for allowing multiple versions of a module to coexist in a single application, or for objects that maintain some global relationships through their constructors. They are also useful if you basically want 'first-class' types, which I've occasionally thought would be useful for typing individual client-server connections. (cf. first-class type tags).

The brands don't actually need to be statically typed, since they could be checked dynamically. They could be cryptographically enforced, using signatures or similar.

Anyhow, as Sean mentions, existential types have a tough time dealing with branded class relationships. The existential concept (there exists some kind of integer and math) has no name for the particular existence once the elements are separated (let's hold the math over here and send the integer to my friend), no way to reconnect them when they come back together.

Conversely, if you augment ADTs with brands, you're basically turning their modules into first-class objects because you can have many different brands/instances of each module and their corresponding ADTs.

Then you'd clearly be

Then you'd clearly be spreading misinformation to that programmer. Just observe that you can wrap every object into a respective ADT value and back, but the inverse does not generally work.

I'm not equipped to discuss this myself, but it sounds like there's some violent agreement here. I think Sean McDirmid was saying ADTs are like classes but make a library less expressive for the library user, and here you're saying the presence of ADTs makes a language more expressive for the library writer. These claims can both be true.

Not everything is an ADT

Matt,

[edited to tone down the language--I should have seen your smiley ;) regarding the absolute claim about everything being an ADT]

We're deeply in opinion territory now. But despite the risk of going in more deeply, I'll respond with a few reasons why I think we can't just make everything an ADT:

  • First of all, ADTs require static types. While I like static types in many situations, and maybe you do too, not everyone does. Whether static types is a good thing (which I would equate to a mix of programmer productivity and software quality) is not something one can prove mathematically; it is inherently an empirical question. And, the empirical evidence is not yet in (although it is coming in--we are doing some work in this area, as are others). My best guess is that the empirical evidence is going to say that static types are good in some situations and not in others. Until we find out, though, it's probably premature to say everything should be an ADT.

    Note, by the way, that objects work quite well without static types! They can even provide strong encapsulation without static types.

  • As my essay argues (elaborating on prior work by Reynolds, Cook, and others), there's a tradeoff between the extensibility that objects provide, and the stronger reasoning about binary relationships that ADTs provide. I think you are trying to avoid that tradeoff by saying that objects can be ADTs, but as we already discussed (and even agreed, I think) this requires solving the problem of some component implementers implementing the ADT not using objects, and then discovering later on that we need them. I understand you are working on a solution to that problem (making the fully functorized style working seamlessly) but again, until you have a solution that the community can evaluate, this is a premature claim. Based on current technology, and for the reasons in my earlier post in this thread, I think defining objects as refinements of ADTs is a bad idea in cases where you may eventually need the interoperability of objects.
  • The problem with the philosophical argument that one should try to minimize the assumptions one makes, is that ADTs make the fundamental assumption that there is one representation for your datatype. That is a stronger (more constraining) assumption than the object-oriented assumption, which allows multiple representations. Again, maybe you can fix this with a fully functorized style, but that remains to be seen. But for now, the "number of assumptions" made by ADTs and objects is at least incomparable.
  • I do not agree that objects are fundamentally more complex than ADTs (in terms of "adding too many moving parts"). I would suggest that one can give the semantics of a dynamically typed object calculus with encapsulation with fewer and simpler rules than the semantics of a statically typed lambda calculus with ADTs.
  • Now, I am not one of those people who claim everything should be an object. I am quite sure, however, that not everything should be an ADT.

Thanks for all the fruit

Jonathan,

You're right that we've strayed into opinions (at the very least unsubstantiated claims). I won't disagree with the points you've made, except to suggest that you should keep an open mind about everything being an ADT. What I mean by that is that all values are only defined up to their logical properties/relationships and are otherwise "abstract". I'm pretty confident this will work well as a programming style. Of course I'm not going to claim that it's the best style for everyone... or at least not today. :)

Matt

P.S. Sorry I missed the flame version :)

ADTs don't require types

ADTs require static types.

That's not correct. The direct untyped equivalent of ADTs is tagging/wrapping with private names. You can also achieve similar effects by other means, e.g. indirection of a public index through a private map. Even encryption is closely related.

Objects are not the only abstraction that provides that.

ADT's are quite good, and can follow identical interfaces while coexisting in the same program. This makes them effectively interchangeable for some interfaces, and you could argue that some of them are therefore "subtypes" of some others.

More than once I've implemented "objects" in C by directly using assignments to members of structs having pointer-to-procedure types. Thus, you have a function that returns an object of foo type, and then you have a procedure that modifies a foo into a foo-subtype bar by changing its methods using assignment, or by using realloc to expand its size, or by casting the additional memory to an anonymous struct type giving the added member variables. Bar is still type foo and foo's methods can be called on it.

The type/subtype relation you mention is directly supported by template programming without objects, in several languages.

Objects are not the only form of procedural abstraction; in functional languages, procedures themselves are variables and can be passed around or created during runtime, and really powerful procedural abstractions can be built by doing so. Also, nothing possible with objects is difficult to do using procedural closures.

Finally, plugins all written to the same interface are not usually implemented as "objects" (though they might be depending on the language) and by virtue of having the same interface or being forced to have the same interface, programs can handle them uniformly - making them effectively the same "type" to the program even though they might not have a single data member or method in common. One can (I have) implement a time-keeping mechanism for a game as a sequence of reified procedure calls in C, each with its own arguments captured prior to putting it in the sequence -- even though they don't have so much as a single procedure name in common, and the "reify" function takes a void* function pointer and arbitrary varargs. Yes, it's ugly and not typesafe. But it's debugged and works like a charm and I never have to touch it to add new plugins and new functionality to the game.

Hmm, what else? Oh yeah, Go has interface types. So there aren't really objects and subtypes as such. User-defined types can implement any arbitrary subset of interfaces, even having no structural similarity and no simple type-subtype relations other than everything being a subtype of every interface it implements.

If I sit here and think for a while, I can probably come up with more examples.

ADTs are good - but not for interoperability

Hi Ray, thanks for your comment on my essay!

ADTs are indeed good for some things, but different implementations of an ADT cannot easily interoperate in the sense I mean. Two implementations of a Set ADT interface can coexist, but you cannot test whether an instance of one Set implementation is a subset of an instance of the other Set implementation, because the types are different and incomparable. As Cook's essay points out (and Reynolds before him), this allows ADTs to provide optimized implementations of binary methods, but it precludes interoperability of different implementations of an abstraction.

I would argue (following Cook) that Go's interface types, as well as records of function pointers, express the essence of objects. So these examples support the thesis of my essay. Of course, objects are much easier to define and use if the language supports them natively!

I'm not sure I know exactly what you mean by template programming, but generally type parameterization (without adding either existential types or subtyping) is not enough to support the rich forms of interoperability that are needed by framework code. A framework, for example, may need to have a dynamically constructed list of plugins, each one a different implementation of a common abstraction. Either subtyping or existential types seem to be required to support this.

ADTs can interoperate better than you might think

In Ada (from about 93 forward) they use type in the traditional "Pascal" sense and type_name'class to means what a class name usually means in object-oriented languages. I wouldn't say they completely succeeded in unifying ADTs and objects but They clearly demonstrated that the two concepts are not as complementary as some people claim. I don't think this undermines your main points in any way and I generally agree with you paper. In fact it might actually strengthen your argument as the designers for the Ada revision clearly identified the interoperability as the key missing feature required to support the object paradigm. The interaction with genericity is particularly interesting to understand.

Thanks, I'll have a look.

Thanks, I'll have a look.

Or their equivalent

many modern frameworks and ecosystems could not have been built without objects or their equivalent.

"Or their equivalent" is the key phrase here. "Objects" is a very loosely and ambiguously defined term, and exactly which features of objects are essential to success vs. existing for historical reasons is debatable.

We can agree that "interoperable extension" is a critical success factor for certain kinds of systems, but that doesn't necessarily constrain solutions to "objects" as usually conceived.

The possible solutions implied by "or their equivalent" is likely to be a fairly large solution space, and we should be careful not to equivocate on the poorly-defined word "object" to constrain that space without proper justification.

"Objects" is a very loosely

"Objects" is a very loosely and ambiguously defined term, and exactly which features of objects are essential to success vs. existing for historical reasons is debatable.

And heavily debated!

Cutting room floor

I had thought of including that addition but I was already exhausted from having internally debated qualifying the statement with "It goes without saying", "As we all know", "Everyone would agree", "All right-minded people are aware", or "It hardly needs to be pointed out".

Lol. It works better coming

Lol. It works better coming from a second person anyway. :p

Cook's definition of objects

Definitions are important; my essay used Cook's definition (essay, Lambda the Ultimate discussion) which stated (in the essay) that "an object is a value exporting a procedural interface to data or behavior." This is equivalent to Reynolds's 1975 definition of "procedural data structures." Another way to describe the definition might be "an object is a first-class structure that carries its own behavior with it."

People may of course continue to debate whether this definition is the right one for objects. In my opinion, however, Cook makes a compelling case--one that is supported by OO history, quotes from pioneers such as Alan Kay, etc. Everyone I have talked to in the OO community feels that objects should provide at least what Cook's definition implies. Some would include inheritance or state, but most people I have talked to agree these are less fundamental than the property that Cook uses.

I think Cook's definition is at least a reasonable one, if not a consensus, and it has the virtue of being unambiguous. In my essay I argue that anything we know of which provides the notion of interoperable extension that I defined, meets this definition of objects. I'd be very interested, of course, to see a counterexample!

Conflation considered... you know

In my essay I argue that anything we know of which provides the notion of interoperable extension that I defined, meets this definition of objects.

The problem with this is that using the heavily-overloaded word "object" to describe this will inevitably lead to confusion. You may be excluding inheritance or mutable state from your definition, but the statement "Objects are inevitable", without qualification, does not automatically imply that exclusion, and seems a misleading way to characterize what you're actually saying.

It also doesn't say anything about the appropriate level at which such abstraction should be defined - e.g. if the concern is interoperability, then (as you observe in the essay), modules that provide something like COM-style exports fit the bill without necessarily requiring an internal language that resembles an OO language in any traditional sense.

However, someone seeing the general term "objects" used to describe this could be forgiven for thinking that this implies that some traditionally OO language should be the preferred vehicle for this "inevitable" technology, when that may not be the case at all.

This perception is reinforced because the essay is also explicitly defending "object-oriented programming" and "object-orientation", which terms generally imply much more than the restricted definition of objects that you're using.

Here's a representative quote from Section 5 that suffers from these issues:

This interoperability cannot be duplicated in other programming paradigms without essentially simulating objects.

The problem with this is that OO "paradigms" tend to involve much more than just the definition and use of objects in the sense you've defined, so it's misleading to use your conclusion to justify those paradigms, as you seem to be doing here.

The flip side of all this is that these obvious issues give those opposed to objects an excuse to dismiss what you're saying.

You'll note that these criticisms are not aimed at your core technical claims, which actually sound fairly reasonable to me although I haven't properly evaluated them yet. I do think it's important to identify and examine features, like interoperability mechanisms, that are critical to the success of production systems.

But the value I would hope to get out of such work is to find ways of moving beyond the ad-hoc systems that plague industry, towards languages and systems with a more rationally constructed and justified foundation. It would be a pity if instead the work was merely used to justify the very mistakes we should be trying to correct.

(It has also not escaped me that framing your contribution in the context of a never-ending flamewar du jour will certainly help stimulate discussion, as this thread demonstrates. But to help advance the state of the art may require a more careful separation of the technical concepts from the socially-based fluff that surrounds them.)

It also doesn't say anything

It also doesn't say anything about the appropriate level at which such abstraction should be defined - e.g. if the concern is interoperability, then (as you observe in the essay), modules that provide something like COM-style exports fit the bill without necessarily requiring an internal language that resembles an OO language in any traditional sense.

What does traditional sense mean here? Jonathan's argument is that you are essentially recreating objects, whether your internal language directly supports that or you have to do some extra work to get there.

The flip side of all this is that these obvious issues give those opposed to objects an excuse to dismiss what you're saying.

Audience is important. Such an essay written with respect to an ICFP rather than OOPSLA audience would be very interesting, though I think the for/against camps essentially come down to ideology and reasonable arguments are not effective either way.

It would be a pity if instead the work was merely used to justify the very mistakes we should be trying to correct.

I'm not sure the essay is accessible enough (e.g. it is not a B. Victor essay made for mass consumption) to reach such an audience that would jump to such conclusions.

Objets sont mortes, vive les objets

What does traditional sense mean here?

Things like objects as the core abstraction mechanism, methods as the only kind of procedural abstraction, pervasive mutable state, a rather specific model of class-based implementation inheritance, etc.

Jonathan's argument is that you are essentially recreating objects, whether your internal language directly supports that or you have to do some extra work to get there.

Neither recreating nor extra work is necessarily implied. A language could have a facility for producing Cook-definition objects without necessarily having all of the baggage mentioned above as core features that can't be avoided.

The Clojure example discussed elsewhere in the thread could be considered an example along these lines. It's all very well to say that something like Clojure is object-oriented in the Cook/Aldrich sense, but in that case the definition of object-oriented is being radically expanded as a result of constraining what "object" means. This seems unhelpful to me.

I think the for/against camps essentially come down to ideology and reasonable arguments are not effective either way.
...
I'm not sure the essay is accessible enough (e.g. it is not a B. Victor essay made for mass consumption) to reach such an audience that would jump to such conclusions

The essay itself seems a bit ideological to me, but I don't think it needs to be to get the technical message across. It also seems to be equivocating somewhat along the lines I described, so it's not just the reaction of a broader audience that I'm thinking of.

Funny, my only complaint

Funny, my only complaint about the essay is that it's not ideological or provacative enough. To each his own, I guess.

The question I want to answer is a question about objects

Ever since I read Cook's essay, the term "object" has acquired a precise technical meaning in my vocabulary. I recognize that not everyone agrees with Cook. But the term "object" ought to mean something, and Cook's definition is arguably the best candidate out there.

Should I shy away from the term object in my essay just because some will misunderstand it when they read my title (and no further)? The definition I use is not hard to find in my essay--even the abstract has a pointer to the definition. I am interested in the question of why object-oriented programming has been successful. I do not pretend to have the full answer to that, but I believe my essay gives an important part of the answer. The answer in my essay does not rely on all the characteristics found in OO languages, but it does rely on the most fundamental of those characteristics.

What term would you use instead of "object"? Reynolds's term, "procedural data abstraction," is less ambiguous, but it is in a sense even more misleading in my context, because my argument relies critically on objects being used to abstract things other than data. I thought of using the term "service abstraction," following Kay's description of objects as servers offering services to their clients through a message-based interface (see The Early History of Smalltalk). But this may be misleading too, in the world of service-oriented architectures.

As for defending object-oriented programming--well, it is true that most object-oriented programming languages includes more features then my definition of objects. But it is also true that non-OO languages do not provide good support for my definition of objects--it can be done with a record of functions, but I know few OO people who would be satisfied with that. In that sense, I think my defense is on target.

In the end, I stuck with object, because it is the success of objects that I want to explain, and because I hope that my essay will help move the world towards a more rigorous, technical definition of the term object (following Cook's lead here).

Ought is fraught

But the term "object" ought to mean something, and Cook's definition is arguably the best candidate out there.

The term already means many things. Do you really want to base your message on an attempt to squeeze toothpaste back into the tube?

Should I shy away from the term object

I think it's important to distinguish carefully between your/Cook's restricted definition of "object", and the various and ubiquitous broader definitions, to avoid unintentional equivocation that may even affect your own thinking on the subject. The easiest way to do that may indeed be to use some other term.

I am interested in the question of why object-oriented programming has been successful. I do not pretend to have the full answer to that, but I believe my essay gives an important part of the answer.
...
But it is also true that non-OO languages do not provide good support for my definition of objects--it can be done with a record of functions, but I know few OO people who would be satisfied with that. In that sense, I think my defense is on target.

One important distinction that I want to preserve is that even if the features of your definition of "object" were an important part of the success of OO languages, and even if current alternatives do a poor job of satisfying this requirement, the kind of results that I would find useful would help us to move forward to better languages. Achieving that may very well entail rethinking much of what is generally accepted to constitute object-orientation and object-oriented programming today.

It's not clear to me from your essay whether you would agree with that, and it could easily be interpreted the other way, as a defense of the full OO status quo.

I thought of using the term "service abstraction," following Kay's description of objects as servers offering services to their clients through a message-based interface (see The Early History of Smalltalk). But this may be misleading too, in the world of service-oriented architectures.

"Service abstraction" seems less encumbered, at least. Perhaps you could qualify "service" in some way to distinguish from services that don't qualify as objects in your sense. You could also consider qualifying "object" somehow, e.g. "service object".

I hope that my essay will help move the world towards a more rigorous, technical definition of the term object (following Cook's lead here).

Is that an important or useful goal? More to the point, is it achievable?

A new revision uses "service abstraction"

Anton,

Thanks again for the constructive criticism. I have tentatively decided I like the idea of using a different term to denote exactly the element of objects that I am studying, and I revised the paper to use service abstraction for that (the new version is now at the original URL; you can find the old version as objects-essay-v1.pf). I'm still trying to make some general conclusions about objects, but I'm trying make the case through claims about service abstractions and then discuss the consequences for objects.

I'm not certain this will stay--I need to check with the Onward! PC to make sure they don't mind the change--but if it does I hope it will make the case more clearly and steer away from irrelevant objections.

i read it after the revision

and it all made plenty of sense to me. :-)

I think the argument based

I think the argument based on how language shapes thought. To me, that we see objects as animate beings is key to understanding why objects are so accessible compared other core constructs of other paradigms. An object has behavior, it has an identity, it has encapsulated state, which are properties we attribute to physical things.

But the technical argument in this essay is very clear and easily testable. Are you claiming that no large frameworks easily exist without objects? In that sense, wouldn't a study of the Haskell ecosystem be useful in figuring what the alternative is and measuring its complexity somehow?

A small note for camera copy, but I think the DLL Hell comment might merit a Szyperski cite.

Can large frameworks exist without objects?

I'm not sure I can claim that no large frameworks would exist without objects. I'm trying to limit the number of absolute claims in my essay, which already has a few ;-). FoxNet is an (admittedly small) counterexample, as I discuss. There may be a few large, but highly static, frameworks which don't need objects. However, I am arguing that most frameworks rely critically on plugin interoperability, and therefore do require objects or their equivalent. That claim ought to in principle be amenable to empirical validation.

Regarding language shaping thought--surely it does, to some extent! I think it is likely that psychological effects have played a role in the success of objects. However, such effects are out of of scope in my essay, both because I don't consider myself qualified to make that argument, and to keep the essay more focused.

Thanks for the Szyperski pointer--"Greetings from DLL Hell" I assume.

It would be very interesting

It would be very interesting to analyze large frameworks and see (a) what they have in common and (b) if that commonness resembles objects. I would think that (b) might actually be true, but I have little Haskell experience to know even where to start looking. Unfortunately, confirmation bias is a huge danger here, and even if one does the study objectively, or even empirically, they'll be accused of bias by those unhappy with the results anyways. There are also problems with programmer bias, in that, even though they might be using Haskell, they could be tainted by OO experience in the design of the framework.

At the end of the day, I feel like we are losing our field. "Objects may have won" but the moral high-ground has gone to the FP community who constantly deride OOP as being the wrong, attacking a Java straw men to prove their point. There is demand for a stronger more provocative essay on the topic.

Rich Hickey's take on OO

http://codequarterly.com/2011/rich-hickey/

I especially like:

When we drop down to the algorithm level, I think OO can seriously thwart reuse. In particular, the use of objects to represent simple informational data is almost criminal in its generation of per-piece-of-information micro-languages, i.e. the class methods, versus far more powerful, declarative, and generic methods like relational algebra. Inventing a class with its own interface to hold a piece of information is like inventing a new language to write every short story. This is anti-reuse, and, I think, results in an explosion of code in typical OO applications. Clojure eschews this and instead advocates a simple associative model for information. With it, one can write algorithms that can be reused across information types.

And yet, Rich Hickey

And yet, Rich Hickey supports and promotes objects in Clojure:

Clojure models its data structures as immutable objects represented by interfaces, and otherwise does not offer its own class system.

There is nothing in Jonathan's essay that goes against what Hickey says, where his attack is primarily on Java classes and not objects in general.

What does this quote have to do with the topic's paper?

If you like it so much, print it out and stick it on the fridge... This discussion shouldn't be focused on what we like, but how we can apply abstraction to build better software. How would you build a better Widget API using Clojure's "simple associative model for information"? Also, would you apply Rich Hickey's comments that "objects thwart reuse" to the Widget API given in the paper?

While I dislike Jonathan's essay (probably because I am not the target audience), and may one day respond to it with constructive criticism, he at least carefully defined his argument. It is simply noise to juxtaposition a paragraph in an interview with an essay, because Rich doesn't define what a "typical OO application" is or examples of "explosion of code". Also, he uses the phrase "anti-reuse" ambiguously, as if there is only one kind of re-use.

That said, Jonathan's paper basically does explicitly define the re-use that comes from "interoperability" and "uniform treatment": The Open-Closed Principle. But this is also a weakness in his paper, since people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques. See footnote 13 on page 5:

The definition of modular extension captures Meyer’s open-closed principle [20], which states that abstractions “should be open for extension, but closed for modification.” It is particularly important when an abstraction is designed by one organization, but extended in an unanticipated way by another; in this case, the second organization may not be able to modify the original abstraction. 13

________________________
13 Such modifications may be literally impossible if the abstraction is distributed in binary form; or they may merely be infeasible in practice because the second organization wishes to avoid modifying (and then maintaining) a component it did not develop.

As an aside, I do not like this section of the paper. I hate the idea that designers can't anticipate how others will use their abstractions. It's not correct. Designers learn how problem domain experts think, and try to predict as many different variations as possible, and then try to create a stable interface that can satisfy all those variations. In other words, the author doesn't clearly explain what is unanticipated, and gives the sense that ghosts can emerge from the machine. How goals will be achieved is what is unanticipated.

At the same time, I am not convinced Widget goals need to be encapsulated in the Widget objects. For the Widget API, an alternative model is emerging, called Responsive Design, which essentially uses logic programming.

Rather than have a StackPanel Widget container which can only achieve its goal of laying out its children in a fixed way, HTML5 web developers are essentially using logic programs to query the system (browser media) and apply different layouts and sizes based on the media query results. In fact, not only does the position of a Widget change, but its display contents as well. The paint() goal is completely removed, such that the logic program instead computes a scene graph. Procedural abstraction is only used to hot-swap out logic programs, usually using $.ready. For example, applying a Google Wave-like ShortScroll bar.

Why, then, does Jonathan say the paint() goal is a good example of using objects to achieve high-level goals? The assumption is that a Layout Manager should automatically resolve the size of every Widget. To do this, the problem is decomposed such that every Widget reports how much size it would like to have, and also how much size it actually took.

What's also funny is that Jonathan thinks his Widget API provides two services, when most real-world developers I know would think of it as a single service:

the abstraction captures the negotiation between a UI element and its container concerning how large the UI element should be, as well as the goal of painting itself on a display device

.

...and that the paint() goal should be split into two separate scene graph services / sub-goals: measure and arrange. Then rendering can be encapsulated in a separate component which can make smarter decisions and optimizations, including computing efficient "damage repair" and "hit detection" when composing scene graphs. Most common "layout Widgets" implement constraint-based layout algorithms. A common variant is called linear constraints, where Widget positions are expressed as a system of linear equations. Fixed, relative, stacked and grid layouts are all specializations of linear constraints. Ergo, rather than make "open" a layout Panel which measures and arranges its children, it would be more general to simply expose a linear constraint solver. For layouts that cannot be expressed in terms of linear constraints, you simply glue the two together. And it is easy enough to name constraints like StackWithVerticalOrientation, StackWithHorizontalOrientation... or vbox and hbox for short.

Also, I would argue the Smalltalk-80 GUI API was not a huge success relative to the success of bitblt, which enabled easy portability. For example, the way Smalltalk-80 decided which Controller "seized" control of the event loop and could intercept events was insecure and not modular, in my humble opinion. Many GUI libraries, like Swing, also have perpetual inconsistent event handling, because of poor event loop handling. In turn, this caused IBM to build SWT as a Swing replacement, simply to fix problems caused by developer's inability to do interoperable event handling!!! How hard is it to program OnMouseEnter and OnMouseLeave such that there is a consistent ordering to those events?

Puff, puff, puff. I am out of steam for now.

'unanticipated' and other quibbles

Hi Z-Bo,

Thanks for your feedback. I'm sorry you didn't like the essay, but perhaps I can make some improvements to it that you and others would appreciate.

Regarding the term 'unanticipated,' I use it in what I think is the usual sense, but perhaps I should clarify:

By *unanticipated extension* I mean an extension for which neither the extension's code, nor the particular feature(s) being added by the extension, were explicitly anticipated by the designer of the abstraction being extended. Of course, in order to design an extension point at all, the designer must have some idea of the class of features the extension point is intended to support, but it is typically impractical to enumerate all features that fall into that class.

I think I will add a paragraph like the above to the essay; does it clarify my meaning for you?

Also, can I ask what you mean by saying people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques?

Regarding Widgets and Smalltalk--my goal is to talk about the history and what is in widespread use, not to claim that the designs I discuss are the best. My Widget example is taken more or less directly from Apache Pivot, but is very similar to what you see in Swing or SWT. I like the idea of specifying constraints via logic programming, but I think to explore this would be a diversion in the essay. Smalltalk-80 is not the most advanced GUI framework, but it was the first, I believe; although it was not commercially successful, it had a huge influence on later frameworks that were. But--any suggestions you may have about how to get the central points of the essay across more effectively would be appreciated.

Not quibbles

There are not slight objections.

any suggestions you may have about how to get the central points of the essay across more effectively would be appreciated.

First, regarding logic programming GUI framework versus an object-oriented GUI framework. There is a reason WHY object-oriented GUI frameworks have been successful, and you completely missed it in your paper. It is exactly because of interoperability and extensibility, but the example you give doesn't illustrate that. Think of the evolution of Microsoft Office, which typically had many plug-ins written in Visual Basic and not Win32. These plug-ins used COM (component object model) OLE (object linking and embedding). Typically a Visual Basic plug-in would even use a different API to paint to the screen. Also, since Office plug-ins do more than just paint, and also need to implement a Caretaker pattern to serialize their state when Office tells it that it wants its memory back, this is a pretty fancy example using the most widely deployed application in history.

There is a pretty clear reason why a logic programming approach would not work here. I explicitly say the renderer is taken over by the system in this scenario, which precludes delegating to different drawing APIs. This ability to glue different models together allows people to create different approaches to solving problems in the same domain. (As a philosophical point not worth putting in the paper, I believe this has significant evolutionary meaning, as groups need the ability to diverge and go down separate paths before converging on an optimal solution.)

can I ask what you mean by saying people confusingly apply the Open-Closed Principle to objects and ADTs in different ways due to being complementary abstraction techniques?

In terms of language mechanisms for open recursion, such as super and inner, How many ways can you name that OO programmers use OCP? Likewise, how many ways can you name that "ADT programmers" use OCP? I can list it for you, but I would want to create a fairly exhaustive list, and I can't do that for you right now. But I just know objects are far more complicated when applying OCP.

By *unanticipated extension* I mean an extension for which neither the extension's code, nor the particular feature(s) being added by the extension, were explicitly anticipated by the designer of the abstraction being extended. Of course, in order to design an extension point at all, the designer must have some idea of the class of features the extension point is intended to support, but it is typically impractical to enumerate all features that fall into that class.
I think I will add a paragraph like the above to the essay; does it clarify my meaning for you?

It just adds words! I already said what you should say. How goals are achieved cannot be anticipated. That's pithy. Maybe you had something else in mind, though.

[Edit: I suppose a totally unanticipated usage of Visual Basic would be third party vendors using VBX and then OCX to create non-visual libraries for things like file compression.]

You're right, these weren't

You're right, these weren't quibbles.

Thanks for the Microsoft Office example, it is a great illustration!

Regarding the Open-Closed Principle, you're definitely right that its application to objects is complicated--but I am just looking at the dynamic dispatch nature of objects (inheritance is out of scope), so I think the issues are simpler in that case. Either you use an abstraction, or you reimplement it.

I did have slightly more in mind than "how goals are achieved" with respect to anticipation--often the designer of a framework knows the kind of goals to be supported, but has not enumerated all the possible specific goals that plugins might have. But in any case, you were right that a clarification was needed.

a thing by any other name still smells like ... what?

I often use object as a term barely different from thing when speaking about code, in any language. I have a coworker who says, "It's not an object, because we're using C, and nothing is object oriented."

So I remind him we can write object-oriented code in any language. All that differs is whether a language provides special-purpose support. "But there's no vtable and this case is not polymorphic," he objects. (Yes, using object as a verb there is unintentionial pun.)

I explain my minimum requirements of an object: associated state and behavior. I point at a C struct definition and say there's the state. Then I point at the functions used to manipulate this struct and say there's the behavior. When I need more than one version, I can take the original concrete version and abstract it with a vtable, so polymorphism can be added as soon as I need it. Just because it hasn't yet abstracted itself via abstract interface doesn't mean it's not a concrete object already.

During the 90's a lot of folks discussed what was meant by "object orientation", with a laundry list of necessary features, leading to various typologies describing what you get with some or all the qualities. Frankly, I never cared. Since then, apparently folks dropped the trailing orientation from the phrase, now saying object alone implies the other word too. This is ambiguous in so far as now everyone has to ask, "What kind of object do you mean?" Clearly plain English usage as synonym with thing must be permitted in polite company, absent ideology nazis with an agenda to enforce.

When I had math classes in college, many things tended to have definitions that sounded like this: a Foo is a N-tuple <X,Y,Z,...> with the following operations defined, etc. Here the tuple is state and the associated operations are behavior: a mathematical object. It's not a complex idea, and has nothing to do with specific issues of implementation or tactics of code development and maintenance.

I have some familiarity with Smalltalk. I think Alan Kay's objective (yes, punning again) was to characterize the model a coder thinks about when manipulating Smalltalk entities, as they are presented to folks using one or another user interface. As a higher level language, Smalltalk did not afford to coders easy access to a mental model involving bytes and movement of bytes, like C does. Instead, the natural unit of stateful organization was a simulated higher level entity called an object. I can't bear the idea of reprising all the back story about domain specific nature of objects, though, so just pretend I made an effort here to explain the notion of domain as context for defining objects in a model.

Anyway, I'm arguing that enumerating entities and things they do (according the model simulated by each one) is the main point of objects as an organizing principle, and not anything about how they get implemented specifically using technology at hand. In this sense, object is just a less fancy term than entity and less likely to make a grade schooler screw up their face and say, "What?" when they first hear the term. In other words, object means thing with an emphasis on affordances in interface for manipulating one.

In his paper's abstract, Jonathon Aldritch says he wants to answer Cook's question:

If procedural data abstraction is the essence of objects, has it contributed to the empirical success of objects, and if so, how?

After a speed skim to be sure, that's not something I want to read about, so I can't respond to Aldritch's paper without having read it. However, it does seem like it's possible to obsess about the significance of procedural in a way that distorts Kay's original idea.

Service Abstractions, yes. Interoperable, no.

The essay seems to be in flux. The second paragraph of the abstract now reads:

This essay attempts to answer that question. After reviewing Cook’s definition, I propose the term service abstractions to capture the essential nature of objects. This terminology emphasizes, following Kay, that objects are not primarily about representing and manipulating data, but are more about providing services in support of higher-level goals.

The phrase 'service abstraction' does capture what I consider to be the essence of objects. I'm happy with that.

Though, objects aren't useful for abstracting all kinds of services. In particular, objects seem to be remarkably poor at abstracting the services we call 'frameworks'. (Or put another way: we use the word 'framework' to describe a useful class of services that are difficult to abstract procedurally.)

Unfortunately, the essay goes on to discuss extensibility and interoperability as though proposing that OOP does them well:

Using examples taken from object-oriented frameworks, I illustrate the unique design leverage that service abstractions provide: the ability to define abstractions that can be extended, and whose extensions are interoperable in a first-class way.

An 'ability to define abstractions' does not imply that this ability exists on a path of low resistance. Developers need careful design and discipline to exercise this ability. It is quite challenging.

First, extension: Object layer extension is achieved by strategy patterns, observer patterns, context objects, factory patterns, and similar. These require some foresight and design. In that respect, it is not clear that objects are much better than many other not-readily-extensible models that might use scripts or functions.

[Should we be considering class-layer extension mechanisms such as implementation inheritance? They're a traditional part of OO, but arguably not part of 'object'. Hmm.]

Second, interop: the essay seems to assume the consequent, by focusing on how easily interop works AFTER you have a standard set of interoperable service abstractions and everything is an instance of them. But the real challenge for interop is getting to that point. One painful reality is that we have too many standard interfaces, and the 'not invented here' syndrome often means that every independent developer tries to create a new one. Each of these interfaces involves multiple aspects or models - for data, for updates, for extension. These models will be incompatible, sometimes subtly, and create much 'impedance' for interop. Attempting to treat different service abstractions uniformly tends to limit developers to the common subset of features, whereas 'direct' interop involves problem-specific glue code that increases system complexity. Object interfaces tend to focus on incremental updates, but there is no generic function to translate update events on one model to update events on another (i.e. no function of type `(a -> b) -> (a -> a) -> (b -> b)`) so developers must specialize these every time. Further, the different models will tend to group data and updates in different schema, thus requiring ad-hoc joins and splits, which requires keeping a lot of information around. Between those forces, OO developers working at interop spend much of their time and code gathering, scattering, and managing data instead of expressing useful logic for the interop. Worse, specialized databases to support these operations are encapsulated within the interop objects, forming ample breeding grounds for state bugs, maintenance errors, and inconsistencies.

Jonathon Aldrich continues on to say "the success of objects was not a coincidence: it was an inevitable consequence of their service abstraction nature", which (following the phrase "unique design leverage") seems to be implying that objects are the only kind of service abstraction. But that would be wrong.

Objects are not the only way to abstract services, not even way back in 1970. And some other service abstractions are much better with respect to extension, interop, and even reuse.

For example, consider publish-subscribe systems, blackboard metaphor, tuple spaces, data-centric networking. Such systems have a common property: the active elements, the agents which provide services, are not directly named, referenced, or communicated with. Rather, communication occurs indirectly through a shared medium. You don't call a janitor to invoke a janitorial service. Instead, you add a nameless janitorial service to the system, then use a common medium (e.g. the internal volume of a trash can) to communicate what should be thrown away.

Those systems are naturally extensible: you can integrate many new services without invasively modifying existing services. Such extensibility is the reason the models were developed in the first place, i.e. for multi-agent systems.

And though the interop and integration challenges remain, they are more readily separated into independently developed, semi-transparent 'translator' agents. And it is easy for multiple services to share a translator rather than replicating the effort per integration. Also, communication tends to be more persistent - representational state instead of incremental updates - so translation is easier (less implicit context to manage and update).

But if these other service abstractions are more effective in many important ways, why are they far less popular than OO? My hypothesis: objects are not the best service abstraction (by a number of metrics), but they were among the easiest service abstractions to represent in context of an older world where procedural programming was king, stovepipe systems were common, and concurrency was irrelevant.

Objects are not inevitable.

The whole robotics industry, for example, seems to be moving away from OOP more towards publish-subscribe (with OMG's DDS or Willow Garage's ROS) to address the relatively severe challenges of ad-hoc hardware configurations. Various pubsub UI services are also gaining some interest, i.e. so we can easily add widgets that are aware (at a semantic level) of what happens to other widgets (e.g. OWF).

Also, while there will always be a role for behavior and data abstraction, perhaps we could favor something much more constrained than `objects` as they exist today. For example, we could favor a more component-oriented design (enabling external composition, preventing entanglement, improving reusability, standardizing composition). We could also forbid local state, shift state to external resources (enabling runtime update, smooth failovers and fallbacks, live programming, orthogonal persistence, ad-hoc extensions due to more accessible state).

I believe we could do much better than objects with respect to both service abstractions and behavior abstractions. That doesn't mean we inevitably will do better. Even if people were to recognize the benefits of alternative models, there are transition costs that most individuals and companies would be hesitant to pay. And then there's just the fallacy of thinking (hoping) that popularity is strongly correlated with technical (or even psychological) merit.