Lambda the Ultimate Object

I am writing a book "Lambda the Ultimate Object". It's quite advanced at the moment, and already covers all the usual topics of OOP, though I'm still working on CLOS features that most other languages are lacking (I did method combinations, now working on multiple dispatch, then dynamic vs static dispatch, then the meta-object protocol). Feedback welcome.

http://fare.tunes.org/files/cs/poof/ltuo.html

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Sorry for duplicates

I clicked the "submit" button, there was no feedback, so I clicked again many times, and it created many copies of the topic. Oops.

"LtU is now running in a new, more stable environment", allegedl

Yeah, despite that claim on the Home page, LtU seems to be not much of an active forum. I don't know how it decides which threads should appear at Home. Your post is (multiply) in 'Recent Posts' and 'Discussions'.

"LtU is now running in a new, more stable environment", allegedl

Yeah, despite that claim on the Home page, LtU seems to be not much of an active forum. I don't know how it decides which threads should appear at Home. Your post is (multiply) in 'Recent Posts' and 'Discussions'.

Edit: and Haha when I submitted that comment, I got a Server Error

Multiple Dispatch? Return Polymorphism?

It took a while to really wrap my head around the idea that the data elements of stateful objects can be viewed simply as the set of local variables bound in the context of the object allocator, and the corresponding "objects" simply as continuations that jump back into the context of the allocator at the entry point of an allocator-local dispatch table. If each continuation carries with it its own set of values of the mutable bound arguments, then each continuation-considered-as-method has its own set of bound-arguments-considered-as-member-variables. So lambda plus continuations really is the foundation of object-oriented programming with member variables.

CLOS is a good system and supports multiple dispatch (ie, the type of the first argument is not the only type taken into consideration in choosing the correct method).

I think that the value of multiple dispatch is routinely underestimated because it makes method dispatch a property of sets of types rather than a property of individual types. Other people think that the value of multiple dispatch is routinely overestimated (by weirdos like me) because the added expressiveness comes at the cost of mental shortcuts and syntactic sugar that people use to organize and understand their programs in terms of the simpler rules that describe single dispatch. The debate is a valid discussion and neither POV is necessarily right or wrong.

You don't seem to be covering the "under the hood optimizations" aspects of creating object systems, but I got best performance results for multiple dispatch by calculating a "type signature hash" of the argument types and then using method name + type signature hash as keys to a global dispatch table. But all of that was implemented at C-level and may be inappropriate to address in a higher-level guide to type systems.

Expressiveness also benefits from return polymorphism. Return polymorphism is the idea that the return type as well as the argument types is visible or inferrable at the call site, and can therefore be given the same meaning as any argument type in determining what method to call. It is basically free once multiple dispatch is implemented, but is not a feature of most languages that have multiple dispatch.

There is again some disagreement about its value. Return polymorphism violates the conditions under which some classes of proofs about program correctness are valid. I am convinced that equivalent proofs can be constructed without relying on those conditions but AFAIK that question has not been formally addressed.

§1.3.1 OO is the Paradigm of Programming with Inheritance

I readily admit to not understanding much about OOP, and being willing to listen. I've read through to the end of §3, and I'm much more confused than when I started. So I've stopped. I'm going to wash the whole doco out of my brain as soon as I've written this feedback.

Increasingly, the doco reads like a rant ("Things OO isn’t (that many claim it is)", "Is my definition correct?"), not an attempt to survey the subject matter. Now I'm all for a good rant, but pages and pages of it is just exhausting, and ultimately uninformative unless the reader already knows the subject thoroughly.

The trouble I think is that 'Object' is such an abstract/general-purpose term as to be useless. The same goes for 'Class', 'Inheritance', 'Encapsulation'/'Modularity'.

You might say the same for 'Functional' as in FP -- don't all programming languages support defining and using functions? But FP is grounded in Lambda calculus and specifically Beta-reduction.

The Imperative model is similarly grounded in the vonNeumann/Turing machine abstraction. This comment is worth quoting

I’m troubled by the fact we don’t have a mathematical model for OOP. We have Turing machines for imperative (procedural) programming, lambda-calculus for functional programming and even pi-calculus (and CSP by C.A.R. Hoare again and other variations) for event-based and distributed programming, but nothing for OOP. So the question of “what is a ‘correct’ OO program?”, cannot even be defined; (much less, the answer to that question.)

Perhaps OOP is not one thing, but rather a mish-mash of different things, as many as there are programming languages claiming to be OOP. One thing I am sure of is: OOP is not FP, and LISP/Scheme is a terrible language to demonstrate any programming concept. (Looking at your "Prototypes:" paper.) LISP is dynamically-typed, interpreted, with LAMBDA but also with PROG and GO and SETQ, and with Macros that are indistinguishable from proper code. It was brilliant as a first stab at functional programming in 1966. It's so all-over-the-place as at today that using it for examples proves nothing. (In the same way you can write Fortran in any language.)

Your reducing it all to Inheritance I find a complete non-starter:

the actual central concept in Object Orientation is Inheritance, a mechanism for programming by modularly extending partial specifications of code.

Talking about "code" is vacuous. "code" representing what? Types? Data structures? Classes? (And what definition of Class?) Methods? Interfaces? Messaging?

Haskell exhibits Inheritance of TypeClasses (which are not Classes in the OOP sense -- btw your note 89 is just wrong). All Monads are Applicatives; all Applicatives are Functors. Haskell's model of polymorphism makes subtyping next-to impossible, then Haskell is not OOP. Then Inheritance is not sufficient to define OOP. (There were attempts at O'Haskell that have fallen by the wayside, there is O'CaML -- I broadly agree with your assessment of Garrigue it's an "unhappy" model for OOP.)

I'll add that Inheritance of the Is-a form strikes me as just inadequate for modelling the real world. For some purposes (like biology Phylum/.../Genus/Species) it's brilliant (because there is genuine inheritances of genomes). But for many purposes it isn't -- even the classic figures of the plane:

* A square is-a rectangle is-a quadrilateral.
* A square is-a regular polygon (which rectangles generally aren't).
* A rectangle is-a orientable figure (has property prone vs upright);
as is-a ellipse; as aren't square nor circle.

If you've ever designed a (relational/normalised) database schema for a decent-sized business area, you'd rapidly realise Inheritance is an abstraction of very limited application.

I'm extremely doubtful that Inheritance is the single or even a major defining characteristic of OOP. If it is, that explains why OOP has never made much sense to me.

Inheritance.... sigh.

Inheritance of method implementations is not the essence of object oriented programming. You could make a pretty good case for inheritance of method interfaces being important, but data structure layout in memory, or having methods on related types share the same source code, doesn't matter.

It's all about providing interfaces. If you have a 'subtype' relationship it usually makes sense for the subtype to mostly have the same set of interfaces as the parent type. But not always and not always all of them, and whether the interfaces are provided by the same or inherited code is just a detail.

You want all the things that it's meaningful to count to have the same interface to methods that count them. The interface might be just one method that tells you how many there are, or it might be half a dozen methods to set, increment, decrement, reset, and return the count. But if you're abstracting different kinds of countable things, then it would be nice if the counted things are objects that have a standard interface to methods that do counting things.

You want something to be able to pass an argument in some way that says, "whatever this is, it can be counted, using the same interface that you're using to count things."

Likewise things it's meaningful to print, or read, or do whatever other general procedure with.

It doesn't make any difference at all whether any of them are subsets of any other, nor whether the methods involved use the same source code. Only whether the called procedure gets data that supports the interfaces it uses and whether the calling procedure gets back a result that supports the set of interfaces it needs to continue.

The type information tells you what set of interfaces are defined for a particular datum. Interfaces tell you what methods you can use and what semantics they have.

Of course, this is only one view of Object Oriented programming and what's important about it.

§3.1.4 OO isn’t Encapsulation [modularity/interfaces]

It's all about providing interfaces.

_all_ about? In pretty much every HLL these days, best practice is to build the application from Modules, especially those providing library routines that support common 'business needs'. Modularity means making public some functions/methods -- that is, "providing interfaces"; keeping implementation details private. That's common with OOP, just as common with many languages, so not sufficient to characterise OOP. Rideau says that's at best half the story [**]:

“encapsulation” usually denotes the ability to code against an interface, with code on either side not caring which way the other side implements its part of the interface, not even being able to distinguish between multiple such implementations, even less to look inside at the state of the other module. Viewed broadly, this is indeed what I call modularity, which in my theory is indeed half of the essence of OO. But the word modularity much better identifies the broader purpose, beyond a mere technical property. And even then, modularity only characterizes half of OO, so that people who try to equate OO with that half only crucially miss the other half—extensibility
[and so back to Inheritance. See also Note 7:]
[William] Cook’s definition, that embraces the modular aspect of OO while rejecting its extensible or dynamic aspect, runs contrary to common practice.

[**] But conceding encapsulation/modularity is even as much as half the story is counter §1.3.1 I quoted in my earlier post saying Inheritance is the central concept.

Until there's a clear statement of what's Inherited/extending what/interfaces tofrom what, I see no characterisation that distinguishes from modern HLLs-in-general.

You want all the things that it's meaningful to count to have the same interface to methods that count them.

That sounds like merely overloading/ad-hoc polymorphism. The term is from Strachey 1967, so predates even Smalltalk.

Let me hazard a suggestion: what distinguishes OOP-style Encapsulation/Modularity/Interfacing/Extensibility/Inheritance from FP or Imperative E/M/I/E/I is what's inside the capsule:

keeping related fields and methods together. A field (a.k.a. attribute or property) contains information (a.k.a. state) as a variable.
[quoted from here, and naughty-naughty they don't give a cite.]

AFAIAA, the earliest HLL that implemented encapsulating little bits of State was ALGOL 60 own variables. (But this wasn't yet OOP: there was no new() method to initialise the variable.)

I see Rideau §6.3 Stateful OO considers and rejects Encapsulated State as essential to OO. Then goes on to list all the OOP languages that support it. Then I conclude §6.3 and indeed the whole book is just idiosyncratic to the point of being plain wrong. Rideau is talking about something else, which seems to be following the Humpty Dumpty principle: "When I use a word [OOP] ... it means just what I choose it to mean." No, words/language doesn't work like that.

§3.1.4 OO isn’t Encapsulation [deleted]

(duplicate deleted -- caused by more Server Errors)

Do methods "belong to" a type? If so how do you decide which?

Let's say I have a routine

Foo(Polygon, List) --> (Text)

If this routine must be expressed as a method, then is it

Allocator Text::Foo(Polygon, List)

because it can allocate a new object of type Text and allocation failure is a likely cause of error?

or

Text::Foo(Polygon,List)
because it can change the state of an existing Text object?

or

Polygon::Foo(List)-->(Text)
Because it may change the state of the Polygon? Does it matter that it changes the Polygon's state only part of the time but changes the Text's state all the time?

or

List::Foo(Polygon)-->(Text)
Because it may change the state of the List marginally more often than it changes the state of the Polygon? (Still, neither as often as it changes the state of the Text?)

Polygon::Foo(List)-->(Text)
Because the programmer feels that the value of the polygon is a more important factor in deciding the new value?

or

List::Foo(Polygon)-->(Text)
Because if the list is empty that might cause the method to fail?

or

List::Foo(Polygon)-->(Text)
Because it depends more on other methods of class List than it does on other methods of class Polygon?

or ...

???

Seriously, what are the criteria for deciding?

methods are to a type as wool is to water

“The question is,” said Humpty Dumpty, “which is to be master—that’s all.” — Lewis Carroll [also where the Wool and Water reference is from, since Rideau is fond of quoting]

"belong to" is such a woolly concept as to be no help here. Rideau doesn't use the word except in the negative when summarising Joe Armstrong. "Why OO Sucks" [2000].

1. Functions and data are different, don’t belong together. [Rideau's summary]

Objects bind functions and data structures together in indivisible units. I think this is a fundamental error since functions and data structures belong in totally different worlds. [Armstrong]

The relationship of methods vis-a-vis data (structures, type, subtype, variants) is put this way in The Expression Problem [Wadler 1998]

One can think of cases as rows and functions as columns in a table. In a functional language, the rows are fixed (cases in a datatype declaration) but it is easy to add new columns (functions).
In an object-oriented language, the columns are fixed (methods in a class declaration) but it is easy to add new rows (subclasses).

So neither 'owns' the other: they're orthogonal dimensions.

The answer is also going to depend on which language's semantics you're looking at. (Allocation behaviour is too much in the weeds of implementation details.)

Yah but there's a point to it.

The question about how to decide what "object type" a method belongs to is intended to be Socratic. Or in other words I'm sort of being an ass on purpose and trying to trip people up.

Whatever your answer to the question of how to decide what methods belong to which class, the more you think about that answer the more cases you'll see where it doesn't apply or where the answer is outright self-contradictory. That usually means there is a flawed assumption underlying the question.

And in this case my opinion is the same as yours; the flawed assumption is that there is or should be a definite answer, because in general methods express *relationships between* data, and data can be of many different types. And with that point, about 90% of the current practice of OOP unravels.

There is something that I think OO programming ought to be, but it's about routine interfaces (function names or small sets of function names) being their own types, declared with constraints and invariants and semantic requirements. Routines declared to be of that type (ie having one of the reserved names of some interface) would be checked against those constraints and invariants and requirements to determine whether that's a valid declaration or a type error.

So basically it would be a system that guarantees function overloading should have consistent semantics. Declaring "+" as an overload for string catenation for example should provoke a type error because the interface containing "+" has an additional required member "-", the two are required to be inverses of each other, the arguments of "+" are required to be commutative, the operation is not allowed to mutate either argument, and a couple of other things.

But that is addressing something that's completely sideways to every OOP language that exists today and wouldn't even be recognized as being related to OOP according to most definitions. I probably ought to call it something else.

Tilting at windmills

... on purpose and trying to trip people up.

Whatever your answer to the question ...

Who are these "people"? Whose "answer" are you imagining? None of your posts have cited anybody saying what you claim is self-contradictory/I can't see you've even read some of Rideau's text. And yet you're condemning "90% of the current practice of OOP". Quite possibly you're just making up imaginary positions. This is a caricature of "Socratic" method. Turning to your muddled words of what "ought to be":

it's about routine interfaces (function names or small sets of function names) being their own types, declared with constraints and invariants and semantic requirements. Routines declared to be of that type (ie having one of the reserved names of some interface) would be checked against those constraints and invariants and requirements to determine whether that's a valid declaration or a type error.

Well, is there any language or paradigm exhibiting those features? Or perhaps parts of those features appearing in different languages?

* "functions ... being their own types" seems to be a category error: functions/methods _have_ a type.
* That type is composed from the types of (multiple) arguments and result; there might be several types involved, which is why it makes no sense to talk of methods 'belonging' to a type or vice-versa.
* "declared with constraints", yes that's also part of the method signature (what distinguishes an overloadable method from a parametric polymorphic function).
* "invariants and semantic requirements" are documented as laws; can't be more than narrative because
* "checked against those ... invariants and requirements" is in general undecidable; and would require checking compliance of all methods used in declaring invariants, which drags the compiler into an infinite regress.
* (See Liquid Haskell for an attempt at this.)
* Liquid Haskell is built on top of the "checked against those constraints" part, which is Haskell's typeclass and instances mechanism. (Two syntactically distinct constructs.)

So Haskell has
* Type declarations (mention no methods);
* Typeclass declarations, give the most general signature for the method(s) they're introducing (mention no Types);
* Instance declarations give the implementation for a Typeclass's methods, at a specific Type.