Lambda the Ultimate

...but it may be just that the Eiffel docs I can lay my Googly fingers on are either two high level and vague or are the Ecma standard which is written in obtuse standardese.

But from what I can gather shared references must be a fairly common outcome of assignment in Eiffel.

a.b := ..., which is a threat to class invariant and an extremely rude violation of encapsulation is syntax error in Eiffel. Right hand reference to a field: ... := a.b is not. Remembering the scope of this discussion, querying a field does not risk the class invariant.

Getter routines are considered as superfluous baggage and not used.

By "shared reference", do you mean aliasing caused by assignment of references? Aliasing is a major problem to correctness proofs, but an encapsulation violation...?

As the definition to encapsulation we this time have protection of the invariant.

...just what exactly _is_ a class invariant? I agree the standard definitions I see don't even mention the word state.

To say it is a boolean expression that must be true at the end of the constructor and beginning of the destructor and at the start and end of every public method is a little bit vague.

What variables can be part of that expression? How complex can it be?

One alternative definition I'm playing with is conceptually hugely simpler... but in practice harder.

If you can identify the valid portion of the state space, ie. if an object instance state ever moves out of the valid portion, then it is defective. Then the class invariant is conceptually simple... "the instance is in the valid portion of it's state space."

This definition pushes the vagueness into
- we can identify the full state space,
- and we can tell which points in that state space are defective,
- and we know what defective is.

But once we have done that the class invariant expression is dead easy.

I don't know anything about E and Caja, I'll look them up. But Java is _definitely_ not the sort of a language I'm looking for. Almost the anti-thesis.

It is all too easy to share internal state between classes in Java and hence have one class screw up the invariant of another.

What variables can be part of that expression? How complex can it be?

In Bertrand Meyer's presentation of class invariants, he says that "The state of an object is defined by all its fields (the values of the class attributes for this particular instance)." and "The invariant may only involve the state. Assertions may use functions, but such functions are an indirect way of referring to the attributes." That seems to imply that the answers to your questions are "Every variable in the object" and "Arbitrarily complex".

At a more formal level, the class invariant is part of the set of axioms that define the class ADT. Those axioms are in turn an implicit way of defining the valid state space (which is likely to be easier than explicitly enumerating the valid state space). Of course, those axioms are rarely formalised when people write classes. Part of the point of Meyer's Design-by-Contract approach was to encourage developers to explicate the ADT axioms (the other part being that you could then check those axioms at runtime to ensure that objects stay within the valid state space).

Note that Meyer's version of "class invariant" is specifically about object state, rather than the more general notion of invariant that David mentions above.

Java is _definitely_ not the sort of a language I'm looking for. Almost the anti-thesis. It is all too easy to share internal state between classes in Java and hence have one class screw up the invariant of another.

You asked only whether there are languages that "can" fully encapsulate state. Java is one of them, though not the best example.

As far as "too easy to share between classes in Java", my impression is the opposite. Encapsulation is important, but I do not believe it should exist at the level of 'classes'.

As far as "too easy to share between classes in Java", my impression is the opposite. Encapsulation is important, but I do not believe it should exist at the level of 'classes'.

I agree, and we don't encapsulate anything at the class-level (in fact, we don't really use 3GL classes except as data+code). We provide extreme hiding of state-process through other means.

For an extreme viewpoint of David and my perspective, read P.M.D. Gray's book The Functional Approach to Data Management: Modeling, Analyzing and Integrating Heterogeneous Data. It is as hardline stance as you will find on the issue. P.M.D. Gray has also written a couple of position papers on why classes prematurely encapsulate logic.

Another look at encapsulation comes from Johan De Kleer and Kenneth Forbes' book Building Problem Solvers, which similarly argues against the use of encapsulation simply to protect the visibility of procedure steps from client classes. They're rule engine folk.

One important thing to note is that Java doesn't support instance encapsulation at all (as requested in the original note), only class encapsulation. Methods of class A can access the internal state of any instance of A, not just "this". Instance instantiation is supported in Scala, but I haven't seen it used much in the wild.

Methods of class A can access the internal state of any instance of A, not just "this".

That is a non-issue; you're perfectly free to keep your instance of A far away from other instances of A.

...at one end we have C, where the ruling ethos is "the compiler shall do nothing to slow the programs of the perfect programmer. The perfect programmer is free to tell the compiler at all stages "I know what I'm doing, just do it".

At the other end, we have huge real world bug lists that show nobody on the planet really knows what they are doing, and we need all the protection we can get.

Thus I'm looking for super-restricted, "bondage&discipline" languages that enforce...um, this is where it gets cloudy and dodgy, something, lets call it X.

We've had B&D languages like Pascal that enforced static type-safety.

We have had functional languages that enforce side-effect free and immutable state.

So now I'm, from the perspective of the real world systems I shepherd, looking for another class of restriction.

I want to be able to...

partition every bit (0 or 1) of state in the system into object instances...(think of them as air-tight boxes)
and then seal those bits (in those boxes) by the very design of the language,
so those bits can _only_ be read or modified by that instance's methods and nothing else.

This may be a strange thing to be doing, it may be the wrong thing to be doing, it is obviously in complete conflict with current industry practice....

...but hey, if you want to be discussing something else, go start your own topic. :-)

Let us in this thread, just grapple with the intellectual exercise, the thought experiment, of whether any language implements full encapsulation of state, or is close to implementing this.

...the more I look at the problems of huge real world Ruby and embedded C programs that I deal with day to day....

...the more I'm convince that full enforce encapsulation of state may become a unifying principle behind all OOP Design Principles.

ie. Give me your 2a... and I suspect almost every OOP Design Principle I can shake a stick at falls out either as a consequence, or as "the only way to do it then".

I'm afraid I don't understand what you are saying, nor what you actually mean by "fully enforced encapsulation of state" or how it relates to 2a.

Let me just point out that encapsulation is by no means specific to OO. In fact, OO is rather inflexible about it because it only allows encapsulating one "concept" at a time.

From your link "Instance variables ("properties" in MOO) are all private and not externally visible. Methods ("verbs" in MOO) must be written to expose them."

To enforce invariant safety methods must not expose properties. The alternatives are in from worse to best
- clone the properties,
- pass object ownership out of that class. ie. Object no longer has that property.
- ask the object to perform the query and action for you.

Exposing the property permits outside classes to modify them without reference to the invariant.

Exposing the property permits outside classes to modify them without reference to the invariant.

This may be the desired behavior, in some cases.

Again, interfaces belong to clients, not class hierarchies.

If client classes are allowed access to properties via intefaces, then it is the choice of the class designer to let client users modify the properties.

An invariant is not behavior. A contract is not behavior. An assertion is behavior. A contract simply states a client/attorney idiom, and watchdogging and enforcement of that contract can occur via mutually exclusive third parties. In this way, the watdchdog is orthogonal to the handcuffing agent, and those two are both orthogonal to the contract itself. You may choose to physically couple these agents together, for various reasons.

An invariant simply "is".

"Exposing the property permits outside classes to modify them without reference to the invariant."

This may be the desired behavior, in some cases.

Yip, in some cases it is the desired behaviour in current programs... except if you are trying to answer my question.

In which case the whole point of the question is I'm looking for languages where such behaviour is explicit banned, or strongly discouraged, or even better, language features make it easy to make such behaviour utterly impossible.

It may be a weird question. It may have nothing to do with current notions of Good Programming practice... but, Sigh! It is the question that plagues my mind at the moment.

I agree that invariants may simply "be"... but I have found a bug whenever I have found one smashed.

I think you misunderstand what I meant by 'expose.' Exposure in this sense does not give the caller the right to mutate the underlying property. Exposure here is just a 'getter' or 'setter' in the JavaBeans sense of the word, though that wasn't the formalism there. Further, in ColdC (and CoolMUD, a predecessor) all values (primitives) were pass-by-value -- except for most object references, which were pass-by-ref and those objects in turn were heavily encapsulated. So there was no opportunity to mutate the state of an object without 'asking its permission' to do so.

Some later ColdC users added capabilities-type security on top of the system.

except for most object references, which were pass-by-ref and those objects in turn were heavily encapsulated

Do you mean Encapsulated or Immutable? Encapsulated doesn't protect the invariant of the owning class.

Invariants are about maintaining relationships between instance variables. Suppose object instance A has instance variables B and C.

If you expose object B via anything other than deep copy... you can invoke mutating methods on that object. ie. The original object A is still holding on to the same object instances B and C as it's object variables, but one of those instances B has been mutated without regard to its relationship with C. Invariant smashed!

Java has a Security Manager. Packages can disallow reflection through the security manager, and even environments can disable reflection (such as application containers). A Security Manager is also what E uses to help ensure its conceptual integrity, but it is sort of in the backdrop out of your way.

.NET Reflection can also be disabled simply by not using any libraries that depend on it, and compiling your .NET binary into a regular platform-specific binary. Mono provides this feature for iPhone developers (iPhone disallows interpreters).

Mono provides this feature for iPhone developers (iPhone disallows interpreters).

I believe it disallows JIT, not interpreters. It would be very hard to prevent interpretation. :-)

"memory-safe" is not in any way sufficient.

Getters and Setters are not sufficient

In fact, they tend to be the problem.

On one side getters tend to expose objects of internal state... which may then be mutated. The object instance whose getter was called still holds the same reference to the same object, but it has mutated without refernce to its relationship with other instance varaibles.

On the other side, if the client that invokes the setter still holds a reference to the object it passed to the setter, it again can mutate it out side of the control of the class that now holds it as an instance variable.

Memory safety and absence of reflection is indeed necessary and sufficient to write a class that respects an invariant. That's not to say that any code you write is guaranteed to enforce that invariant, but that wasn't phrased in the original question. Guaranteeing that an invariant is respected is a very hard problem. You can go a long way by just adding immutability to the above two criteria.

I have seen the odd bit of Kirwan going by, and what he says is indeed interesting.

I came to this search for full encapsulation via trying to understand LoD more.

At first the more I looked at real code, and the more I used LoD the more I felt that LoD was indeed a Law, not a guideline.... but I felt it was too obscure and hadn't quite grasped the heart of the matter.

With full encapsulation of state.. LoD falls out as a natural side effect.

I came to this search for full encapsulation via trying to understand LoD more.

LoD has a lot of research papers on it. Karl Lieberherr wrote a full length book called Adaptive Object-Oriented Software he used for some graduate level courses in a course something to the effect of "advanced topics in object technology". It includes a sample Prolog interpretter written in Karl's DemeterTools-based C++ variant. Also, he has a pretty long reference list in the back of the book. You could spend half a year just reading the stuff on the references.

In general, the Adaptive Objects paradigm is interesting, although I dislike the methodology turned out by The Demeter Tools project at NEU. It simply isn't good design, unless you really have a scenario where you can't stabilize the problem domain. In which case, you are likely in trouble no matter what.

I mainly think Demeter Tools itself is a cool way to configure stuff. Some .NET Inversion of Control containers provide the sort of propagation patterns Karl espoused. Also, Demeter Method was one of the first OOPSLA things to push hard for convention over configuration, which everyone sees as a big duh these days.

At first the more I looked at real code, and the more I used LoD the more I felt that LoD was indeed a Law, not a guideline...

Agreed, to LoD being an actual law of sorts. The way I look at it now, is that each class or set of classes that qualify as an abstraction define a language (DSL), and the LoD ensures this DSL does not "leak" into other DSLs in your app, so you are left with a clearly defined semantics at each layer of your program.

By stating that the LoD applies to a "set of classes that qualify as an abstraction" rather than each class, are we still talking about the same LoD? Also, I'm not sure what insights are supposed to come along with thinking of abstractions as DSLs. My rephrasing of this post would be:

Code should be organized into modules, with each referencing only modules at an appropriate level of abstraction for that module.

By stating that the LoD applies to a "set of classes that qualify as an abstraction" rather than each class, are we still talking about the same LoD

Some abstractions necessitate more than one class, such as a set of mutually recursive data types, or at least, some languages require such a structure due to limited expressiveness. The LoD applies to abstractions, not individual classes; the single class abstraction is the degenerate case.

The insight for DSLs is perhaps personal, or maybe biased to those forced to work in non-functional languages where the connection is less obvious, but "libraries are embedded languages". Once one starts to think of the composition of library types and functions as programming in an embedded language, then one tries to program abstractions to be more composable, by introducing fewer ad-hoc restrictions, and limiting the scope of the EDSL to only what is needed in the domain.

The connection to the LoD here, is that every explicit reference to other types in the interface of an abstraction/EDSL increases the scope and complexity of the semantics of the language by transitively including all operations on the newly referenced type. This unnecessarily compromises your ability to analyze programs in this EDSL. LoD implies one should only extend an abstraction/EDSL with the operations needed without transitively pulling in types and operations beyond what is necessary.

My rephrasing would be: code should be organized into programs written in successively higher level EDSLs.

My rephrasing of this post would be:

Code should be organized into modules, with each referencing only modules at an appropriate level of abstraction for that module.

I'm suspect Sandro meant to describe class-instances (objects) or sets of them as forming a DSL accessible from another system. LoD applies to interactions with objects, not with classes. [Edit: I began writing this before Sandro's above response... I must ask why Sandro believes LoD applies to 'abstractions'.]

Characterizing those constructed object-instance-DSLs as 'code organization' modules seems invalid.

It takes some extensive first-class module capabilities, such that modules can be instantiated with dynamic capabilities, data parameters, and other modules, before one could meet the properties of object-instanced DSLs. And, while module systems with that degree of runtime configurability do exist (e.g. in Oz/Mozart), they say very little about the relative modularity of the code constructing them!

Code can be organized into modules, and should be, but object-instanced DSLs are by no means referencing only code organized into other modules.

I've taken a long while ago to distinguishing code modularity from runtime modularity. I'm pretty well convinced that the two should be orthogonal. I'm very dissatisfied with current coupling between code modularity and runtime modularity. 'Module' in common coupled systems include DLLs or shared objects (possibly loaded dynamically) and 'import ' specs where a module is a file with exports.

Demand-driven multicast systems with domain addressing (e.g. overlapping geographic surfaces, topic-sets,), live service registries (including abstract factory plugin architecture design), and live document UIs (RESTful architecture + publish/subscribe; 'live' forms shared by all with capability) offer far greater promise. Properties include runtime modularity, composition, separate compilation (per service), deep optimization, allowance for broadly aspect-oriented code, tolerance to failure via fallbacks, easy bootstrapping, and resistance to DLL-hell is possible via competing services (in a registry) and multi-cast (competing data authorities). Policy-driven architectures can be added readily by allowing the consumer to prioritize competing services and multi-cast authorities via 'soft' heuristic policies (policy injection).

That, to me, is what "module" should mean when talking about object-instanced modules: services, including factories to create services-on-demand, as 'modules'.

Code's organization into modules is orthogonal, or at least should be.

Anytime you consider the correctness of a piece of code, you're taking a static view of that piece of code. Any concept of run-time module is therefore some different meaning of "module" than the one I have in mind.

I agree the original LoD, as I understood it, applied at the object level. I don't particularly think that the OOP approach of forcing an abstraction boundary at any place where you have a difference in multiplicity is a good idea, though, so I wouldn't want to include that in my summary :).

One final point is that, to me, your "demand driven multicast yada yada" is an example of an architecture that should be implementable as library, standardized upon only for reasons analogous to those for standardizing upon a matrix library. I can see it making some sense to call service bundles "modules" in such an architecture, but I see that as a completely different topic.

Anytime you consider the correctness of a piece of code, you're taking a static view of that piece of code. Any concept of run-time module is therefore some different meaning of "module" than the one I have in mind.

Code isn't "correct". There's no such thing as correct code. Only services can be "correct". And the vast majority of services cannot be analyzed just by looking at a static view of code that implements them.

Anytime you consider the correctness of a module, you're taking a dynamic view of its behavioral contracts within an environment that provides other modules and services. When a module depends on other modules, that's a dynamic thing.

should be implementable as library, standardized upon only for reasons analogous to those for standardizing upon a matrix library

I agree that it should be implementable as a library, but it's one of those libraries that the language should be developed to support effectively... more like standardizing upon a matrix library in Mathematica, or how the console-IO is standardized in many languages.

A powerful implementation of service-registries, extensible plugin architectures, and multi-cast should be part of any future systems-programming or general-purpose programming language's standard-library and distribution... designed for portability, predictable performance, and reliability.

Heck, they should also be part of scripting languages, perhaps more primitive - right up there in the 'imports' statement. Perl with CPAN and Python with its massive library of available imports are useful, but it would be even better if they were described as competing services to be dynamically selected based on environment, policies, performance and level-of-detail, etc. with dynamic fallbacks when environment isn't ideal (i.e. automatically switching to a non-networked version of a service while the network is down).

Code isn't "correct".

Once again we're on different planets :). I consider a piece of code correct if it captures the abstraction you had in mind when you wrote it. Services, as elements of some service architecture, can be considered values in some suitable mathematical universe, and should be amenable to the same modes of static reasoning that one applies to other abstractions.

Once again we're on different planets :). [...] Services can be considered values in some suitable mathematical universe, and should be amenable to the same modes of static reasoning that one applies to other abstractions.

The services I use don't exist in a mathematical universe.

But you could represent many aspects of them, such as contracts and interaction, in order to help implement, validate, and verify services. That's what code is: a description of a service.

That's fundamentally different from mathematical functions, though automation of functional computation is a sometimes useful service.

I consider a piece of code correct if it captures the abstraction [the programmer] had in mind when [the programmer] wrote it.

As a potential client, I'd refuse to call a programmer's code 'correct' merely because it 'captures the abstraction' the programmer had in mind.

This reminds me of common sayings:

• Verification is making sure you built the system right
• Validation is making sure you built the right system
• Efficiency is doing things right
• Effectiveness is doing the right thing

Code can't get away with capturing an abstraction correctly, it also needs to capture a correct abstraction. Further, it needs to do so 'efficiently' with respect to the environment in which it shall be utilized. For example, the code shouldn't deadlock when composed into the client's environment, cannot violate real-time constraints and space-limitations, and so on.

The reason capability-driven design violates Law of Demeter is to reduce the amount of state and complexity required to implement multi-message protocol. Objects returned implicitly represent stages in complex interactions, and communication via the returned object implicitly establishes both context and access-rights. The vocabulary available via the returned object also helps programmers track this context and understand which behaviors are meaningful. Further, the access can easily be revoked or otherwise staged, which can enforce obeisance of protocol contracts and invariants.

LoD guidelines would deny use of such an object. Context can be achieved, but must be managed explicitly... e.g. passing a token or context-object back and forth between caller and callee. The callee provides the vocabulary with which the context-object may be utilized. That isn't a total loss. Using a dedicated abstract opaque type still helps reveal appropriate vocabulary to the user. But it hurts delegation, composition, and simplicity.

Composition under LoD involves a great deal of "vocabulary forwarding" and pass-the-buck programming: you're not simply handing an-object interface to the user, so instead you're repeating the vocabulary for that interface in the class definition. If a class manages multiple instances, you add selectors. Adding a new step to a protocol or a new operation to an interface-object would involve integrating that vocabulary in every user-class at each step. Attempting to avoid multi-message protocols also increases complexity, since one must create "large" command-message or query-message vocabularies in order cover all "steps" with a single message. Not only does this grow the vocabulary, it ultimately turns into a programming language when covering 'steps' starts to include error-handling policies and such.

I've never seen a decent argument in favor of LoD as a discipline or guideline for achieving encapsulation. Only the goal of LoD is laudable - reduced coupling to implementation details.

Instead of LoD, consider anything you have access to without hacking or reflection, you have full and absolute right to manipulate. If there are any contracts and protocols you are to adhere to, those can be enforced via careful distribution of interfaces and revocation. You are fully expected to directly use objects returned from methods or accessed via dotted path. Rather than the encapsulation onus being on the user (as in LoD), the onus is on the service provider.

Of course, the language must still provide encapsulation facilities, e.g. via interface without downcast.

I googled "capability driven design" and first programming related link was this very discussion!

Any better links?

www.erights.org is a good site for that content.

I don't believe I coined 'capability driven design'.

As at 1:43pm NZST 29th Jun 2009 site:erights.org capability generates 334 hits, site:erights.org "capability driven" generates none.

Congratulations! You coined a phrase. :-)

"capability based" is the term most used.

Really, it is model-driven, capability-based.

The idea of organizing things around models and not functions is well-discussed in CS literature (i.e., structured programming, Executable UML, model-driven architecture).

Capability-driven would imply that you have no way to predict how hard the next capability would be to implement. Capability-based leaves open the idea that the model you and your problem domain expert came up with should strongly dictate capabilities (functionality). In this way, it is much easier to achieve accurate estimates on new functionality: the model either easily supports the concept or doesn't.

That clarification better expresses what I intended.

I've been very satisfied with capability driven (object capability model) program design.

Such design often involves returning capabilities to various internal components of object configurations with the expectation they will be utilized, stored, or shared. Whether these capabilities exist prior to the request for them is entirely transparent.

Law of Demeter is routinely violated. Security is not. Desirable invariants and other contract conditions may still be enforced in capability driven design.

Clearly your freezer wasn't cold enough. Try a deep freeze.

Anyhow, you can't maintain invariants you hand out capabilities to systems you were aiming to protect.

But the extra effort needed to protect caps is a bit painful in a language with 'everything is an object'.