Lambda the Ultimate

inactiveTopic I Was a Smalltalk Gunslinger
started 8/16/2002; 2:38:29 AM - last post 8/20/2002; 2:41:35 AM
jon fernquest - I Was a Smalltalk Gunslinger  blueArrow
8/16/2002; 2:38:29 AM (reads: 1627, responses: 8)
I Was a Smalltalk Gunslinger
Panic in corporate development land. The scheduled to be presented to upper management in a week and it still takes several minutes to get a query back. Call in the gunslinger. ...The Smalltalk dialect was unfamiliar to me and the problem domain (screen scraping mainframe terminal data and shipping it to a web browser) was alien. But I needed the money... ...I dived into the code with the help of one of the locals. We poked around with the browsers and debugger... By the end of the day we hadn't found much, but we had eliminated a lot of places where the problem wasn't... We didn't make the noon deadline, but we did have it an hour later. ...We made the change there in the debugger and proceeded. Additional testing confirmed that performance went from nasty to acceptable and that there were no obvious harmful side effects.

Lesson: The method of least knowledge. I never did learn much about the system under development and I still know zip about screen scraping mainframe terminal data, but I was able to use the available tools like the debugger to drill into the code where it was possible to see a potential problem. In fact I think that if we had focused more directly on solving the problem from the start rather than on my understanding the system as a whole we might have fixed it the first day.

Why does there always seem to be this tension between understanding the big picture and solving very specific problems. Abstractions often seem to bury the details of a design. I often see criticisms of java programs that use too much abstraction. What can be done to make drilling down into abstractions to get the details easier? The author provides a nice outline of troubleshooting strategies ranked by the time they consume with ruminations on the tools he has and the tools he would like to have. In an ideal world with perfect semantic descriptions of software systems would this sort of gunslinger or relief pitcher become extinct? A position paper from the OOPSLA 2001 Workshop On Software Archeology: Understanding Large Systems.
Posted to Software-Eng by jon fernquest on 8/16/02; 2:53:59 AM

Ehud Lamm - Re: I Was a Smalltalk Gunslinger  blueArrow
8/17/2002; 2:40:57 AM (reads: 788, responses: 0)
Is this really language related? Put differently: Just how important is the programming language in all this? Are any of these techniques specific to Smalltalk?

Now, obviously, language can make code more readable and easier to maintain. But are there real differences between most mainstream languages today?

An important factor is the availability of tools. IDEs, class browsers, refactoring browsers etc. This is one area where Smalltalk excels.

By the way, my personal experience shows that well written assembler is easier to work with, than poorly written Ada/C++ etc. That's why you should be careful choosing the person whose code you going to take over

Ehud Lamm - Re: I Was a Smalltalk Gunslinger  blueArrow
8/18/2002; 8:11:46 AM (reads: 769, responses: 0)
Why does there always seem to be this tension between understanding the big picture and solving very specific problems.

Beacuse of our cognitive abilities?

In fact, I don't think there's tension. I think that good programmers think on many levels at the same time (problem domain, system architecture, programming language, machine architecture).

Good programming languages allow you to focus on the level you need, when you need.

Abstractions often seem to bury the details of a design.

Information hiding is the goal, is it not?

I wouldn't use the word bury. I'd rephrase: (Good) abstractions let you ignore unimportant details of design.

I often see criticisms of java programs that use too much abstraction.

No such thing as too much abstraction. Choosing wrong kinds of abstractions or simply choosing wrong abstractions is, of course, entirely possible.

Notice that OOP languages tend to push you in the direction of the "abstraction=classes and inheritance" point of view. There are several other important kinds of sw abstractions...

Luke Gorrie - Re: I Was a Smalltalk Gunslinger  blueArrow
8/19/2002; 11:54:04 AM (reads: 743, responses: 4)
The statement "No such thing as too much abstraction" is too abstract :-). Practically speaking, adding abstractions to Java programs usually involves things like adding interfaces, "factory objects" to choose implementation at runtime, etc. This gains flexibility at the expense of adding more code, and making it harder to understand what a program does by just reading the source code.

The problem, I think, is that there's a common notion that it's always a good thing to add those abstractions, and the chosen interface is usually pretty ad-hoc. Consequently, anything forgotten in the abstaction (like how a database interface manages its connections) becomes an undefined "implementation detail", and you can't so easily find (or change) the answer by just looking at or hacking "the code."

That's how it seems from my Java hacking experience, anyway. You are right that really it's just using bad abstractions, but it comes from a drive to hide as much detail as possible that often goes much too far. Saying Java programs "use too much abstraction" definitely means something to me.

I found a Dijkstra quote recently that hits the nail on the head for me:

    ... it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.

This point is often missed - and I won't claim innocence myself.

Ehud Lamm - Re: I Was a Smalltalk Gunslinger  blueArrow
8/19/2002; 12:20:07 PM (reads: 758, responses: 3)
... it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.

This is a great quote. I think it is of great important to language designers - but at least most of them are aware of the fact that they should try an create "a new semantic level".

I know exactly what you guys mean by "too much abstraction," but I think using this phrase is dangerous. It makes people think that simply exposing details is the way to go. This is worng, and leads to bad sw design.

Good software design is about choosing appropriate abstractions, and using the appropriate language facilities for defining them (e.g., templates or macros).

Ehud Lamm - Re: I Was a Smalltalk Gunslinger  blueArrow
8/19/2002; 3:08:04 PM (reads: 798, responses: 0)
Case in point: Would you consider log4j to be an example of too much abstraction?

I have been using the log abstraction (using several of my own designs) as a case study this last semester (my students just finished sitting on an exam, which contained one more variation on the log theme. I am pretty sure they are fed up with it by now...)

jon fernquest - Re: I Was a Smalltalk Gunslinger  blueArrow
8/20/2002; 2:41:35 AM (reads: 721, responses: 0)
A review of a book on screen-scraping with Perl's LWP module:

one day we might all use Web Services and easily get the information we need from various providers using SOAP or REST, but in the meantime the common way to achieve this goal is just to write code to connect to a web server, retrieve a page and extract the information from the HTML. In short, "screen-scraping."

Frank Atanassow - Re: I Was a Smalltalk Gunslinger  blueArrow
8/20/2002; 9:21:55 AM (reads: 789, responses: 1)
Abstraction is not an end in itself. I think there are three primary uses for abstraction in software development. By abstracting away from a particular implementation, you 1) allow the implementation to be replaced by another more efficient implementation and 2) make it easier to replace the implementation by another implementation whose behavior is slightly different. Finally, 3) often abstraction brings out certain patterns which make it easier to grasp or reason about your program as a whole.

Here is an algebraic view on abstraction in software engineering. I intersperse the text with translations of the mathematical concepts into programming terms between parentheses.

Abstracting away from a particular implementation is like taking a set (the implementation) and giving an algebraic axiomatization of it (an API with a contract). You do it by identifying certain operators on the set (names of procedures), giving you a signature (the API), and laws they satisfy (the contract). This characterization induces a class of algebraic models (possible implementations). One of these models is the "intended" one (the implementation you abstracted away from).

A pitfall that mathematicians (programmers) often fall into is that they reason about the class of models (the API plus contract) by reasoning about the intended model (the original implementation). This leads them to ascribe properties of one model to all the models (ascribe properties of the implementation to the API+contract), which leads to incorrect "theorems" or proofs (malfunctioning programs, or programs which stop functioning when the implementation of the API is changed). Falling into this pitfall means you cannot achieve goal #1 of abstraction above.

(What actually happens most commonly in software development, though, is that the contract is not documented!)

Another pitfall is making your signature and laws (API and contract) too strong. This makes the class of models (possible implementations) too small. This pitfall is very easy to fall into. By adding one law (one statement in the contract), you can collapse the whole class of models to be empty (no possible implementations) or isomorphic (implementations may differ in efficiency, but not essential behavior). If empty, you have made a mistake. If isomorphic, you cannot achieve goal #2 of abstraction.

(Again, though, what happens most commonly is that only a fragment of the contract is documented, and that fragment is actually too weak, and then you fall into the intended model trap again.)

Finally, the way in which you abstract away from a set is pretty open-ended. You could identify ten, or a hundred or an infinite number of operators (API procedures), and ten, or a hundred, or an infinite number of laws (statements in your contract), each of which may be arbitrarily complex. If the result is something which is hard to reason about, then you will not achieve goal #3.

Note, BTW, that abstraction does not always mean "forgetting details". As I mentioned above, you can pick your abstraction so all models are isomorphic, for example, and this is can be a useful thing, even if it means you don't get goal #2.

Ehud Lamm - Re: I Was a Smalltalk Gunslinger  blueArrow
8/20/2002; 10:01:46 AM (reads: 823, responses: 0)
BTW, that abstraction does not always mean "forgetting details".

Right! Let me extend the notion of abstraction a bit, and while at it give an example.

Parameterization is a form of abstraction. When you parameterize a module that uses a stack, so that the stack (type) is a parameter, you reduce the coupling between the module and a specific stack implementation.

Notice, that you exposed details (i.e., that a stack is used). Presumably you did that because these details are important to the module's users (e.g., they want to control the memory behaviour of the stack used).

Another example is a sort routine, parameterized by the order relation for the element type.

Generic programming tends to expose these kinds of relations between components.