Abstractionless programming

Since this shiny new forum is rather empty, let me get the ball rolling...

I have been thinking lately about ways to do programming without abstraction. Before you start yelling at me that this is not possible, don't take it too seriously, treat this as a thought exercise.

My motivation is twofold:

non-programmers and beginners can't deal with abstractions, it is their biggest hurdle, they would rather just work with concrete stuff
programmers (even good ones) aren't really any good at abstraction either. Being rather fanatic about abstracting driven by refactoring to get the best designs myself, it has become obvious to me that abstractions are a rather volatile and floating target. Programmers that think they create great abstractions ahead of time instead tend to produce stuff that just becomes a source of problems.

So the idea is to have a language where it is impossible to create abstractions, but rather the programming environment continuously refactors for you as you edit (yes this is a tall order, bear with me). This requires a visual editing environment rather than a tradition source + compiler environment.

My design sofar has you editing a tree based view of the actual unabstracted code. As you drag subtrees around, the environment marks things as shared/unshared (using a different background colour). This way, even functional abstraction can be created without the user even knowing what it is (creating a local change in a shared subtree is like having an argument to a function).

There are a lot of mucky details, but I believe it may be possible to have a system that always refactors for you to have the optimal sharing (thus abstraction) in the background, with the user needing to know next to nothing. Maybe a dream, but a fun experiment.

Most importantly: is there anything remotely similar to this out there? google wasn't particularly helpful sofar.

Intentional Programming

Looks remotely similar to Intentional programming.
IP was a promising hype generator several years ago, but somehow it lost its momentum.
You may want to check http://intentsoft.com to see if Charles Simonyi and Co have any progress.
IP was discussed on LtU: Intentional Programming Overview and Intentional Programming FAQ.

Not that IP is abstractionless, it just feels similar to what you described.

By Andris Birkmanis at Tue, 2004-06-22 08:08 | login or register to post comments

Abstractions are what it's all about...

Abstractions are not just about reuse. They are about meaning. I don't see how a tool like you describe can help with this, which is a. the core difficulty and b. the main reason to use abstractions.

By Ehud Lamm at Tue, 2004-06-22 08:54 | login or register to post comments

The solution

The obvious solution is teach abstraction skills in a better way. My proposed The Little Abstractionist aims to do just that.

Plus, I think every SE couse must devote some time to explain the standard abstraction mechanisms in programming languages.

By Ehud Lamm at Tue, 2004-06-22 09:00 | login or register to post comments

What is `abstraction'?

If by "abstraction" you mean simply the syntactic notion of a language without variable binding, then there are already plenty of candidates: the Categorical Abstract Machine, Combinatory Logic, Joy, Forth, ...

If you mean a language whose semantics lacks the expressivity to express sharing, there is affine logic. If you also disallow discarding, there is linear logic. If you also disallow environment permutation, there is non-commutative linear logic. These things are interesting and useful, but they should only form fragments of a language, since you need the ability to do sharing, discarding and permutation in order to get Turing-completeness.

By Frank Atanassow at Tue, 2004-06-22 13:09 | login or register to post comments

abstraction = sharing ?

You seem to equate abstraction with structural sharing. Your 'tree building' design is already an abstraction. What would be 'concrete programming' ? Maybe manually setting switches and reading leds ? :)

Don't mean to be offensive, but virtually every concept we deal when programming is already an abstraction. Pure thought. Physically it's a mess of electrical signals that apparently has nothing to do with what we think they are. And we can program without the machines, as some people did before computers were even invented...

By Andrei Formiga at Tue, 2004-06-22 14:20 | login or register to post comments

I was designing something very much like this last year

In fact, what I was doing for my office last year was building an engine that manipulated trees to build business rules for our knowledgebase system, outputting them as R5RS scheme. I had a palette and a Swing TreeView. The user could drag from the palette onto the treeview to add a leaf operation or edit a branch. The data came in through the leaves and filtered up to the root, where you got an output value. Quite nifty, and it was specifically targeted at our "product" people, who are basically marketers and "idea people" without a clue as to how to program or what a "function" is. They liked the idea, but it got swept up by an enterprising young project manager and assimillated into his project without ever really reaching fruition.

The really cool thing about it, though, was that it was pure functional without the user ever knowing what a function was. The user built up multiple trees each tree drawing on some fields from the source database and outputting a single new field. Every node in the tree would return a value to its parent. The root node's return became the output field for the database. The user could then select the output fields he/she wanted on the finished product. The system would load these trees into memory, then it had an optimizer such that if two trees had any matching sub-branches (from the leaves up) it would only execute the branches once between trees (continuations).

Also, it was lazy, so that if a particular node had already decided its return value, it wouldn't bother evaluating the branches below it .

I'd be happy to lend you my notes and even some code I wrote to build the system. I'm not under a NDA, and I wouldn't be sharing any proprietary secrets anyways. It's all Java and writes to R5RS scheme.

By the way I wouldn't call what you propose "abstractionless" or "concrete". It's concrete in the sense that you give your user a very concrete picture of how data flows through the program, and how operations relate to one another, but this is not sans-abstraction. It's is merely a different way of looking at pure-functional programming. I would rather call this particular dialect of functional style "flow-oriented", not "abstraction-less". You are right about one thing, at least in my experience. Novices find this method very easy to learn.

By Lu Tze the Sweeper at Tue, 2004-06-22 17:13 | login or register to post comments

Ok...

I guess I wasn't clear about abstractions. What I was referring to is the predominant way of creating abstractions in most languages, which is creating a function (or method, predicate, rule, ...). Yes my designs naturally also deals with removing the need for temporary variables, but as Frank indicated, this has been done before and not that hard, given a visual environment. Data structure abstractions (classes, ADTs,..) are going to be MUCH harder to automatically refactor, I am ignoring them on purpose.

Andris: I have heard of intentional programming before (on LtU actually, if I remember correctly :) and again it is not solving the problem of reducing the need for explicit abstractions. Really, if this: http://www.program-transformation.org/Transform/IntentionalProgramming is anything to go by, then its more like a very evolved macro system.

Ehud: I disagree that abstraction is about meaning. It is exactly this direction in programming which I want to get rid of (the OOA/OOD idea of starting out with real world concepts that are present in your requirements, that leads to useless abstractions). By letting abstractions "emerge" you get much more meaningfull abstraction boundaries (the fact that something is used multiple times may actually be very meaningful). I will concur that it is useful to have "names" attached to large chunks of code, which is potentially something you'd miss out on.

Frank: I am very familiar with linear logic... it is related in a sense: what linear logic does for runtime data structures, I want to do for code :) Just runtime data only deals with sharing, it doesn't even have constructs as "complex" as functional abstraction.

Andrei: its not the same as sharing, though obviously strongly related. I was merely showing increasingly complex editing operations that deal with shared subtrees equate to what would be certain forms of abstractions (functions) in normal languages. Working with concrete stuff means working with code that doesn't define abstractions, i.e. operations on only concrete values... not referring to the real world here :)

Lu Tze: that sounds really cool, I'd love to hear more... just a design doc or something will do, no code required. From your description it sounds similar to what I start with, though it may be that your "code" was so much a DSL that it didn't really require abstractions in the first place (does it create any kind of abstractions for the user on the fly? does it refactor them automatically?). My intentions are to be as close to a general purpose language as I can manage.

Yes it is clear this needs another name. But I was so much looking forward to be able to write "Abstraction considered harmfull" at the top of some paper ;)

By Wouter at Tue, 2004-06-22 21:35 | login or register to post comments

It was indeed a highly domain

It was indeed a highly domain-specific language, although that was not a requirement, nor was the engine limited to domain specificity with a few minor additions. The basic principles of functions held. "A tree can have any number of leaves, but only one root" vs. "A function can have any number of arguments, but only one final value" (not strictly true, I know, but it's sufficient to make my point).

To answer your questions, yes, it did abstract on the fly, in that once a tree was created, anywhere you needed the value of the tree, it was automatically inserted, parameters filled (or asked for, if they weren't concrete) and would be evaluated at runtime. This is actually one of the surviving features of the project and key to filling our need for high performance. Refactoring occurs through the optimizer, and oft-used subtrees can be broken off and suggested to the user as named output values themselves.

The primary point of domain specificity is that in the application where this was used, (knowledgebase build) there was no need to handle I/O, except through a very restricted channed, nor was there any need to handle interactive use of the language or applications written in it. It's feature set was not oriented towards "applications" in the traditional sense.

But as I said, it output Scheme, so in a real sense, anything available in Scheme could be available to a language based on it, and by extension, it is a complete functional (all debating about Scheme's functional nature aside -- the language I built on top was side-effect free) language. If you have contact information, I can send you a design document to look at and if you like, we can work on a more substantial one for what you propose. If you don't like it, well, don't worry about it, but you sounded interested, so I thought I'd follow up.

By Lu Tze the Sweeper at Wed, 2004-06-23 00:46 | login or register to post comments

...

what you describe is still not what I intended, as it deals with function calls (and sharing of trees, as normally temp variables would do), but not _function definition_ (abstraction). Also, "refactoring" in the optimizer has no benefit of code structure to the programmer.

Some of the features you describe are really good though, and many are present in an older visual programming environment I wrote (strlen.com/aardappel). I would love to see your design doc, email me at w v o a t g m x d o t n e t and we can talk more.

By Wouter at Wed, 2004-06-23 07:00 | login or register to post comments

Premature sharing considered harmful :-)

If I understood the idea, it was about the tool finding similarities in "code" as soon as it is entered into the system, and to factor the code for maximal sharing (pattern matching/compression/normalizing? Is DRY principle relevant?).
If my understanding is right, just how stable will be "abstractions" created this way? Intuitively, I would say they could drift every now and then. And the names the developer attached to them will go away, probably. Not to mention problems with version control.

OTOH, it's possible that some abstractions will be much more stable than others, and the tool will detect that, and will offer the developer to document them and package them as a separate library. We need an experiment to back this hypothesis. I would suggest a self-hosting of the tool in itself as a pilot project for finding stability patterns, but this will require quite a lot of libraries, including UI...

Another idea: if the tool has a specialized version control system instead of CVS, it may consult the history for heuristics on where to abstract (the code will remember where it was folded in the past :-) ). E.g., if the developer has named some abstraction, the tool will be less eager to drop it, and more eager to return to it when the codebase changes.
The developer could also lock some abstractions, or libraries of them...

Hope I do not sound overly optimistic, e.g.: how to handle FFI? Or are the points of interaction with other languages marked as frozen abstractions?

By Andris Birkmanis at Mon, 2004-06-28 09:42 | login or register to post comments

Yes, it is very strongly rela

Yes, it is very strongly related to DRY. Trying to find the optimal factoring automatically makes code adhere to DRY.

Abstractions will not be stable, but you should care less: as you don't have to maintain them, just be able to read them. They are probably more stable than you think, as the idea is not to "compress" the code from scratch all the time, but let user actions (dragging & dropping of code) be a major factor in what is shared and what not. It would be too messy otherwise.

Yes, I presume FFI would be treated much like builtin functions. You have to start somewhere. I'd be more than happy if this idea work in a relative simple, protected environment, and worry about inter-language operability later :)

By Wouter at Tue, 2004-06-29 07:18 | login or register to post comments

Optimal sharing

I believe it may be possible to have a system that always refactors for you to have the optimal sharing (thus abstraction) in the background...

That's an interesting idea. What consequence is there to the user once the system figures out the sharing? Just because the code presents an opportunity for factoring doesn't mean it's a good idea to take it (unless pure code compression is all we want).

By Darius Bacon at Sun, 2004-06-27 21:10 | login or register to post comments

P = NP, up to sharing

According to Andrea Asperti: P = NP, up to sharing.

If I understand this, it suggests that an effective algorithm that optimizes sharing would prove that P = NP.

(I haven't read the paper; also note it appears to be unpublished.)

By Frank Atanassow at Mon, 2004-06-28 16:06 | login or register to post comments

...

Darius & Frank, the idea is not find the perfect sharing in all cases (perfect compression), as that would indeed be difficult, and also probably unwanted (not all 2 copies of anything need to be shared, and its sometimes hard to draw the line).

The user's editing actions (user intent) form the primary boundaries for the algorithm to work with. Dragging a subtree creates sharing. If the user however insists on creating a subtree from scratch that is identical to one that already exists, it may not be desirable to have the system make this shared.

The idea is here that a completely inexperienced user can create code and sharing/abstractions will be created for him automatically, resulting in maintainable code. Learning what sharing means will come intuitively, and when the user becomes more advanced, can make use of it more explicitly. At no point will the factoring of the program be surprising / changing radically.

The prime goal is to be able to functional abstraction ("parametrized sharing") automatically.

By Wouter at Tue, 2004-06-29 07:29 | login or register to post comments

Published

Asperti & Mairson's paper was published in Information and Computation in 2001. A very nice paper, but not for the faint hearted.

By Charles A Stewart at Tue, 2004-06-29 11:28 | login or register to post comments

Same paper?

Is that the same paper? They seem strongly related, but the titles and author lists are different.

By Frank Atanassow at Tue, 2004-06-29 13:21 | login or register to post comments

Managing Duplicated Code with Linked Editing

Via CleverCS

Managing Duplicated Code with Linked Editing

We present Linked Editing, a novel, lightweight editor based technique for managing duplicated source code. Linked Editing is implemented in a prototype editor called Codelink. We argue that the use of programming abstractions like functions and macros -- the traditional solution to duplicated code -- has inherent cognitive costs, leading programmers to chronically copy and paste code instead. Our user study compares functional abstraction with Linked Editing and shows that Linked Editing can give the benefits of abstraction with orders of magnitude decrease in programming time.

The paper claims that linking duplicated code beats (functional) abstractions on every position.

What's better then HOFs? What's easier than functions and more comprehensive than macros?
All developers agree - Linked Editing is the best for them.

By Andris Birkmanis at Sun, 2004-08-08 14:44 | login or register to post comments

Non-Prototype Literate Programming via Outlines, w/Cloned Nodes

See LEO: The Literate Editor with Outlines. It's in Python and uses TkInter, so it's a bit clunky and slow and not very "Mac-like" on my Mac OS X box, but it does work and I do find it very cool.

By Paul Snively at Tue, 2004-08-10 16:52 | login or register to post comments