Functional Relational Programming: Out of the tar pit

In a similar vein to John Carter's recent submission, here's an attempt to attack the complexity problem: Out of the tar pit

Abstract: Complexity is the single major difficulty in the successful development of large-scale software systems. Following Brooks we distinguish accidental from essential difficuly but disagree with his premise that most complexity remaining in contemporary systems is essential. We identify common causes of complexity and discuss general approaches which can be taken to eliminate them where they are accidental in nature. To make things more concrete we then give an outline for a potential complexity-minimizing approach based on functional programming and Codd’s relational model of data.

They basically advocate minimising mutable state, moving any mutable state that remains into relations, and specifying the behaviour (i.e. manipulation of relational data) in a declarative/logical/purely-functional language.

Pros: the first half of the paper is a reasonable review of the types and causes of complexity in software.

Cons: lack of any implementation, other than a Scheme prototype of the relational bits, I think (see footnote 25). No source code. Lack of detail about interfacing with external systems.

Here's a link to Ben Moseley's FRP page, where you can find the paper, plus presentation slides and a discussion group.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

I am not entirely persuaded that state manipulation is 100% bad.

I am not entirely persuaded that state manipulation is 100% bad. Manipulating state is necessary in many tasks for performance and convenience reasons. What is bad is that current mainstream programming languages do not offer the means to abstract state manipulation.

I think the ideal programming language should offer layers of abstraction:

1) In the bottom layer, a C-like programming environment where state/control flow is manually programmed.
2) In the middle layer, an object-oriented environment that allows separation of concerns.
3) In the top layer, a declarative environment that allows the programmer to declare the application as a set of relationships between data sets (as the paper says), including functions.

I think that the current situation is very close to the above, but not in a planned way: the various programming languages that exist offer the functionality of one or two layers, but not all. For example, one can use ML or Haskell to code an application's logic, then use C for the really "difficult" tasks.

Ultimately, though, using a mixture of programming languages does not help towards less complexity. The complexity is moved from the language to the development environment.

State

State manipulation is not inherently bad -- rather, limiting your state manipulation to a certain scope of objects for any given task is good. Functional programming makes this nice because you know the only object modified is the one you're returning to the caller :)

Of course, some applications (say, emulating a CPU) are all about state manipulation and would not benefit at all (well, much) from a functional language.

The Headache of Stateful Concurrency

Not only that, but by avoiding the use of state when you can, it is quite easy to avoid a lot of headaches associated with concurrency. Without state, a calculation no longer needs to lock resources to avoid nondeterminism-based errors, nor does it need to perform hairy syncronization to keep stateful objects updated. So, without state, you can just tell a calculation to go off and compute itself somewhere and come back with the answer when it's done, without all the headaches (that I, at least associate) with stateful concurrency.

Now, that said, you still need some state behind the scenes to keep everything working, such as futures for passing evaluation results. There's a trick to doing statefulness with concurrency *right* that tends to scare people away.

Trick or treat?

Is this a trick or some insightful way of dealing with concurrent state?
Because I tend to see futures like lazy evaluations, but with a not so fancy dress on.

The nice thing with state is that it can be freely copied and distributed if it was immutable.

Still, if you want to change immutable state, make sure you refer to it with a new unique name while keeping the old state safe. Otherwise you're in big trouble.

Sharing immutable state is easy. Concurrently sharing (unique) references even more so.

Current databases/applications achieve shared state a-priori by introducing some kind of lock mechanism. But why use locks? Because they allow the illegal re-use of the same reference for different states in a system (=objects).

To achieve maximum concurrency, locked state and re-used names should be completely banned in a distributed setup.

This rule will eventually give us the scaleability everyone wants - its the referential transparency rule!

question

What if, for example, a specific field of a record must be updated and two users have opened the update form? the database will end up in having two records for the same thing, whereas one was expected.

Concurrency and State etc

There are obviously many different approaches to concurrency that improve on the commonly-used monitor-based systems - Erlang, Oz, the various process calculi-based approaches, and even the STM stuff in Haskell.

I guess our question is - how often is concurrency really an _essential_ part of the problem the users need solved?

...and of course we ask the same question about state - how often is it really an _essential_ part of the problem?

...if we can succeed in separating out the aspects of the problem which are accidental, then I think that would bode well. In my experience it is the _accidental_ complexity which kills really large systems...

Amen

It will be a great day indeed when the default approaches to basic architectural concerns in programming languages, such as mutation and concurrency, aren't, by default, the most complex to reason about, and therefore to use correctly, available! Peter Van Roy and Tim Sweeney have already covered what the alternatives might look like at length, so I won't rehash them here (and there was much rejoicing).

MVC?

So I read that paper, going yes, yes, yes, I agree with that, yup, yes, yes....

When I got to the end I said MVC.

Admittedly a somewhat purer form thereof, but MVC nonetheless.