Lambda the Ultimate

(I only skimmed the document so...)
The nice aspect of "decomposing" transactionality and using transformation technics to move from one representation to another (thread versus events) in a single program space. This allows different parts of your design to work with different representation and yet they all work together. I am not sure that this paper is arguing this advantage...

Page 8, bottom right:

This event-driven architecture is similar to that in SEDA [25], but our events are finer-grained: instead of requiring the programmer manually decompose a computation into stages and specify what stages can be performed in parallel, this event-driven scheduler automatically decomposes a threaded computation into finegrained segments separated by system calls. Haskell’s type system ensures that each segment is a purely functional computation without I/O, so such segments can be safely executed in parallel.

Most user-level thread libraries do not take advantage of multiple processors, primarily because synchronization is difficult due to shared state in their implementations. Our event abstraction makes this task easier, because it uses a strongly typed interface in which pure computations and I/O operations are completely separated. In Figure 4, the five steps of lazy thread execution are purely functional: they process an “effect-free” segment of the user thread execution. All I/O operations (including updates to shared memory) are submitted via system calls and are explicitly performed in the scheduler; the programmer has complete control on what I/O operations to run in parallel.

In other words, this library enables you to write code in a straightforward thread-based style, which transparently uses efficient asynchronous I/O ala event-loops. You can also provide a custom application-level scheduler written in event-loop style should you need to provide application-specific optimizations. It also transparently multiplexes all of your "user-level"/"application-level" threads over real OS threads for optimal CPU utilization. And it's all type-safe. Sounds pretty good to me. :-)

Occam has allowed this at the language level for 20 years. Any of the process-based family of languages has a much tighter integration between the thread-view and the event-view. The underlying implementation is very efficient, so that code that looks threaded to the programmer is scheduled as events. The transputer architecture offered parallelism for programs written in this language.

Of course the process languages never really became mainstream, and maybe it's time for this to be reinvented to see if it catches on the second time around.

One aspect that you've quoted that is advanced in contrast to the CSP languages is the automatic extraction of parallelism. In Occam the programmer is responsible for identifying the parallelism in a program explicitly. Allowing the scheduler to decompose the program into segments and decide the constraints over their execution is a good step forward.

Web continuations have already been mentioned, but I wonder how this would cope with a loosely coupled cluster as an execution environment? If the segments are relatively long or the dataflow between them is sparse then this could have interesting results.

Well, when it comes to modern process-based languages there's always Erlang, which is kind of mainstream (at least for some applications). And the KRoC team is working hard at making occam relevant again, by adding various capabilities for mobile channels and mobile processes. OTOH, occam-pi only runs on x86 machines at the moment, so you can't really take advantage of the nifty parallel scalability offered by transputer clusters. But now that the KRoC's transparent networking capability is in place, I imagine that you could do some neat things with dynamically distributed mobile processes.

I have to agree with Andrew Moss here. I feel like the transformation between events and threads via CPS is implicit in, say, all the web continuations stuff. I would love to be proven wrong here though.

I guess I'll just have to parrot naasking here: the benefits are that the scheduler is user-level and customizable and that the user-visible implementations are simple, simple, simple: typically ~200 lines, and type-safe. Pragmatically, this makes a huge difference. I've lost count of the number of "frameworks" of various kinds I've used where crucial user-visible state-machine pre-conditions, invariants, and post-conditions weren't documented at all, let alone checked, and actually extending anything would break the system at the drop of a hat. Having source code has often been of only modest benefit, with individual functions/methods running into the dozens of screens of densely-intertwined code.

So the main result isn't that events and threads are dual; as the paper points out, that point has been made elsewhere. The main result seems to be that this duality can be exploited in such a way that subsystems that traditionally haven't been user-visible are, that these subsystems can be modified realistically by users of the framework, and that this is due to the leverage offered by the expressive type system of a good language—in this case, GHC with its extensions to Haskell 98.

Starting with purity and using something akin to a monadic typing discipline ("all effects take place via a monad which shows up in the type of the result values") allows precise control of which side-effects are possible when. So being able to mix purity and impurity effectively without blurring the boundaries is indeed a type system benefit.

I'd think it'd be the scheduler that forces the evaluation.

Seems that link is now also dead.

I found an older (?) paper here:

A Language-based Approach to Unifying Events and Threads