Lambda the Ultimate

A year ago there was a forum topic about Swarm, a "distributed stack based programming language".

The basic idea is that data can be distributed across multiple computers, and when a thread needs to access some data on a remote computer, it is captured as a continuation, and migrated to that computer, where it is resumed. It is the ultimate expression of the maxim "its cheaper to move the computation, than to move the data".

Then, the idea is to use a "supervisor" process to monitor when the computation must jump between computers, and adaptively rearrange the data to minimize the frequency with which this must occur using some kind of clustering algorithm.

A year ago Swarm was a custom stack-based programming language, implemented in Scala. Of course, the fact that we had to design a completely new programming language was a serious problem, but at the time it was the only way we could get portable continuations.

Swarm has come a long way since then, and after a complete redesign you can now write Swarm code in (more or less) plain Scala. This is possible due to the delimited continuations Scala compiler plugin that will be bundled with Scala 2.8.

Swarm's source code is available under the LGPL, and we are actively looking for people to help us solve the many remaining problems with it.

You can find the source code, and a video explaining the project, at our Google Code page (sorry, no paper yet - the video is the best explanation).