Lambda the Ultimate

You can help me get some docs underway by suggesting an app which benefits from concurrency support, especially if scale would surpass comfortable native thread limits, so cooperative multitasking is suitable and meaningful.

A game. One with thousands of enemies/monsters/soldiers/whatever in the world all running simultaneously with a task for each one.

As interest seems low, I'll note a few ideas I had in mind before this topic rolls over the horizon. Seems worth the off chance of getting more app suggestions.

Term app means any sort of executable specializing parts of a general library so it does something. Every product that includes one or more libraries, such as the standard C library, is an application of those libraries. In recent years, some folks might suppose app means only something you can buy in a store to run on your phone. But the sense I mean includes any general piece of software applied to do something more specific, from template instantiation in C++, to one-off utils we write for small tasks. The sense of product that appeals to someone isn't directly relevant.

The library I'll describe will be in C, but any imperative language would work to get a like result, from assembler to Fortran, as long as modifying state is the way representations are simulated. A language on top like Lisp that compiles to the C involved, or another imperative language like assembler arranged the same way, would have similar underpinning. I'll make every part sound as trivial as possible, if I can. Complexity would come from stacking up lots of the trivial parts. The C version aims to be useful to lots of folks building obscenely complex things in C now, like load-balancing routers with app servers glommed onto them as plugin modules. (It would not be as crazy as things now deemed acceptable.)

The core idea is an engine, sort of a VM, that takes a hunk of memory you delegate for its use, then builds a way to run lightweight processes and fibers inside, so it looks like a daemon embedded in your executable, in a way letting you see into and control every part of it. Any time you interact with something else outside your app, you can defer this interaction to the VM, and it would handle issues of timing such as waiting, blocking, callbacks, retries, and sick non-determistic event sequences. All i/o would pass through API you provide, but with any potentially blocking parts duly noted so the VM can hand those parts off to a worker thread pool (if that's what the environment wants) so blocking doesn't stall the non-blocking async VM engine.

In effect, the craziness and vagaries of code outside an app would get reified as logic in fibers explicitly handling those things, in a way insulating a host app from dealing with painfully awkward edge conditions. The VM would look a little like a toy OS using cooperative multi-tasking, which only runs when you let it, consuming only events you permit, and only in timeslices small enough to make you happy. An associated utility would be able to rewrite C code designed to run as a stand-alone command line executable, so it runs as one or more fibers in a lightweight process lane instead, after a CPS transform makes threads run as coroutines in the VM's trampolined dispatcher. If you know Apple's old MPW development environment, this means compiling C apps to run as embedded coroutine tools in the non-blocking VM runtime.

The C library amounts to a coroutine-tool running engine. So what it does depends on the tools you run in it, and the interface you set up for it to interact with the outside world. Each tool applies the library, and the set of all tools you put together as a suite effecting some overall objective is also an application of the library, just at a higher level.

So by apps I mean 1) things you can get to occur as coroutine tools inside the embedded daemon VM, 2) things you would do in a top-level executable that hosts the embedded coroutine daemon engine, and 3) schemes to arrange client and server processes hosting embedded daemon VMs to effect distrbuted end-user applications.

For example, one app is a view of what the VM is doing. The architecture supports fractally nested embedding, so you can run an embedded daemon in the top-level daemon VM, such as an HTTP server presenting a proc file system view of internal VM state. If you launch this in the coroutine tool VM, and you're willing to route traffic to it, you'd be able to browse VM engine state with a browser as a UI, even if there's no other UI to a headless daemon. If an interactive language runs in there, you can send code from a browser to execute. Clearly a shell is not hard to write this way.

Or embed it in a network appliance and have it run codecs transforming connections in arbitrary ways with your favorite tunneling scheme. If it has a coroutine-tool-based micro HTTP engine inside, you can browse what this component of the network appliance is doing, or directly manage configuration and operations.

One of the apps I hand in mind, for one character to obsess over in docs, is a dynamically reconfigurable browser view of what's happening, which runs in the daemon to model what is displayed via HTML. You would need fancy versioning and undo support, so you could revert to a view based on one you didn't just corrupt by a recent change. Part of this seems to involve a compiler from some language L to Javascript, so the daemon can emit code a browser can execute, but driven by the daemon. If nothing else in this post is LtU relevant, at least this part is.

Another character can aim to create a Hypercard work-alike, but with a new scripting language, whose front end is either a browser or a native app using a widget library, which also runs an embedded coroutine-tool daemon VM, just so the front and back ends can share code promiscuously in a peer to peer model.