Lambda the Ultimate

A few days ago I had a somewhat interesting idea about bootstrapping PL (programming language) tools and components after you assume a process-oriented architecture. This post amounts to sharing. The entertaining part, if there is one, will become apparent on reflection. To make this less concrete, I'll start with a minimal bit of abstraction about what process means.

Suppose asp means abstract sequential process (chosen so pronounced as one syllable with an arbitrary concrete/visual image of a snake suggesting a chain of continuations). Then say nop means native os process, typically implying an address space and whatever the OS does in the way of a process. When you launch an app written in C on some platform, typically it starts a new nop and eventually gets around to calling main() in the main thread, which is the root of the asp which lives in that nop.

An asp is something like a fiber, but a typical C program with no threads and no extra fibers will amount to one nop, one thread, and one fiber, thus seeming synonymous with the asp as executing code in a general sense. The idea of asp is only useful when you think about mapping it differently. You can run an asp in a thread (inside a nop), or you can run an asp inside a fiber (inside a thread inside a nop). If an asp talks to the world only via messages, for example using standard input and output streams, then you can map it somewhere else while preserving behavior. An asp run as a standalone nop would be similar in behavior to one run in a thread or fiber, but be much easier to debug that way in isolation, where it can't corrupt memory in another asp, for instance. It doesn't matter what language an asp is written in, when the only way you interact with it is via messages.

Here's the part I find entertaining. To bootstrap, you can start with an asp written in a way you find totally unacceptable in the long run, as long as it helps you write new versions you run later with better properties. You can write an interpreter that's a slow memory hog that does everything slowly and monolithically, but use it to write assorted tools and transpilers that later run much more efficiently and asynchronously, perhaps compiling itself to a better form to use in subsequent invocations. Since every asp is independent, except to the extent it depends on cooperation of other entities it messages, you can run old and new versions of an asp at the same time, to compare in tests or to stage incremental service upgrades.

The same general daemon nop architecture can be re-used in each version: something that spins ups and listens to connections, running asps in threads, perhaps inside fiber pools within. You can start with one as a root session that starts other daemon nops as requested, when you want parts of service isolated or restartable without a root session reset. Maybe one child nop is the virtual file system, for example. You can rebuild executables and launch new child nops with upgraded features to replace old inferior child nops, then shift service to new instances. As long as you are connecting everything with channels managed by daemon non-blocking i/o features, it doesn't matter how many native processes you have, or where things get assigned, and you can use config specs to change where things go.

I thought it would be fun to be able to write terrible (but easy to understand) early versions of things that can generate new and better versions that run with identical external process interfaces you can swap dynamically. Bad scaffolding architecture doesn't need to infect or influence later improved versions. So you can start out writing things in the most general way you like, then slowly generate concrete translations of that into the target quality you want to read over time through stepwise evolution. This idea might help beginners in PL research, since it means you don't need to be especially careful about your starting point. Process separation implies disposable steps.