Lambda the Ultimate

This essentially just describing an implementation of exactly the Emacs model. And because of course I write everything in Lisp, implementation is trivial :)

I'm not satisfied by this because I don't understand *why* Emacs need to be designed (or happen to become) this way. Moreover, the Emacs model is overly specific (we need a Boolean variable for every mode. Mode itself is just a function to turn on/off mode but it has to perform a specific ritual to be "well behaved". Functions need to perform the ritual of checking advices), which make me hardly believe it is some design from first principle, rather than some accidental artifact resembling an unknown design from first principle.

1. Emacs grew out of an architecture that predates GNU emacs and that resulted from beginning with an interactive full-screen (terminal) text editor and making it customizable, extensible, and self documenting. Many of its features are the result of decades of thinking and experimenting with the intersections of how such an editor works internally, how users experience this, how extension writers organize things so that multiple, separately written extensions can all co-exist without excessive complexity.

Therefore, the best way to learn why the features are what they are is to study existing code while trying to write more. And keep in mind the metaphor: Why is a doorknob the size it is? You can't answer that question without asking questions about the human hand. Similarly, you can't answer questions about why Emacs Lisp is as it is without asking questions about the problems Emacs tries to solve (e.g., still works nicely on a terminal) and about the human factors, especially users.

2. If you want to write a s-exp structure editor, I can't imagine a better way to get started than writing one in Emacs lisp. It is likely to be fairly easy once you get the fundamentals done. For example, node-specific modes, minor-modes, keymaps, properties, etc. are all easy to do with the existing facilities of GNU Emacs.

Happy Hacking.

Just read the book 'bout Emacs. I believe the answers you seek are in there, my child. https://www.finseth.com/craft/

This essentially just describing an implementation of exactly the Emacs model. And because of course I write everything in Lisp, implementation is trivial :)

This is just describing some data structures. Yes, they are ones I think will be closely analogous to the ones used in the Elisp VM, but they could easily be implemented in any general-purpose programming language.

This thread has me pondering the idea of writing a Nanopass compiler for a subset of Elisp, targetting something fairly high-level and dynamic but very unlike Elisp bytecode+VM.

Structure editors are multi-media text editors, nothing more.

By "multi-media text editor" I mean an editor where every character has a (usually empty) property list, the property list can contain a reference to an underlying non-character object, and the display of characters can change to reflect that underlying object (e.g. an embedded image for a gif object).

If an underlying object is editable in ways not usefully represented as text editing commands -- e.g., a canvas that allows interactive drawing -- the text editor can extended that operates on "the character at the point" by operating on that underlying object in ways that also update the displays of that interactive object.

In such a system, a drawing editor could be represented as, for example, a single character document with an editable drawing as the "underlying object" property of that character. The advantage of this is that every such application becomes composable with others in simple ways, inherited from the underlying text editor. For example, an object could be constructed from a drawing editor (represented as single character with suitable underlying object properties) plus a second character with, let's say for example, an underlying project that is a spreadsheet. In the usual Emacs style, command sets can be built up so that as the spreadsheet changes, editable images defined as graphs of the spreadsheet can be automatically updated using something like hooks.

Seemed to work well ages ago. Not sure why it wouldn't now. It's already kinda-sorta possible to do that in Emacs today but the volunteers haven't yet developed that capability very far. Some, yes! but not all that far compared to classic lisp machine emacsen.

This strikes me a bit like the following logic. "This screwdriver is no better than a hammer."

They're different. Saying structure editors are multi-media editors is like saying Int32 is an Object in C#.

I totally agree. In fact Emacs is doing exactly what Thomas described. Technically you can do everything -- Emacs also don't have native drawing primitive but it supports using a SVG as the character substitute image, technically you can also do everything by creating SVG at runtime. I've attempted to build a music notation program using a SVG in a character, it was a totally nightmare, I have to reimplement all the cursor movements, selection, editing command etc, so if I'm reimplementing all these, why I do it in Emacs?

I think there are two kind of statements that often get confused: 1. A is equivalent to B (or A can do C) 2. A is equivalent to B shallowly (or A can do C naturally). The first kind of statement is almost always boring and trivial -- Turing machine is equivalent to untyped lambda calculus (in computation power), but who would want to code in Turing machine anyway. This applies similarly to Emacs. Technically you can deeply embed anything into Emacs -- like a 3D game engine -- but it's of little practical interest because it is usually unnatural and contrived, and moreover, a deep embedding doesn't benefit much from a regular and uniform model of the meta-environment. In my example, it turns out that I have to reimplement all those editing command separately from Emacs' own and need even more machinery to let them communicate.

Following the above spirit, my motivation was exactly to refine Emacs' model so that more stuff can be done in it shallowly/naturally (and maybe also make the model itself more uniform).

Separate two concerns.

On the one hand, there is a "make or take" decision: should one take Emacs off the shelf and extend it, or, seeing no third option, write something to use instead of Emacs. In the abstract, without regard to a specific project in which I am involved, I can't even begin to solve that "make or take" question.

Suppose though, that the answer is "make" - not use emacs but build some clean new thing to do the job. OK - what style should it's architecture be?

Here I have an answer: the architecture should be Emacs. Not literally any existing implementation - but the architectural style all Emacsen have in common, perhaps with some fresh attention to opening up possibilities for display, and handling multiple threads.

The Emacs architecture has qualities that evoke words like those in the subject line: adaptability, parsimony, composability, extensibility, ...

Emacsen abstractions like frames and windows, buffers that can be displayed in more than one place at a time, mark stacks, kill rings, dynamic hierarchies of keybindings, a main event loop based at turning input events into calls to procedure rather than message sends, very carefully designed mechanisms for temporal and contextual (context within the underlying data) interaction modality --- these and more are all very harmoniously and parsimoniously elaborated in emacsen and their absence in 99.99% of interactive programs (with no clever alternatives in those programs, either) is tragic.

The xi-editor does not natively target tree document structure, but it represents strings using rope, which have tree structure, and IIUC it does allow plug-ins to use the rope's tree structure in their document tree. Levien knows a lot about text representations, via work at Google on projects on fonts and the Android text stack.

Levien does not seem to be actively working on xi-editor at the moment, but the codebase, while not feature-complete, is functional and the design ideas are interesting.

See https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.html