Emacs modes: what is it?

I've been using Emacs as my OS since time started and I've always taken its programming model as granted, part of the nature. However, now I'm looking back at it when I'm trying to design an editor/OS based on S-exp rather than text, I found I understand Emacs very poorly.

So here's the question:
What exactly are mode, buffer local variables, hooks and advices?

Hooks and advices look like AOP. However, emacs hooks and advices usually make heavy use of buffer local variables. Lots of them also interact with modes by looking at the mode variable. So I don't think AOP captures the full picture.

Sure, mode looks like context-oriented programming (see ContextL LtU thread ). However I never see mode associated with dynamic scope, they just got turned on or off (for a particular buffer). Is this "resembling ContextL" impression just another instance of fitting a too-general concept into a much more specific (but not understood) concept?

And finally, let me give the context that all those question arises: I want to clean up the Emacs model, and apply it to a S-exp (tree) based editor. So the practical question is: Can we find a cleaner/more elegant version of the Emacs model, and generalize it to tree document structure? Will we have node-local variables, node-local modes, etc? If so, how will all those "attachments" interact between, say, parents and children nodes?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Think of a simple model, not an analog

To model the environment of an elisp form, you could go with:

  1. A pointer to a structure permitting lookup of global variables, of which there will be one per elisp bytecode interpreter and
  2. A pointer to a structure permitting lookup of buffer local variables, of which there will be one per buffer; then
  3. Mode can be represented as a special entry in structure #2 and
  4. Hooks are lists of thunks bound to a regular global variable in structure #1.

I'm not familiar with advices, but from eyeballing some docs they look like a source artefact transformed away when the relevant macros are expanded. The relevant macros I suppose are globally scoped forms.

This essentially just

This essentially just describing an implementation of exactly the Emacs model. And because of course I write everything in Lisp, implementation is trivial :)

I'm not satisfied by this because I don't understand *why* Emacs need to be designed (or happen to become) this way. Moreover, the Emacs model is overly specific (we need a Boolean variable for every mode. Mode itself is just a function to turn on/off mode but it has to perform a specific ritual to be "well behaved". Functions need to perform the ritual of checking advices), which make me hardly believe it is some design from first principle, rather than some accidental artifact resembling an unknown design from first principle.

understanding and using emacs lisp

1. Emacs grew out of an architecture that predates GNU emacs and that resulted from beginning with an interactive full-screen (terminal) text editor and making it customizable, extensible, and self documenting. Many of its features are the result of decades of thinking and experimenting with the intersections of how such an editor works internally, how users experience this, how extension writers organize things so that multiple, separately written extensions can all co-exist without excessive complexity.

Therefore, the best way to learn why the features are what they are is to study existing code while trying to write more. And keep in mind the metaphor: Why is a doorknob the size it is? You can't answer that question without asking questions about the human hand. Similarly, you can't answer questions about why Emacs Lisp is as it is without asking questions about the problems Emacs tries to solve (e.g., still works nicely on a terminal) and about the human factors, especially users.

2. If you want to write a s-exp structure editor, I can't imagine a better way to get started than writing one in Emacs lisp. It is likely to be fairly easy once you get the fundamentals done. For example, node-specific modes, minor-modes, keymaps, properties, etc. are all easy to do with the existing facilities of GNU Emacs.

Happy Hacking.

The Craft of Text Editing; or, Emacs for the Modern World by CF

Just read the book 'bout Emacs. I believe the answers you seek are in there, my child. https://www.finseth.com/craft/

More useful than that

This essentially just describing an implementation of exactly the Emacs model. And because of course I write everything in Lisp, implementation is trivial :)

This is just describing some data structures. Yes, they are ones I think will be closely analogous to the ones used in the Elisp VM, but they could easily be implemented in any general-purpose programming language.

This thread has me pondering the idea of writing a Nanopass compiler for a subset of Elisp, targetting something fairly high-level and dynamic but very unlike Elisp bytecode+VM.

A more elegant Emacs

an we find a cleaner/more elegant version of the Emacs model, and generalize it to tree document structure?

Back in 2010, I sent Tom Reps an e-mail asking why the Synthesizer Generator failed to become a commercial success. Let's just say he was profoundly offended by that question, and refused to entertain my inquiry other than to say I was sadly mistaken that it was not a commercial success. I wouldn't be shocked if he plonked future emails from me, since he didn't reply to further questions.

I think if you are looking for a more elegant version of text editing that has been generalized to tree document structure, Reps' PhD thesis is your starting point.

Other places to look into include Stratego XT managed by Eelco Visser.

Other interesting ideas: First-Class Copy and Paste, paper by Jonathan Edwards.

https://gtoolkit.com/ - Glamorous Toolkit, which independently re-discovered the benefits of first-class copy and paste.

And various Smalltalk IDEs developed and/or inspired by Dan Ingalls for various Smalltalk-like languages.

lisp machine experience

Structure editors are multi-media text editors, nothing more.

By "multi-media text editor" I mean an editor where every character has a (usually empty) property list, the property list can contain a reference to an underlying non-character object, and the display of characters can change to reflect that underlying object (e.g. an embedded image for a gif object).

If an underlying object is editable in ways not usefully represented as text editing commands -- e.g., a canvas that allows interactive drawing -- the text editor can extended that operates on "the character at the point" by operating on that underlying object in ways that also update the displays of that interactive object.

In such a system, a drawing editor could be represented as, for example, a single character document with an editable drawing as the "underlying object" property of that character. The advantage of this is that every such application becomes composable with others in simple ways, inherited from the underlying text editor. For example, an object could be constructed from a drawing editor (represented as single character with suitable underlying object properties) plus a second character with, let's say for example, an underlying project that is a spreadsheet. In the usual Emacs style, command sets can be built up so that as the spreadsheet changes, editable images defined as graphs of the spreadsheet can be automatically updated using something like hooks.

Seemed to work well ages ago. Not sure why it wouldn't now. It's already kinda-sorta possible to do that in Emacs today but the volunteers haven't yet developed that capability very far. Some, yes! but not all that far compared to classic lisp machine emacsen.

Hmm...

This strikes me a bit like the following logic. "This screwdriver is no better than a hammer."

They're different. Saying structure editors are multi-media editors is like saying Int32 is an Object in C#.

I totally agree. In fact

I totally agree. In fact Emacs is doing exactly what Thomas described. Technically you can do everything -- Emacs also don't have native drawing primitive but it supports using a SVG as the character substitute image, technically you can also do everything by creating SVG at runtime. I've attempted to build a music notation program using a SVG in a character, it was a totally nightmare, I have to reimplement all the cursor movements, selection, editing command etc, so if I'm reimplementing all these, why I do it in Emacs?

I think there are two kind of statements that often get confused: 1. A is equivalent to B (or A can do C) 2. A is equivalent to B shallowly (or A can do C naturally). The first kind of statement is almost always boring and trivial -- Turing machine is equivalent to untyped lambda calculus (in computation power), but who would want to code in Turing machine anyway. This applies similarly to Emacs. Technically you can deeply embed anything into Emacs -- like a 3D game engine -- but it's of little practical interest because it is usually unnatural and contrived, and moreover, a deep embedding doesn't benefit much from a regular and uniform model of the meta-environment. In my example, it turns out that I have to reimplement all those editing command separately from Emacs' own and need even more machinery to let them communicate.

Following the above spirit, my motivation was exactly to refine Emacs' model so that more stuff can be done in it shallowly/naturally (and maybe also make the model itself more uniform).

words like: adaptability, parsimony, composability, extensibilit

Separate two concerns.

On the one hand, there is a "make or take" decision: should one take Emacs off the shelf and extend it, or, seeing no third option, write something to use instead of Emacs. In the abstract, without regard to a specific project in which I am involved, I can't even begin to solve that "make or take" question.

Suppose though, that the answer is "make" - not use emacs but build some clean new thing to do the job. OK - what style should it's architecture be?

Here I have an answer: the architecture should be Emacs. Not literally any existing implementation - but the architectural style all Emacsen have in common, perhaps with some fresh attention to opening up possibilities for display, and handling multiple threads.

The Emacs architecture has qualities that evoke words like those in the subject line: adaptability, parsimony, composability, extensibility, ...

Emacsen abstractions like frames and windows, buffers that can be displayed in more than one place at a time, mark stacks, kill rings, dynamic hierarchies of keybindings, a main event loop based at turning input events into calls to procedure rather than message sends, very carefully designed mechanisms for temporal and contextual (context within the underlying data) interaction modality --- these and more are all very harmoniously and parsimoniously elaborated in emacsen and their absence in 99.99% of interactive programs (with no clever alternatives in those programs, either) is tragic.

Levien's xi-editor

The xi-editor does not natively target tree document structure, but it represents strings using rope, which have tree structure, and IIUC it does allow plug-ins to use the rope's tree structure in their document tree. Levien knows a lot about text representations, via work at Google on projects on fonts and the Android text stack.

Levien does not seem to be actively working on xi-editor at the moment, but the codebase, while not feature-complete, is functional and the design ideas are interesting.

See https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.html