Lambda the Ultimate

I never heard Reppening use tactile programming to describe this, so I'm assuming this was a comment/opinion sneaked in by whatever editor of the tactile programming language wikipedia page. Tactile programming is not a common enough term to be very notable yet, but it is a nice name for this area. As far as I know, nobody in the VPL research community is looking at this problem right now. I would be very interested if there was anyone...

I wonder if a slate programming language can be close to as efficient as a keyboard-based language. Perhaps the syntax is still "free-form", just not free-form text. So we could use tiles but place them freely in a 2D space with overlapping and such to build programs. Unfortunately, I don't have anything else to compare to; e.g., there are no word processors that allow one to efficiently compose reports on a tablet without being key typing centric (perhaps use speech). Speech and touch are an interesting combination to consider.

Adapting Common Lisp's SLIME interaction protocol so that any values displayed are directly inspectable as objects rather than just printouts.

If you like SLIME, you may want to look at Symbolics Genera & Garnet. Genera is generally considered to be the better of the two in terms of "directly inspectable as objects".

I am a licensee, owning a MacIvory II and having, uh, experimented with the virtual machine lately. So, yeah, I'm all over that! It inspired a lot of Slate's "Chalk" UI design document which basically tries to adapt Morphic-style direct manipulation with CLIM-style direct manipulation.

But I have read comp.lang.lisp archives, and posts such as this one.

Seriously, who would want to program by touching 'slime'?

[above the age of 8, I mean]

Clearly, a more marketable/palatable term could be used.

I liked "Chalk" in the context of Slate, but of course people don't like touching that with their fingers, either. And "Paint" has artistic connotations.

In any case, a successful "touch" UI has to just appear to afford exactly what the user thinks it should do, and not worry about metaphor until users really get involved with it.

I think there's an angle towards combining autocompletion with a form-based GUI, where a form is simply something that can be displayed graphically (typically as a rectangle, nested within other forms, and having sub-forms).

You'd type the beginning of a function/operator name, and choose one from the autocompletion list. This would add a custom form (widget) for that operator to the source. The form would then have a custom GUI for interacting with (parameterizing) that operator invocation/function call. For example, a function call to a function search could have GUI knobs and options to set parameters etc.

I find the Lisp connection interesting here, too. In Lisp, expressions are already called forms, and are clearly demarcated, analogous to widget borders in GUIs. (Of course, outlining, which seems to pop up in every attempt to make source code more "hyper" - e.g. in the Dylan tools, and UserLand Frontier - goes back all the way to InterLisp. And InterLisp was where Dave Winer first saw outlining, which led to Frontier, which was the software on which the original Lambda the Ultimate ran. Lovely, huh? ;))

I remember reading about it before but had not considered it in this context.

Presumably, one single gesture could trace out a "word" (or phrase of keywords?). And the lexical path space could be context-dependent.

One could prototype such a thing using JS itself using one of the multi-touch abstraction layers in development. I can't figure out if I should spend an afternoon on this myself, but it'd be good if someone could put it together to see if there's a feasible usage mode.

Drag-and-drop is horrible according to Fitts' law, and totally inappropriate for touch (its not that great for the mouse either). Hopefully, whatever comes out has decent UX design.

I can't see why drag-and-drop would be horrible according to Fitts' law. Are you making some kind of assumption about the size of the target area that you could expand upon a little?

Three fundamental problems with drag and drop:

1. The distance from where you are dragging to where you are dropping can be long.
2. The palette used to drag from consumes valuable screen real estate even when it is not being used.
3. Because you you select the value (to drag) before you select the input site (where you drop), you cannot take advantage of any context at the input site.

Problem 2 can be mitigated by hiding the palette when not in use, which is what you see on tablets: you have a small button that you tap to bring up the palette when you want to drag something from it. This can mitigate problem 1 somewhat if the button is placed relative to the screen rather than physical space (when you move around space, the button is always in the same place). On a tablet, this is almost acceptable, except if you have a large number of options and scrolling is involved in the palette, which is pretty slow and will never become much faster as the programmer learns the system.

However, on a phone-sized display, you don't really have enough space to materialize a decent-sized palette and still expect to have your intended drop site still visible. You could hide the palette once you've selected what to drag, but then you still have the scrolling palette problem exacerbated because the screen is much smaller. I don't think you see much drag-and-drop in phone apps because it is fundamentally unworkable.

The big problem with drag-and-drop is 3, the inability to leverage context. If you know about the context on where you are going to drop, you can have a concise palette with options that are only applicable to that drop site. But since you choose the value to drag first, the computer just doesn't know and the program has to do all that filtering themselves. Now, if you are talking about a dynamically typed language, maybe there isn't much context that the computer can know about, but with a statically typed language this is very serious.

Context menus don't have any of these problems: you tap on the input site first to bring up a menu. You can make the menu hierarchical to avoid scrolling and allow programmers to input faster as they learn the system (because they can eventually navigate the hierarchy faster). On the other hand, context menus have less directness than drag-and-drop, so there is a usability tradeoff to be made.

I'm currently working on a paper that talks about this, I'll post something in about a week or so that touches on this.

Wouldn't it be feasible to touch a pair of elements and raise a relationship-oriented context menu?

I've seen some work done with a 'hand' metaphor rather than a 'mouse' metaphor. You can tag items, group them, and perform actions that relate elements within the group or to some external element. Strategy games use this approach a great deal, and a clipboard is a related concept.

You could drag one element to another and then activate a context menu related to what you could do with those two elements. But those elements have to be close to each other, on the same screen even, for that to be efficient.

I didn't say anything about 'dragging'. Just touching.

Ah, I found the initial reference that got me interested in the hands metaphor - Hand vs Pointer [c2].

Very interesting, thanks for the reference!

I can see the potential of a hand, for touch we could create slots in the chrome that you could place things in to build up hand context. It does seem a bit more complicated, but I'll have to think about it more.

That's a pretty detailed analysis. Your paper sounds interesting, I'll look forward to seeing a link.

I'll post something in about a week or so that touches on this.

Nice pun!

Any progress on that paper? I'm very interesting to read more. You're bringing up some good issues -- although palette size issues are less of a concern on tablet devices, and size of palette can depend a lot on syntactic complexity of the language. Anyway, I'd love to read more; and especially to hear what you think a *good* touch-oriented programming interface might be like.

(We've had good experiences with TurtleArt/TurtleBlocks -- which isn't touch-oriented, but is definitely palette oriented. Etoys and Scratch have also been very successful; both palette-oriented.)

The submit is due Friday, so I'll post it after that.

I do think EToys and Scratch have been successful in their targeted domains, but not beyond that. They lack an efficiency that programmers tend to get only with keyboard languages. Getting back that efficiency on a touch-based device is one of my goals.