Home

Interactivity considered harmful

After reading many posts lauding interactive tools as an integral part of the next big thing in software development, I figured I could offer this as counterpoint. The paper Magic Ink: Information Software and the Graphical Interface very eloquently argues that most software today, especially information-intensive software (think IDEs and many other GUI-based PL tools) are really badly designed. The most memorable section subtitle being interactivity considered harmlful. This is a real treasure trove of wonderful design ideas for interfaces for information-rich applications.

This paper follows in the grand tradition of Edward Tufte, whose book The Visual Display of Quantitative Information was an incredible revelation for me.

Somehow, I do think that some of the ideas behind Intentional Software fit in here -- although I make no claim as to whether the actual implementation of those ideas is an appropriate realization.

By Jacques Carette at 2007-04-02 02:07 | Fun | General | Software Engineering | other blogs | 65684 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

eh

This seems like a general summary of what people have been thinking about for awhile. As far as I can tell, the author argues for context-sensitive information displays and very rich capabilities to design them, achieved through some sort of futuristic WYSIWYG design suite but not programming. I disagree on the last point - I view the code view as one incredibly malleable projection of a system (and while it is not reconfigurable/reorientable in practice, I think it should be).

The movie theater graphic (1/3d the way in) was an interesting example of the ambiguity of the author's desired end goal. Depending on context, the graphic is good or bad: the graphic is good for time and movie oriented searching, but abysmal for theater-oriented searching, which is an issue for those of us stuck with public transportation. He only presents one view, and it is unclear how to reorient the graphic without the worst-case usage of interaction. What was really neat to me is that this revealed the ambiguity of his desire.

The article was wordy so I skimmed a lot. Jumping to the recommendations, there wasn't much in terms of what wasn't understood coming into the article:

1. It's currently hard
2. People with talent do well
3. Education is important
4. We need better tools - the meat of what we can actually do (esp. as the LtU audience):
Simplicity: "For a platform to be reliable, it must either have a single implementation, or be so utterly simple that it can be implemented uniformly. If we assume a practical need for open, freely implementable standards, the only option is simplicity."
Generality: "a good platform must provide safe access to everything that is technologically possible".

This sounds like a push-pull, and is probably why much of the LtU audience is reading the article :) Animations, flow, context-sensitivity, consistency, and a security model (???) were the features that seem important to the author. That sounds like a contradiction. Additionally, the author seems to use graphic design as a hammer, but I should reread the article on that account.

I've been thinking about the area a lot recently. One relevant topic is data flow (MaxMSP, functional reactive programming, etc) and have been evangelizing it (http://www.flapjax-lang.org). FRP is a hot topic here, though I haven't seen anyone talk about IDE integration. The Flash authoring environment is good for simple animations, but if you look at anything sophisticated, it quickly becomes entirely programmatic. Oddly enough, I have better luck understanding a Java project than a Flash one. FRP has several nice properties that seem to beg for integration - I'm about to start the Ph.D. process and might look into it, but am curious if anybody else has thoughts there (Jonathan Edward's paper on Subtext comes to mind). Another branch of work I'm following is the several constraint programming and statistical learning projects from UW, with Dan Weld currently wielding the torch (http://www.cs.washington.edu/homes/weld/pubs.html). While FRP has been discussed here, I don't believe any of these projects have been. Dan Weld's projects force us to define what we mean by context-sensitive and propose concrete realizations. HaXe and Laszlo, with multiple compilation targets, seem like other relevant projects here.

By lmeyerov at Mon, 2007-04-02 04:22 | login or register to post comments

The usual caveats apply

Considering the paper isn't exactly a peer-reviewed publication, but free on the internet, the usual caveats about free advice apply. No doubt he is over-stating his case, but his examples does show at least one very important point: 2D graphic layout. For the things that involve a time, it's best to use one of the dimensions of the screen for that information, rather than giving a small button to click. I'm sure that observation and some good experimental evidence for faster task accomplishment, is worth a real paper. Very few places utilise the screen as a 2D canvas to present information.

Personally, I think all the stuff about context-sensitivity is far, far harder than he's making it out to be. Inferring semantics from textual snippets is basically trying to do type-inference on natural language. By the usual rule that anything a human can't do reliably, we can't get a computer to do at all, the proposed feat is pretty high up.

By genneth at Mon, 2007-04-02 09:02 | login or register to post comments

Check out some of my new

Check out some of my new SuperGlue work. I've integrated SuperGlue with an IDE to enable live programming, which still means that someone has to write code, but at least they can see the results of what they are doing while they are writing. I was wondering if this would be possible in an FRP language like FrTime or FlapJax.

By Sean McDirmid at Mon, 2007-04-02 09:51 | login or register to post comments

FlapJax

I didnt know a word about FlapJax. Im "playing" width the compiler and i really like it. I want to use it to my personal site of Songs, Letras de canciones.
Thanks in advance!

By brazukil at Mon, 2007-05-21 20:15 | login or register to post comments

FRP IDE integration

During my own course of experiments with FRP I too have wondered about IDE integration... what I envision is a graphical tool (much like LabVIEW or Max/MSP) allowing the programmer to create data flow graphs with arrows and boxes, running in parallel with another tool which allows them to graphically construct a GUI. Any widgets placed into the GUI are made available for use in the signal graph, and vice-versa (this is the same idea used in LabVIEW and schematic/layout capture tools such as Cadence). Individual signal nodes can have algorithmic code associated with them, in the programmer's favorite language, and can be edited the "old-fashioned" way (this is how schematic capture tools which support HDL code work, and I believe also LabVIEW and Max as well).

Now, thanks to the nature of FRP systems that can modify their own signal graphs, combined with a system such as MetaOCaml to compile the algorithmic code, we can go the SuperGlue route and make entire system live: add a button to your GUI, wire up its signals, click it, and watch what happens. Add support for an O'Caml-style time-travelling debugger (possible because the FRP system "knows" where all the state is kept) and the user will have a convenient way to test different changes from some known state without having to revert back to the initial state. The FRP system's state-omniscience also gives you state marshalling "for free", allowing programmers to share "buggy" states with each other, save a debugging session for later, etc.

By Chris King at Mon, 2007-04-02 15:09 | login or register to post comments

Random observations

I left that paper more convinced than previously than back-in-time debuggers were valuable. I also now really want a debugger that can dynamically create sequence diagrams for running methods, with pop-up annotations detailing what state got changed and clever decluttering. For extra bonus points, show me how the sequence diagram changes on subsequent runs of the method.

Applying history and context to IDEs is decidedly not new, either for code editing or code reading. I still feel the risk of ambiguity is too high to apply history and context to languages themselves, but could be convinced.

While I've certainly written my share of exciting and forward-thinking but vague non-credentialed manifestos, I now have a bit more sympathy for those who have chided me for it.

By Dave Griffith at Tue, 2007-04-03 15:30 | login or register to post comments

Ambiguous?

I still feel the risk of ambiguity is too high to apply history and context to languages themselves, but could be convinced.

Do you mean that a language based on a situation/action perspective is ambiguous? This must be allowed as a possibility because otherwise there would be no need for named action or situation. That is the action would be determinate and the situation would be a state space.

Programming with a situation/action perspective, or activity representation, must be a backtracking language to account for unexpected results. The situation/action perspective is goal oriented. A program is successful by reaching its goal not by executing a specific process.

By Hank Thediek at Tue, 2007-04-03 23:25 | login or register to post comments

accessibility

Graphics are great for the sighted. For myself, who is merely red-green color blind, the menu of color names for ordering flowers is much more useful than a strip of small colored flowers. That would be an obstacle to me in a user interface. I once had to take a picture of an LED with a digital camera and put it into Photoshop to determine whether it was red, yellow or green.

By James McCartney at Tue, 2007-04-03 16:26 | login or register to post comments

different or restricted perception

That creates a really interesting dilemma. It is better to create inclusive interfaces which can be used by as many people as possible. However, I think that it is also a mistake to optimize for the needs of the few at the cost of the efficiency of the many. To tie this in to PL - read The Development of Chez Scheme where K. Dybvig states his compiler-design philosophy (paraphrased) where "the advanced features have to pay their own way, and do not affect the cost of the common features".

I prefer interfaces with many different views to suit different needs. I am a strong non-believer in one-size-fits-all, especially when it comes to language design and IDEs. Of course, we are very far away from having the engineering body-of-knowledge available to create languages/IDEs with multiple views (back to the link with IP!).

1D ASCII is close to lowest-common-denominator (but not quite, as I know a brilliant programmer who is completely blind). Having a language/system which are extremely difficult to use by blind people is not good (and they exist: think Simulink; the underlying ASCII files are quite monstrous).

By Jacques Carette at Wed, 2007-04-04 00:36 | login or register to post comments

Did anyone notice...

The reference to LtU?

Movie redesign

Last film...

"Little Schemer"
An elephant journeys to find Lambda the Ultimate
Starring: Friedman & Felleisen, Car, Cdr, Cons, Cond

It's also the only film to not star Jet Li. Damn it... I wanted "Shaolin Lambdas"!

By genneth at Thu, 2007-04-12 09:37 | login or register to post comments