Why a diagram is (sometimes) worth ten thousand words

Jill Larkin and Herbert Simon, Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 1987.

The advantages of diagrams, in our view, are computational. That is diagrams can be better representations not because they contain more information, but because the indexing of this information can support extremely useful and efficient computational processes. But this means that diagrams are useful only to those who know the appropriate computational processes for taking advantage of them. Furthermore, a problem solver often also needs the knowledge of how to construct a "good" diagram that lets him take advantage of the virtues we have discussed. In short, the ideas we have presented, not only provide an explanation of why diagrams can be so useful, but also provide a framework for further study of the knowledge required for effective diagram use.

While this paper is not about programming languages, it describes research that may be relevant to the discussion of visual programming languages versus textual languages as well as other language design decisions. Seeing as human factors were recently mentioned, and that it seems to be cognitive-week here on LtU, I thought this paper might be of interest.

While I am sure this paper can be discussed from many angles, I'll only mention two. 1. While this paper is old, I still find it remarkable how old fashioned it seems. I am not sure exactly what causes it to feel this way; probably something in the writing style. 2. While I haven't searched the literature, I don't recall seeing research on programming languages that uses the framework suggested by this paper even though Simon's work was, of course, well known to computer scientists in general, and PL researchers in particular.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Old Fashioned?

I don't think that a paper written in 1987 is at all old fashioned. But I do think that the argument is unconvincing. Topological information can be carried by using algebra with a textural representation. This can be easy to parse and execute. But in general this is one very hary problem.

The paper Meaningful Modeling: What's the Semantics of "Semantics". gives some idea of how difficult language design and grphical language design in particular actually is.

To clarify: I did not mean

To clarify: I did not mean that the paper is old fashioned because of its date of publication. I said the paper sounds old-fashioned to me, and in fact meant to emphasize that this is remarkable since 1987 was not that long ago...

I know what you mean

I said the paper sounds old-fashioned to me, and in fact meant to emphasize that this is remarkable since 1987 was not that long ago...

I think I know what it is you are detecting, Ehud.

It is the feel of the days when people still believed that AI was on the verge of success...

:-)In fact I think it is

:-)

In fact I think it is something more subtle. I expected the AI vibe to be more worrying than it turned out to be. I guess my feeling has more to do with the broad perspective and intellectual courage that is often lost when disciplines mature...

diagramming as indexing?

Topological information can be carried by using algebra with a textural representation.

If I understand Larkin and Simon properly, they don't claim that diagrams necessarily carry any more information than sentential representations, just that they're a more convenient representation for searching, pattern matching, and drawing the resulting inferences — somewhat like the case of indexed tables in databases, which in theory are isomorphic to unindexed tables, but in practice may be much quicker.


In the representations we call diagrammatic, information is organized by location, and often much of the information needed to make an inference is present and explicit at a single location. In addition, cues to the next logical step in the problem may be present at an adjacent location. Therefore problem solving can proceed through a smooth traversal of the diagram, and may require very little search or computation of elements that had been implicit.

Separation?

The authors do a good job of extolling the virtues of diagrams and of explaining the importance of going back and forth between the diagram and the text to get a good representation of the problem. They then try to separate the visual (topology) and the text (algebra) and try to show that visual is better. This is a categorical mistake. Topology and algebra are joined at the hip, they are points of view about the same thing. You can't have it both ways.

Topologies and algebras

"Joined at the hip" doesn't make any difference to the human brain and its unconscious inference systems. We work with the wetware we have.

Also, I think it's helpful to remember that text itself is just another diagram - a more complex one with more edges and corners - until its parts are interpreted as algebraic ideals. For example, "h" is the height of the ball, but (*intuitive leap*) it's also just a real number, so I can do this and this and this with it...

That example illustrates why I don't think it's appropriate to say that diagrams are universally better than text. "Better" suggests a utility judgment. An AI professor of mine pounded into my head to always ask "For what?" when making those judgments.

I think a diagram is most useful at the beginning, to make something concrete. From there, it can serve as a representative of an "equivalence class" of similar problems, and from that you can make abstractions. But diagrams are terrible at abstraction (besides, say, lossy abstractions of real systems). Because of their concrete nature, you can only do it by leaving the varying part off to be put in later. As soon as you replace that "varying part" with a symbol - a box, an arrow, or whatever - it's a variable, you're halfway to algebra, and you might as well go all the way. For a lot of useful abstractions, you can retain the labeled diagram to leverage our brains' affinities for the concrete.