Home » forums » LtU Forum

InterState: A Language and Environment for Expressing Interface Behavior

An interesting paper by Oney, Myers, and Brandt in this year's UIST. Abstract:

InterState is a new programming language and environment that addresses the challenges of writing and reusing user interface code. InterState represents interactive behaviors clearly and concisely using a combination of novel forms of state machines and constraints. It also introduces new language features that allow programmers to easily modularize and reuse behaviors. InterState uses a new visual notation that allows programmers to better understand and navigate their code. InterState also includes a live editor that immediately updates the running application in response to changes in the editor and vice versa to help programmers understand the state of their program. Finally, InterState can interface with code and widgets written in other languages, for example to create a user interface in InterState that communicates with a database. We evaluated the understandability of InterState’s programming primitives in a comparative laboratory study. We found that participants were twice as fast at understanding and modifying GUI components when they were implemented with InterState than when they were implemented in a conventional textual event-callback style. We evaluated InterState’s scalability with a series of benchmarks and example applications and found that it can scale to implement complex behaviors involving thousands of objects and constraints.

By Sean McDirmid at 2014-07-23 04:14 | LtU Forum | previous forum topic | next forum topic | other blogs | 22767 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Thanks for posting this

This paper contains a lot of ideas that resonate with my own slowly-evolving thinking about how to express composable interactive behaviors.

I like how their use of (dynamic, prototypical) inheritance ends up blurring the distinction between structural (what it is) and behavioral (what it does) facets of an agent. I also like that their model accepts that composition of concurrent behaviors will inevitably lead to contradictory constraints, and thus treats disambiguation (the left-to-right precedence rule, in their case) as a fundamental feature.

Existing systems for 3D graphics/games animation already pretty much accept that latter point (that composition of concurrent behaviors amounts to blending different layers) - see almost any nonlinear animation system, or Ken Perlin et al.'s Improv paper. Component-based engines/editors like Unity also tend to show the former (blur the line between structural/behavioral components by treating them uniformly; allow the structure/behavior composition to be modified on-the-fly). My biggest complaint about the paper would probably be that it doesn't compare to the state of the art on the 3D/games side of the fence.

By Tess Foley at Mon, 2014-07-28 15:32 | login or register to post comments

As I am oft to mention,

As I am oft to mention, Rodney Brooks' behavior oriented programming also supports blending via prioritized reflex-like behaviors, this was successfully used in Kodu.

It is difficult to cite the 3D engine stuff because their feature sets aren't well published. We really tried in the déjà vu paper but didn't get very far.

By Sean McDirmid at Tue, 2014-07-29 10:49 | login or register to post comments

Yes, I should read more Brooks

What little I've read has been really valuable - the subsumption architecture helped to orient my thinking a lot. I'm sure the robotics folks are just shaking their heads when graphics folks like me talk about the "state of the art."

Yes, Kodu (and YinYang) are more examples of systems that I would say "get it" with respect to prioritization/layering being needed for simple composition/concurrency. My sense is that this whole family of techniques scale to large programs much more easily than the nominaly more "pure" and composable FRP.

I agree that it is basically impossible to comprehend (much less cite) the state of the art in the space of commercial game tools. The main thing I'd argue for is setting the bar appropriately. I should actually give InterState much more credit for focusing on the GUI/IDE and live editing experience.

By Tess Foley at Tue, 2014-07-29 16:35 | login or register to post comments

Pointers?

What would be a good starting point to learn more about Brooks's work? Also is there something beyond the fact that Kodu is a rules-based system that lends itself to composition? It almost sounds like there's some sort of mechanism for explicit prioritization/layering?

By ibdknox at Tue, 2014-07-29 17:17 | login or register to post comments

The paper that I was

The paper that I was thinking of in particular was "A Robust Layered Control System for a Mobile Robot" (link). There are of course many more recent papers on Brooks' publications page, but a lot of it builds on the same fundamental approach of the subsumption architecture.

On Kodu, see The Design of Kodu: A Tiny Visual Programming Language for Children on the Xbox 360. Section 3.6 talks about the prioritization scheme (rules higher in the list take priority over others), and points out the inspiration this takes from Brooks.

The point, I'd claim, is that there are lots of different kinds of composition we might define, and the model for composition will in turn influence the kinds of components we can express. The point of the subsumption architecture is that while pure "wiring" (at which FRP excels) might technically be a fully general compositional model, it tends to break down in cases where layering excels.

Edit: as another example, the more procedural-imperative "script" model in something like Scratch naturally excels at expressing sequential composition ("do A then B").

Edit Again: Poking around Brooks' page, I'm also really enjoying the Behavior Language User's Guide.

By Tess Foley at Tue, 2014-07-29 20:02 | login or register to post comments

Kodu programs are expressed

Kodu programs are expressed as rules, each rule has a priority based on its position in a rule sheet. When two rules require the same actuator (say movement), the lowest priority rule yields and it's later clauses aren't evaluated (the clauses before the actuator remain executing). So a low priority "patrol" behavior yields when a higher priority "pursue" behavior becomes active.

This is quite powerful and allows one to easily build robot actors with fairly sophisticated emergent behaviors. Composability comes from this resource-based implicit coordination, which appears to be appropriate for robotics and robot-like domains (say video games).

By Sean McDirmid at Tue, 2014-07-29 23:23 | login or register to post comments

Biology!

Sounds a lot like how cellular processes (transduction, etc) take place in the body. Tuple spaces could be related as well if you modeled the behaviors as data, but they didn't take off as far as I can tell.

It seems like there's a number of people starting to look into emergent behavior (or self-assembling) systems in industry. This netflix post, as an example, talks about using a self-assembling pipeline to do their playback system.

We're sort of at the beginning of trying to understand how composition will work in our case (rules-based temporal logic language on top of a database), but increasingly I'm convinced it'll look something like this stuff. Sean, I know you went down this road with yinyang - any insights?

By ibdknox at Wed, 2014-07-30 17:54 | login or register to post comments

Brooks' work is very

Brooks' work is very biological inspired. You can read about my similar design in the "coding at the speed of touch paper," but it's quite OO. I can't seem to come up with a design of the paradigm that works beyond robotics/games; it might not be general, which is why I'm currently working with Glitch/managed time to auto coordinate. Behaviors can then be implemented on top of glitch, which uses priorities anyways in breaking conflicts. Tuple spaces are first come first served, I think, which isn't good enough in this case (low priority behaviors must yield to higher priority ones).

By Sean McDirmid at Thu, 2014-07-31 02:06 | login or register to post comments

all broken links?

is it me or are there all broken links in the paper? i wanted to play breakout!

By raould at Mon, 2014-07-28 23:09 | login or register to post comments

not just you

broken for me as well.

By ibdknox at Tue, 2014-07-29 04:54 | login or register to post comments

There might be an embargo

There might be an embargo until the conference (October), but you could also contact the authors to see if you can get access.

By Sean McDirmid at Tue, 2014-07-29 10:52 | login or register to post comments