Lambda the Ultimate

inactiveTopic User-defined functions in Excel
started 12/1/2002; 6:49:44 AM - last post 12/9/2002; 4:56:08 AM
Ehud Lamm - User-defined functions in Excel  blueArrow
12/1/2002; 6:49:44 AM (reads: 4555, responses: 16)
User-defined functions in Excel
Improving the world's most popular functional language: user-defined functions in Excel. Simon Peyton Jones, Margaret Burnett, Alan Blackwell. 10 pages, November 2002. Submitted to PLDI 2003.

We describe extensions to the Excel spreadsheet that integrate user-defined functions into the spreadsheet grid, rather than treating them as a "bolt-on". Our case study is unusual in that it highlights the way that programming language insights can be applied to a product not normally considered as a programming language.

Obviously related to end-user programming, programming by demonstration etc.

Excel is quite an innovative product (flame on!), it would be interesting to see if the work presented in this paper ever appears in the product.


Posted to DSL by Ehud Lamm on 12/1/02; 6:50:13 AM

Ehud Lamm - Re: User-defined functions in Excel  blueArrow
12/3/2002; 1:39:58 AM (reads: 3344, responses: 0)
Notice that sction 4.6 is about debugging. At first it seems that debugging in Excel should be as problematic as debugging lazy functional programs. But spreadsheets have one huge advantage: the source form is the execution form. Debugging is integrated with programming.

Noel Welsh - Re: User-defined functions in Excel  blueArrow
12/3/2002; 3:36:21 AM (reads: 3358, responses: 1)
I raise you "Spreadsheets for Images"

http://graphics.stanford.edu/projects/spreadsheets/

Ehud Lamm - Re: User-defined functions in Excel  blueArrow
12/3/2002; 4:26:52 AM (reads: 3395, responses: 0)
Sounds interesting. It is sad that the spreadsheet paradigm isn't being actively explored. One interesting product, that is now dead as far as I know, was Lotus Improv.

Isaac Gouy - Re: User-defined functions in Excel  blueArrow
12/3/2002; 9:33:57 AM (reads: 3311, responses: 1)
One interesting product
Xerox Analyst "an incredible spreadsheet that could display images, conjugate Russian verbs."

Ehud Lamm - Re: User-defined functions in Excel  blueArrow
12/3/2002; 10:48:20 AM (reads: 3329, responses: 0)
But how did these products extend the spreadsheet programming model?

Anton van Straaten - Re: User-defined functions in Excel  blueArrow
12/3/2002; 11:43:45 AM (reads: 3291, responses: 1)
I think the spreadsheet paradigm is really a subset of some sort of "active document" paradigm, which covers more free-form products like Mathematica & MathCAD, as well as even HTML pages (a stretch right now), Microsoft's COM/ActiveX/.NET, etc.

Products like Mathematica have been successful in their niche, but more general-purpose user-editable active documents seem to have faced big usability barriers. Part of the problem may be identified in the above paper - the "trapdoor approach", in which actual customization of a "document" must take place in traditional programmer-land, requires significant investment on the part of the user to learn and use. This turns creating active documents into a traditional programmer's task, effectively off-limits to the average user, who is restricted to things like the "dumb" embedding of external objects in e.g. a Word document.

If users could, spreadsheet-like, easily click on an object/component and, using a language of comparable simplicity to Excel's formulae, relate it to other objects on the same page or on other pages, it would represent a huge advance in end-user programming. One could imagine (some) users creating their own user interfaces to complex systems which provide components that enforce some basic level of sanity.

This is nothing new, conceptually - it's a variation of the same component vision that has been thrown around at least since the early days of OpenDoc and OLE, if not since Vannevar Bush's conceptualization of hypertext systems in 1945. Although systems like this are theoretically possible now, they still don't usually rise to the level of being particularly end-user-programmable, and certainly aren't common. Customizable user home pages at portal sites might be the most ubiquitous, if impoverished, example.

Peyton-Jones et al may be onto something important here in terms of the usability of such systems. Excel is easy to reason about because it's lazy functional. As soon as you insert imperative components that interact in arbitrary ways with the rest of a sheet, mutating cell values for example, the user's ability to reason about the behavior of the sheet, in all but the simplest cases, is severely compromised.

So, we need to start developing lazy functional components that interact with their host environment in a pure functional and highly transparent way. QED. ;)

Ehud Lamm - Re: User-defined functions in Excel  blueArrow
12/4/2002; 12:18:19 AM (reads: 3278, responses: 0)
it's a variation of the same component vision that has been thrown around at least since the early days of OpenDoc and OLE

In a sense. But the important thing about spreadsheets is the execution/programming model they provide (based on recalculation). This is what makes spreadsheets more than just a collection of cells.

rev - Re: User-defined functions in Excel  blueArrow
12/4/2002; 3:38:52 PM (reads: 3226, responses: 0)
Something I've always wanted was the merging of a system like LaTeX with the abilities of a spreadsheet. Eventually, I'll write a LaTeX pre-processor in Squeak that imparts these abilities. That is, in a LaTeX table, be able to refer to cells, do math or arbitrary calculation/computation.

I've long been interested in the spreadsheet as a way to do programming, especially when not writing an app per se. I don't know how to create new functions for StarOffice (but it does have a VB-macro like language), but I could always learn that. For Dynapad (http://dynapad.swiki.net), I plan on creating a spreadsheet app that is very easy and natural to extend using Smalltalk.

One thing I"ve been trying to figure out lately is how to extend the Newton Spreadsheet, QuickFigure for NewtonWorks, with user-defined formulas written in NewtonScript. I found the frame (in NewtonScript speak, a frame is a Lua-table... a hash-table-like structure that make up NS objects) that contains the functions, but haven't been able to add anything to it. This would be a great tool to have as a person who knows some NewtonScript. :)

Anton van Straaten - Re: User-defined functions in Excel  blueArrow
12/5/2002; 12:27:26 AM (reads: 3174, responses: 1)
But the important thing about spreadsheets is the execution/programming model they provide (based on recalculation). This is what makes spreadsheets more than just a collection of cells.

I agree, but I'm suggesting that perhaps this model might be extended beyond a grid mainly consisting of numbers, to collections of components that are programmed, and whose state is exposed, in similar fashion.

Of course, I can only provide a detailed description of this in person, since it requires quite a lot of waving of hands....

Ehud Lamm - Re: User-defined functions in Excel  blueArrow
12/5/2002; 12:52:33 AM (reads: 3226, responses: 0)
Ha!

Isaac Gouy - Re: User-defined functions in Excel  blueArrow
12/5/2002; 7:29:33 AM (reads: 3156, responses: 0)
Well that made me laugh ;-)

this model might be extended beyond a grid mainly consisting of numbers, to collections of components that are programmed
Essentially that's what Xerox Analyst did all those years ago. I believe Analyst let you write image-transformation cell-formulas, in the same way that you write numeric cell-formulas. You could have a row of cells containing images that were progressively transformed.

(Somebody out there knows how it really worked - I don't)

Noel Welsh - Re: User-defined functions in Excel  blueArrow
12/5/2002; 8:47:25 AM (reads: 3151, responses: 0)

Reading the Wikipedia entry on Lotus Improv I was struck by the similarity it holds to certain other systems. Basically the Improv revolution was to allow you to name cells and then perform calculations using those names rather than some arbitrary locations. Does anyone else know a programming system that lets you bind names to values and express calculations in terms of those bindings? ;-)

This is not to say there aren't cool things you could do with a spreadsheet model but I think the PL theoretic aspects are quite simple and its the visual representation that is a large part of the power.

Stuart Allie - Re: User-defined functions in Excel  blueArrow
12/5/2002; 3:12:34 PM (reads: 3147, responses: 0)
There's a spreadsheet for linux called SIAG (Scheme In a Grid) that uses scheme - among other languages - as its formula/extension language. Perhaps the "killer app" for functional languages is as extension languages for widely used apps. AutoLisp springs to mind as one commercially succesful use. Mind you, the Autolisp manual includes examples like:

(setf a (getpoint) setf b (getpoint) add-line (a b))

instead of (add-line ((getpoint) (getpoint))

so it's not exactly a shining example of good functional programming... [okay - i cant remember Autolisp or Lisp well enough, it's been >10 years since I used either, but you get the idea I hope!]

Anyway, if we could get something like scheme, or haskell, or [insert favourite language here] in use by piggy-backing on top of a widely distributed app, that'd be good. Wouldn't it?

I just remembered - OpenOffice has java embedded in it in some way, perhaps the functional lang community should consider sneaking a good language in alongside java?

Just thinking out loud really.

Jörg F. Wittenberger - Re: User-defined functions in Excel  blueArrow
12/6/2002; 10:00:46 AM (reads: 3092, responses: 0)
I feel I've seen that before...9 years ago... in teapot!

http://www.moria.de/~michael/teapot/

(3rd paragraph after the feature listing)

Isaac Gouy - Re: User-defined functions in Excel  blueArrow
12/6/2002; 11:03:34 AM (reads: 3080, responses: 0)
by piggy-backing on top of a widely distributed app
This disruptive technology / trojan horse theme seemed to come up a couple of times at the LL2 presentation. You may enjoy the disruptive technology slides.

OpenOffice has java embedded in it in some way, perhaps the functional lang community should consider sneaking a good language in alongside java?
To follow the experience reported in user-defined functions in Excel wouldn't it be better to extend the OpenOffice formula language rather than the OpenOffice macro language?
Rather than "sneaking a good language in" their design is ruthlessly driven by principles of user-centered design.

bryan rasmussen - Re: User-defined functions in Excel  blueArrow
12/9/2002; 4:56:08 AM (reads: 2988, responses: 0)
>I just remembered - OpenOffice has java embedded in it in some >way, perhaps the functional lang community should consider >sneaking a good language in alongside java?

It offers more than that: http://udk.openoffice.org/