Lambda the Ultimate

I havent programmed in Haskell for a few years but the last time I looked, the way in which mutable state was handled still didn't feel quite right. Also, using more than one monad wasn't easy (eg IO + Maybe) - getting the definition of monad composition right was an effort in and of itself. Some propeller head on one of the Haskell newsgroups said he couldn't see what was difficult about it - which might be another things to add to the ToDo list - if most developers see it something complicated, it probably is, and needs to be addressed.

Also, with multicores having been in general use now for at least 5 or 6 years, I'm expectantly waiting to read any day now about some software shop that has switched from C++/Java to Haskell but nothing yet :-( so perhaps some work to still be done there?

I guess the question mark and fundamental thrust of my message were lost on you. Mark it down to your prolonged alienation.

So, apparently you don't care that I'm asking for a rhetorical correction around here rather than a content focus shift. Because the content is fine, I just don't want to discuss it in this hostile environment where everything that isn't building off of an FP type system is inherently worthy of derision.

The thing is - there is no hostile environment... It's all in your head! Believe me! Try it! So what if there are some FP zealots here? They are far from being the majority.

I have much more interesting and convivial discussions with language developers of various persuasions on Twitter (for example) than I've ever seen here.

I regret hearing that. But I am glad people are having convivial discussions of PL, wherever this happens.

I'm talking to the actual language developers, not academics commenting (or developers sparring rhetorically with an academic perspective) on language developers' works. It's much more rewarding to just talk about the work than to talk about presentations of the work with people who did not do the work or are not directly invested in the work.

Also, I don't get paid to write papers or work on languages - I get paid to meta-program *because* I do this, but I don't have the time / attention to present my work in an academically-palatable way, even if it is.

I don't think anyone here will look down on you for researching non-functional programming languages, as long as you're not researching object-oriented programming language.

Because my OO languages are mostly-functional languages, handling immutable ADTs, pattern-matching, multimethods for the OO crowd, everything is a lambda, no "this", etc. But that doesn't matter, because it doesn't put an elaborate type system above everything else and because I refer to things as "objects".

It's probably hard to paper over the fact that you don't have enough type machinery, but I don't see why you have to use the word "object". What's wrong with "entity-directed programming" or "stuff-related programming"?

Also, I consider OOP to be (multimethod)message-based, anyway.

I'm sorry that you feel alienated. I, for one, would love to read about your work on cutting-edge non-FP languages.

+1

But can you justify why ghc would be a good idea to integrate with a pre-existing RDBMS?

It is difficult to govern how much resources, or control over resource scheduling, should be given to ghc and how much resources should be given to the RDBMS, e.g. for managing concurrency. Modern DBMSes have user-land operating systems that aggressively manage resources for the actual OS. For this reason, environments like the SQL CLR do not encourage trying to spawn extra processes to get work done, because the CLR primitives are abstractions in terms of how the SQL OS sees things. You are better off following Mike Stonebraker's oft-misunderstood advice and building outward, treating databases as middleware.

Besides, Haskell might be the last functional language to want to pick for modeling databases. Laziness is very bad in this context! Very bad! I'd rather have Standard ML. And guess what, in purely functional database management, SML based languages are common.

Practically, shooting for small improvements in RDBMSes, most procedural languages need higher-order functions. Writing the system management bits like index maintenance, backup scripts, etc. is just all too painful and time consuming. Main DBMS vendors then build separate wizards with separate codepaths into the DBMS system to help DBAs. But this is really not a good solution, since it creates two subsystems for the same functional spec - doing index maintenance, backups, integrity checks, etc. Microsoft has wasted thousands of man-years on SQL Server trying to build solutions to this problem, simply because DBAs don't have higher-order functions. To illustrate the problem, look at Ola Hallengren's SQL Server community-award-winning maintenance scripts: http://ola.hallengren.com/ They're great, but the code looks like an outsider to be an ugly sloppy jalopy.

Another basic pain is rude aborts and the level of control over system processes that administrators get. For example, in TSQL, a killed process circumvents exception handling, always. This limits the range of idioms you can apply to solving problems, and often forces narrow, verbose solutions or ultimately requires writing the code in a better language than TSQL. I know this, because I've re-written a lot of "bad" code written in TSQL.

I'm a bit curious why you say: Laziness is very bad in this context! Very bad!

I think laziness can work with databases quite well, though I'd want first-class sets and set-composition operators to get the most from it.

First, laziness is not evil incarnate. I just think it would be a very bad idea to have the language rooted in laziness, since that would effect everything including syntax decisions.

Second, laziness doesn't have good resource characteristics. It would make it easy for people to write canned reports with denial of service vectors. If you need this kind of laziness, externalize the lazy computation and treat the results as cached - this is what I've seen data mining approaches using Haskell do. If you require real-time feedback, you must manually reason about what accuracy guarantees you can accept (a realization covered by Carl Hewitt decades ago when dealing with questions that have unbounded answers -- you must do the best you can).

If you need laziness, ask for it when you need it - explicitly.

I grant that laziness isn't a good fit for all programming models. In particular, lazy implicit state can accumulate an arbitrary amount of weakly-aggregated history, resulting in space leaks and denial-of-service risks.

Personally, I blame 'implicit state' rather than laziness.

If you need state, ask for it when you need it - explicitly. State can then be strict up to the current instant.

Laziness is a good default - one that supports ad-hoc refactoring and separates concerns between abstraction and evaluation. If we have terminating functions, laziness is purely an optimizer decision.

abstraction and evaluation?

For example, What would Spine and Spineless look like with first-class pattern matching instead?

I'm unfamiliar with Spine and Spineless, so I'll need an explanation or reference.

Whether laziness is 'the best way' - I doubt it. But it's better than strict. Full beta reduction and other rules can be good if Church-Rosser and terminating.

First-class pattern matching is modeled as a function as `type P a b = a -> Maybe b`. We can enhance this further by use arrow models to statically organize patterns based on possible initial elements in a pattern match. It is unclear to me why you would suggest first-class pattern matching is an alternative to laziness. It seems to me that the concerns are orthogonal.

We should understand the emotional implications of PL design, as well as come up with new programming experiences focused around human needs.

Working with designers (most with social science backgrounds) has been very illuminating in this regard.

Yes, I sort of presumed the need for human needs, I guess I zeroed in on the social/collaborative setting as one of the biggest :) At that level, I would add a cognitive understanding, e.g., for FP vs. declarative vs. OO vs. imperative vs. PBD .. . I think these are both important for the process of language design and what goes in them.

Emotional understanding doesn't jump out at me (e.g., I haven't read much from the PPIG community). Is this for something like the novice / non-professional domain, or do you think it's relevant to software engineering as well? Any examples? E.g., I can see this being relevant to say Scratch... but the next version of C#?

Cognitive psychology works great when it can be informed via empirical methods; e.g., you can measure performance in a repeatable experiment. However, many aspects of a programming experience are not measurable, that are closer to "aesthetics" without being related to look and feel. The impact of such design then triggers an emotional response that can't be measured very well. This to me is the basic difference between HCI (more empirical) and design research (less empirical, more emotional), but that is also controversial (the lines are blurry). As far as I can tell, PPIG deals more with the empirical side.

Programmers are humans! The reason the design in programming community has focused on non-professionals is that professionals (in any field, not just programming, but consider architecture and CAD) are more invested in their existing tools, and have worked around or accepted any painful aspect of the tool. The barrier of entry for something new is very high, and perhaps only incremental changes to existing tools are acceptable.

When the C# team works on v.next, they have many constraints. They can't change things too much, they have to work within the confines of the existing experience and C#'s design philosophy (verbosity is ok, be very consistent, don't add anything that is too dangerous...like traits). Most of the interesting work in this area then has to revolve around tooling; e.g., check out what MSR's HIP (human interactions in programming) group.

Did you have concrete examples of emotional considerations? I might have misunderstood..

Well, if we are talking about professional programming languages...

Let's compare Scala to C#. The design philosophies are very different: Scala is very much about power and expressiveness, C# is more conservative and more about stability and reliability.

In Scala, you have power and freedom, you feel very free to build elaborate constructions, you are free to choose your own paradigm, you are free to focus on terseness if you want. But there are some drawbacks. First elegance is achievable, but you have to work at it, and this results in some anxiety: you wonder if your code as great as it could be given Scala's potential? Second, too many choices on style results in a phenomena known as decision fatigue. There is just too many way to do something, each with its own implications.

C# is more safe: where Scala says yes C# says no. It constrains you, only certain styles can elegantly be encoded in C#. You just can't do many things and have to conform to what was intended. This drives me crazy sometimes, but on the other hand limitations can be liberating: I no longer need to worry about elegance (immediately un-achievable) and the choices I need to make about style are minimal. Using C# potentially involves less anxiety than Scala. This also explains why languages that are obviously worse to us are often very popular (PHP, C++, Ruby...), even though technically the languages seem inferior, they still trigger a favorable emotional response in many people.

Wonderful example, thanks!

I would love you to elaborate on this idea, including your experiences with social science people.. we all know about language wars and emotional coupling to programming systems .. why is it so?

Until we understand how programmers becomes so deeply wedded to the languages they work with intensely, there's little hope of weaning them off archaic systems.

Again I have to point at the most successful project here, namely C++: keep you cake but put icing on it. And you may not only continue to shoot yourself in the foot, here's an RPG to help if that's your style ..

I fully agree. And an IDE cannot much help with cooperative and collaborative work unless the language is also good for it.

I've been thinking about how to scale a language to thousands of concurrent developers. And I have some ideas on how to provide systematic technological pressure for cross-project code-review, reuse, and refactoring. But my own work here is nowhere near 'science' (just 'principled').

Still, this consideration certainly has a huge impact on module systems. And I think that live programming / upgrade is a prerequisite for scaling to a degree where social interactions become primary.

The 'locally optimizing' concern can be resisted by focusing on cross-project refactoring and configuration management considerations as part of the module system - i.e. low-hanging fruit from other projects is tweaked to fit two or three projects and so on, and broken into extra modules so that a linker-constraint-solver can find a more flexible solution for the local configuration (based on non-local requirements or preferences).

Don't JIT compilers already effectively achieve this in a much cheaper and energy-efficient way?

They apply optimizations to "hot" code (aka. "where it matters", detected at run-time, the only reliable time to do so), and they also effectively optimize away code that is never really reached (or only reached very rarely) by not actually running that code (very often).

Why try to apply static optimizations for hours(!) when you can just run the program? You'll just waste elecrity optimizing a program that may not even run for hours!

If anything JIT compilers are the opposite. They are very time constrained so they cannot do much optimization at all. Why delay until runtime what you can do at compile time?

If your program isn't run, then why write it at all? It's not like compiling is a significant cost compared to writing it.

You can also determine the hot parts in a static compiler (profile guided optimization), and focus your search efforts there.

If a program is going to be run on average X times on N computers, then with JIT compilation it's going to be compiled X*N times instead of 1, so it's not more energy efficient either.

I think this is especially true for functional languages, where extensive clever inlining, algebraic transformations, aggressive unboxing (such as over applications of parametric polymorphic functions), application of deforestation opportunities, clever call frame arrangements for efficient "last calls" and so on and so forth can yield a much, much more efficient program - and this is before obviously very important optimizations on a lower level SSA or CPS or whatever IR. Whatever the language design, a kick ass runtime system and static compiler is job #1, IMHO.

- Scott

I'm not being facetious, btw. I'm familiar with work such as Supero (for Haskell) and such, but AFAICT they still haven't demonstrated any real gain, nor are we seeing that work being integrated into Haskell. Instead we're seing low-level optimization such as the LLVM backend being integrated.

(The real problem AISI is essentally the "wall of undecidability". JIT compilation only has the "wall of semi-deciability" :))

A lot of aggressive optimizations are performed, but this is counterbalanced by the fact that a lot more optimizations are necessary just to 'break even'. If performance is the only thing measured, one might still dismiss functional programming.

Most performance for functional programming comes from finding effective, composable ways to express problems - iteratees, reactive programming, etc..

I'm confused .. are you stuck in Haskell? Ocaml regularly achieves a significant fraction of the performance of C. It's not as fast usually but it's quite acceptable in most cases.

My Felix compiler sometimes outperforms C. Surely the language isn't as high level as Haskell or Ocaml, but it is designed for performance and one can usually fall back on inline C if necessary.

Much of the performance comes from inlining, and much of the utility of that comes from abandoning the concept of object code and linkage in favour of whole program analysis.

Hpc does autotuning bordering on synthesis, but for domain theories. To date, FFTW and linear algebra are the poster childs, and stencil codes seem as well.

Generalizing is a big deal. Some of my favorite papers at pldi recently have been about this, and we're putting together another on tree codes (attribute grammars).

The more I work on it, however, the scalability becomes an issue, even with fancy ML guidance. Interesting space :)

I'm not engaged in any type of research on this topic, but I do recall numbers being available in separate papers on specific topics such as Haskell's "simple" foldr/build deforestation scheme; inlining HOF to avoid closure creation for, and achieve direct calls to, functions passed into map/fold/filter (including composed, partially applied functions, etc.); C#/CLR's unboxing (noboxing?) of values in instantiation of polymorphic functions - and so on.

It would be a bit of the lord's work to produce a survey on all of these types of optimizations and the measured benefits they achieve singly and in various combinations on a series of benchmark programs.

Of course, I'd love to see kick butt SSA level optimizations, register allocation and so on along with a handsome runtime system accompanying these higher level optimizations.

I don't mean to say that a successful functional language has to be "as fast as C" to be successful. It just needs to make a credible effort at performance wherever possible and give warm and fuzzies to the programmers who are using these abstractions.

I also don't mean to say that performance is always very important, just that the very flexible, uber-dynamic, performance insensitive Python/Ruby/TCL language space (Does Scheme and Lisp fit in here too or elsewhere? dunno) is pretty well staked out already, at least for the moment, and other newish entrants might hold some promise of future success (Groovy?).

Thus I'm wagering that a successful functional language will stake out its territory elsewhere, likely in the statically typed and more performance sensitive arena.

- Scott

I am convinced you can find a threshold that will completely eliminate supercompilation as a transformation that will give "any real gain", but by those standards almost no other optimization that is currently implemented in GHC will provide any real gain either. Look at Figure 9 in the most recent draft and you can see the runtimes for some benchmarks from nofib, ranging from +6% to -98%. There's some more benchmark results available in my thesis (p. 68), where the runtime of the benchmarks compared to ghc -O2 is between +2.6% and -29.4%. The point is not that a 30% improvement compared to a mature optimizing compiler is great, the point is that we can either preserve or improve the performance of up to 400 line Haskell programs (+Prelude) in a matter of seconds.

The status page for GHC says that Max Bolingbroke continues his work on supercompilation and that the plan is to make his supercompiler a part of GHC over the next year or so. I understand that you want to supercompile your programs yesterday, but we should keep in mind that practical supercompilation for large (or even moderately sized) Haskell programs is still an
open research problem.

I'm not particularly interested in the decidability of the problem myself. We have a problem, decidable or not, what can we do to eliminate it in practice?

JIT is time constrained? Sure, but JIT compilers have much better information. For example, a JIT could inline a virtual method call whereas a static compiler can never do that because there's exactly one case where the virtual call actually dispatches to a different method than in the n-1 cases. The JIT compiler can adapt at runtime by inserting an inlined function with a "trap" in the one rare case where the inlined call is incorrect.

PGO assumes that the program is going to follow a static pattern. It doesn't account for different parts of the program being "hot" during different phases of the run.

I'll happily concede that, given infinite time, you can optimize better statically, but we don't have infinite time.

A static compiler in some cases can inline a virtual method call, i.e. when it knows the associated object constructor. This is not a rare pattern, since we often create objects then use them locally.

In some of my earlier design efforts, I had pursued this optimization further by tweaking how constructors work: instead of creating one 'new' object, developers would compose an 'object configuration' then construct it all at once. This supports the common pattern where object configurations are static after initial construction. The goal was to support more efficient allocation, garbage-collection, and a lot more inlining within the configuration.

I agree with you that JIT or hotspot/trace compilation does support a lot of optimizations. I'd be more comfortable with this if we have hard real-time, bounded-space, formally proven JIT.

A static compiler can inline virtual method calls just fine. More generally a static compiler can inline first class functions just fine. If you have a call f(x) where f is passed in dynamically (virtual method calls being a special case where f is a record lookup obj.foo), then a static compiler can often determine what possible values f can have at runtime. If the compiler can prove that there is just one possibility for f, then it can easily inline it. If it determines that there are multiple possibilities then it can insert a conditional similar to what JITs insert at runtime, and possibly inline some of the calls in the branches.

For example if you have:

    obj = ...
    obj.foo(x)

And the compiler determines that the the object's foo field is either bar or baz, then it can do this:

    obj = ...
    if obj.foo == bar:
      bar(x)
    else: // foo is known to be baz here
      baz(x)

Now it can proceed to inline these statically known calls.

Even if the compiler isn't able to prove anything it can still make use of profile information. If the profiles indicate that most of the time obj.foo is bar, then it can generate the same code that JITs do:

    obj = ...
    if obj.foo == bar: // profile information determined that bar is a common occurrence
      bar(x)
    else: // nothing known here
      obj.foo(x)

PGO assumes that the program is going to follow a static pattern. It doesn't account for different parts of the program being "hot" during different phases of the run.

That is a theoretical reason often put forward in favor of JITs, but comes up very rarely. In fact I can't really think of a practical situation where this would come up and where JITs could make use of the information to do important optimization. Can you give an example? Does this rare case weigh against the overhead of JITs that you suffer in all cases?

I agree that for very dynamic languages JITs may be the way to go for the foreseeable future, but I was thinking more along the lines of more well behaved languages like Scheme/ML/etc.

In a whole program analysis where there are two known instances of a virtual function it is trivial to inline both statically and simply switch on the current case at run time.

At most the JIT could eliminate the switch, it's unlikely it could come close to the performance of the two statically inlined cases.

[My knowlegde of this trick is due to its use in MLton compiler in slightly different circumstances]

Static analysis can facilitate high level optimisations way beyond what any JIT could possibly do (simply because the compiler has a lot more knowledge of the whole program).

The list of authors for Preliminary design of the SAFE platform, namely A. Dehon, B. Karel, B. Montagu, B. Pierce, J. Smith, T. Knight, S. Ray, G. Sullivan, G. Malecha, G. Morrisett, R. Pollack, R. Morisset & O. Shivers, would overwhelm my dinner table and hosting capabilities, but would otherwise be excellent.

Reflecting on how exciting I find work on the interface between PLs and OSes makes me think that this is the application area that will dominate the direction of PL design in the near future. And I'm beginning to think that scripting languages, for all the heat and light they generate, have stopped being capable of true surprise.

Well, I have tried to construct a non-very-well-thought-trough novel functional compiler which translates to a small number of primitives.

It doesn't seem to work. Handling a rich set of primitives is just necessary for compiler performance on corner cases, or even handling the corner cases. There doesn't seem to be a way out of that, and I kind of regret not going that route in the first place.

What I am more concerned about is that, after trying a somewhat novel approach, I am clueless why some compilers produce more performant code than others. There are just too many variables, and historical wisdoms, involved to really understand it. If I would understand it, at least I would know how to progress forward.

To clarify what exactly I see as missing from common approaches that is needed here, particularly on the language side:

* A way to express an algorithm as a set of separate pieces and a description of how they can be combined, in a nondeterministic way. We do this deterministically all the time with function calls -- "do this piece, then this piece" -- but I need a way to say "do these two pieces in either order, or alternately do those three pieces in this particular order".

* A way to express these pieces in a generic form with parameters, and express relationships between those parameters. That is, a way to express "do this piece with parameter i, and do that piece with parameter N-i, with constraint i%4==0" (where i is arbitrary, and N is a given value).

* A way to express the optimization criteria -- i.e., best wall-clock runtime given a particular input data set, or an average best runtime given a range of input sets -- and accuracy criteria.

And then of course I need the implementation that will actually produce a good working result out of all of this. That's not entirely a separate problem; one of the critical pieces here in practice is that the language needs to be expressive enough that the user can guide the implementation.

Take a look at Programming by Sketching. Ras has been working on this for several years along with several of his students. There are many good papers on this so I'll just give you a starting point. There are some nice demos of the system, but the AES example was particular eye-catching as it infers the structure of the lookup-table that gets used in the function. I feel a bit embarrassed pointing you towards my work as it is much less sophisticated although it does address one of your points directly: how to explicitly embed the search problem in the language semantics.

Thanks! I'll have a look at those.

I was trying to dig out a link for SPIRAL (tuning of linear transforms) when I came across iWAPT, which may be a much better starting point on where autotuning currently is than the previous links.

Also take a look at Don Batory's work (http://www.cs.utexas.edu/users/schwartz/). He's done quite a bit on combining software "pieces" ("features" in his terminology) in a variety of ways, leading to software product lines. One of his more recent papers deals with the problems of optimizing such combinations, since the combination of seperately created pieces will often lead to cross component optimization possibilities

cheers

Maturity. I realize what I thought was wrong/needed 10 years ago was completely off base. I did my first real pub exactly 10 years ago to OOPSLA, which was about adding unit-style modules to Java. It was a nice idea, but at the end of the day it didn't seem to solve real problems.

I now realize that more power, performance, theory, complexity isn't going to really move us forward right now; they aren't really addressing the bottlenecks of programming technology. Look at what is really preventing us from making programming more reliable, more persuasive, more cheap, so that we can get an order of more useful code written tomorrow than we can today. Incremental improvements that make things slightly better won't get us to the next big step.

Finally buried I hope.

I've long suspected AOP was an extreme example of how a broad sweeping vision gradually gets bogged down in technical details, ossifies, and eventually becomes too brittle to sustain. Which happens to pretty much all paradigms, but it happened really fast to AOP because, really, the broad vision wasn't accompanied by a broad technical approach in the first place. I heard... Kiczales?... speaking about the big picture of AOP a few years ago, and saying you can describe what an algorithm normally does and it's clearly understandable, but when you explain along the way how abnormal situations are to be dealt with, the whole description is a mess. So what if your language somehow allowed you to describe the normal flow of the algorithm as one modular subunit, and the special cases as another. Which, as I recall, he admitted he didn't know how to do, and he reckoned AOP was waiting for someone to devise a theoretical foundation. That broad vision has no sign of join points in it.

I think the difficulty with this is that most naive approaches to factoring an algorithm into "big idea" x "details" will fail. In existing languages, there usually is a way to express an algorithm such that the high level is clear and the details are well encapsulated, but finding it is hard work.

Agreed, most naive approaches won't work. The tantalizing possibility, so it seems to me, is that perhaps there is some way of organizing programs that makes this sort of thing much easier, that we're not thinking of because our thinking is shaped by paradigms whose joints aren't articulated to bend in that direction (so to speak).

I was once seduced by this vision of being able to encode all kinds of knowledge in nice reusable modules, using some uberglue (weaving?) to configure and integrate them with little effort. But (a) aop as aspectj could never really deliver beyond small examples and (b) the cost of the technique was just too much.

I'll take a more controversial position than that: metaprogramming is not a viable solution to improving programming, at best it is just a nice implementation technique and at worst is an example of being too clever to be sustainable.

Metaprogramming (examples including macros, reflective towers, template meta-programming, meta-object protocols, staged languages) seems well proven to cover weaknesses in a language.

AOP strikes me as dubious. But there are other approaches to declarative metaprogramming that seem more likely to work well. Staged constraint programming is well proven in many roles, for example, and I see no reason it wouldn't work for linking.

Metaprogramming has been shown to be interesting, I won't argue with that, but interesting != successful. Meta-programming where used successfully has been fairly ugly (C macros, C++ templates), the more elegant uses are still academic curiosities.

This gets to another point: when do you call something successful? AOP was classified as a success until it silently disappeared into apathy. Just saying "metaprogramming" is successful does not make it so, nor is it fair to claim that people just didn't get a language/technology that is claimed to otherwise be successful. Until we understand why what we think is great technology fails, we won't be successful. Its not all marketing, often times the great technology just plain doesn't work in practice.

Lol. I agree that much metaprogramming has been quite ugly, historically, whether it be macros or meta-object protocols or something else. But I don't think this is a characteristic of 'meta'-programming. Rather, it's just a characteristic of 'programming'. Programming, where used successfully, has been fairly ugly (C, C++); the more elegant languages are still academic curiosities. ;)

I don't think most people 'get' metaprogramming. But I would posit that the success of metaprogramming does not depend on everyone getting it. After all, there are many second-tier clients - the people who use libraries that are developed based on metaprogramming.

I've never heard AOP considered 'successful' before. I did pursue it for a while but decided that it violates too many security and modularity principles (i.e. developers should not need to know internal details such as 'joint points' in a module, and the implementation shouldn't be able to access them anyway).

I would suggest that AOP failed because it was much uglier than macros and templates.

Like some other things I could mention AOP was always "the next big thing". But as they say, it is hard to predict, especially the future.

I'm a C++ dev at my day job, and I would say that template metaprogramming is "successful". We use it all over the place in our codebase. This is boring, bottom-of-the-stack software we're writing.

The biggest detriment to it is the terrible syntax that C++ has. But learn enough languages and you get desentized to bad syntax eventually. Our interfaces are much richer, our programs more robust and our runtime performance better than it would be without metaprogramming.

I don't think it has to be widespread to be successful. Most of the C++ community sees template metaprogramming as being "too hard", but lately it seems that most of the community doesn't really know C++, they just know C and put C++ on their resumes.

'Much uglier than macros and templates'.  There's a round condemnation. :-)  Not disagreeing, mind.  Though I do wonder if the general concept also affords some more elegant technical form.

BTW, iirc Shriram did some work advancing the thesis that AOP is modular after all.  (Alas, I never studied those papers myself, having other things on my plate <cough>fexprs</cough>.)

It is as if it is threaded throughout the entire discussion touching everything :)

Since no one said it: This was one of the best replies ever.

I think this isn't a concern of programming language so much as of a truly 'integrated' development environment - i.e. the unification of user-interface and user-programmable systems, augmented by personalized user-agents.

So, first solve the problem of secure, large-scale, user-programmable services.

I always fail at some level whenever I try to separate a language from its running environment and have pretty much give up. (Sort of like the code in DNA doesn't seem to make any sense without the cellular machinery to decode it.) .. and I'm not even sure I'll be able to recognize it when secure, large-scale, user-programmable services become possible and the problem can be marked SOLVED once and for all.

For something at the boundary of language, runtime environment, music and beer - checkout ixi lang :)

edit: My mention of "beer" is actually a quote from Thor Magnusson's talk on ixi lang - "I wanted a language that I could use even after a few beers" :)

As an undergrad in the mid-1980s, I had a conversation with another student (we were both doing undergraduate projects involving compilers, as I recall) about programming language design. He told me there was no point in pursuing PL design in the long run, because in twenty years [sic] all computers would be optical computers that program themselves. I said —with a straight face— if that was so, the computers would want a really good programming language to program themselves in.

the computers would want a really good programming language to program themselves in.

Damn straight.

I guess this means we should just wait for the Singularity, and the computers will design the optimal language for us. Problem is, the Singularity ain't near.

I agree that machines would 'want' an effective language for efficient and local reasoning about difficult non-local properties (safety, security, robustness, resilience, graceful degradation, resource management, job control, runtime maintenance and upgrade, disruption tolerance, persistence, optimizations, consistency, timing, modularity, extensibility, etc.).

But I think we'd also find a lot of differences. Machines don't much need namespaces, once-and-only-once, reuse, refactoring.

Can we design a language good for machines without compromising qualities useful uniquely to humans?

My guess is that when machines are as smart as we are, they will also be as dumb (i.e. lack direct access to lower levels).

I agree that many programs should lack direct access to lower levels, if only for reasons of cooperation, resilience, and security.

But I wonder what it means with respect to the language. Would a machine's programming language look the same as our own?

Stupidity requires immense intelligence.  Computers today are (for the most part, anyway) not at all stupid, any more than a hammer is stupid.  It's a tool that's designed to perform a task.  You have to move past the design stance to the intentional stance before it's meaningful to ask whether you're stupid.

I think there's a level of intelligence analogous to Turing power for computation, complete with a Church-Turing thesis.  I'd expect artificial intelligences to have many of the same foibles we do, and I suspect they'll be just as impressed by Richard Feynman, Christopher Strachey, etc. as we are.  So when I said I thought they'd want a really good programming language, I really did mean that they'd want many of the same PL properties we would want, and that they'd be building on our work in developing languages rather than starting over (except, of course, that they would be just as prone as we to 'those who do not study history are doomed to repeat it').

I was basically saying the same thing (in my own snarky way).

To continue in the same vain: I suspect that trying to program your fellow machines, rather than convincing them rationally, will be considered akin to brainwashing...

I can just picture AI politicians accusing each other of programming the electorate.

I can picture the Prolog contingent claiming that only logic programming can support rational deliberation.

I don't want my AI programming assistant to be impressed by me, ashamed of me, worried about me brainwashing it, or wishing it was a real boy. Sign me up for the model that just "thinks" about programming stuff.

The problem is that I suspect that you can't have one without the other. If you want the model that just thinks about programming stuff, simply hire humans, make sure they are not unionized, and convince them to wait indefinitely for their stock options to be worth something.

Assuming "the singularity" becomes technologically feasible at some point, I think we can get all the HCI interaction we need by directing the machines to study human behavior without trying to impose upon them human incentives and desires. Or if you're suggesting that these desires will invariably emerge or evolve in any advanced intelligence, I disagree there as well. But I haven't thought about it too much. If you have a reference that convincingly argues otherwise, I'll check it out.

It's a huge field. I'll try to think of a good starting point. (I suspect that the desires comes first, the intelligence second, not the other way around, but it's a complicated co-evolutionary process.) This sub-thread is not really on topic for LtU anyway, of course...

To make this slightly less off-topic: The machines overlords will probably need something like Elephant, if they want to get anything done in the real world.

I never quite grokked Elephant. Substituting "make commitment -> assert", "exists commitment -> unify", "cancel commitment -> retract" seems to just get me Prolog with nothing much else to it.

I think some experiments in generating programs using evolution-style variation and selection have resulted in programs that we'd have no clue about why and how they work .. just that they somehow do.

That said, we may not have access to our lower levels as individuals, but as a collective we've already started using silicon for computation, thereby augmenting our "lower levels". In other words, our eventual machinery does not seem to be constrained by our current machinery. If we extend that to computer languages, our current concern may only be about a boot strapping language that the machine will eventually discard in favour of some other architecture that could be qualitatively different.

I am interested in what you have in mind in the first paragraph. Genetic Programming? (No, I won't plug the book now).

haha! yes, of course :)

I've been intrigued by this aspect of genetic programming (that the result can run far away from our analytical abilities) ever since I ran into Adrian Thompson's work on hardware evolution in '97.

For the moment, it seems like evolution-inspired techniques are some of the best we have for unstructured search of various kinds of problem spaces.

fixing zune bug genetically/automatically.

I've long been fascinated by the Push programming language, and the idea that a stack-based language is ideal for genetic manipulation (in addition to all its other benefits)

I did genetic programming on a stack based language for my end of high school thesis, and genetic programming didn't work compared to hill climbing with random restarts. Genetic programming (i.e. with crossover) is supposed to work much better. Some time later I read a paper where they compared hill climbing to a genetic algorithm on an artificial problem that was previously designed as an example problem where genetic algorithms are strong. The hill climber was much much better. Then they proceeded to design another artificial problem where a genetic algorithm would outperform hill climbing. It was surprisingly hard, and they went through several failed attempts before they barely succeeded after tweaking the genetic algorithm a lot and not tweaking the hill climber. Note that genetic algorithms are a much more general class of search methods than genetic programming, so it should be easier to design a problem where one of them work. Does anybody remember which paper this was? For some reason their conclusion was "we need more research to find artificial problems where genetic algorithms work" instead of "lets not waste our time on genetic algorithms, they don't work".

why didn't they write a genetic algorithm that found the good problems for them?!

Associative arrays did make a comeback at some point. Was it Javascript that breathed new life into them? I'm not sure. Still, they turned out to be a great programming interface, if not a great universal data structure.

I'd say Javascript is just one member of a long tradition: Perl, PHP, Ruby, Python, Lua, and Javascript are all basically hashmap languages. There are probably others that I can't think of. Maybe it all goes back to Bourne shell?

OO put a tiny bit of new spin on it, but not much... just as the lambda people looked at OO and said, "Oh, I get this, it's just closures!", the hashmap people looked at OO and said "Oh, I get this, it's just more hash tables!!"

I would start with AWK I suppose (though AWK wasn't the first to have "tables", SNOBOL was earlier). But I was asking about the comeback, especially as programming interface used for meta-programming and parameter passing, if indeed that's a fair characterization.

No, it trades usability for performance.

You want a language which do more things that most current languages do and yet you say that the language implementation should be small?
I think that you can't have both!

If a language fails doing this, then it (with few exceptions) is conceptually broken.

No, it trades usability for performance.

Some do (for performance or other reasons) – that is why I said ‘with exceptions’. But most are just broken, or perhaps have been in use longer than they should.

You want a language which do more things that most current languages do and yet you say that the language implementation should be small? I think that you can't have both!

The many overblown implementations may delude one to think that useful programs are necessarily complex, and that complex programs are necessarily large. But neither of these is actually true, as shown by other examples (see e.g. REBOL, K, Lua, each implemented within several hundred kB).

Linked lists need not be a fundamental data structure. It is a huge shame that e.g. Haskell is so strongly dependent – in practice if not in theory – on lists. Linked lists are a sheer atavism, a legacy from the early days of functional programming when they were, by accident, mistaken as an inseparable part of it.

I disagree. In many case, random access is not needed, the fundamental unit of abstraction is the sequence, and lists are well-suited for this purpose. They are not necessarily "fundamental" but you argued for "practical data structures", and I think it's clearly practical to have lists.

As an example of the pervasiveness of lists, look at the syntax for array litterals/initialisers/values in most languages, or the abstract presentation of variable arity functions : they are, in essence, lists.

Lazy lists have an even greater application area, as they can represent any (possibly infinite) stream of produced data (producer/consumers pattern; lazy list cons is a non-inverted form of 'yield'), or events. Look at the work in Oz/Mozart (eg. the CTM book and others Peter Van Roy's papers) to make lazy streams the unit of orchestration for deterministic concurrent programming.

There is the argument made in Fortress that lists are too linear to be well-suited for parallelism, where tree-like structures (or random access structures) where data can be easily split between workers are more appropriate. Guy Steele argued that functional programmers relied too strongly on list and non-clearly-associative folding of them, and should change their habit a bit to parallelize more easily. I don't think that's enough to make ordered sequences "legacy".

I do not mean that linked lists are totally useless, only that they need not be the central data structure of a programming language, functional or not.
If (lazy) streams are needed, then we should use them.
But in general, random-access – or at least bidirectional – sequences are much more useful than unidirectional lists. For one thing, lists may be hopelesly inefficient with large number of elements. They are just insufficiently practical, compared to the alternative.

As an example of the pervasiveness of lists, look at the syntax for array litterals/initialisers/values in most languages, or the abstract presentation of variable arity functions : they are, in essence, lists.

How is that relevant? I was discussing data structures, this is syntax.

the fundamental unit of abstraction is the sequence, and lists are well-suited for this purpose

My feeling is that A) cons lists are a horrible violation of abstraction (you manipulate bare list elements; there's no sequence ADT) B) cons lists are subtly intertwined with the syntax and semantics of the Lisp/Haskell languages and any change will introduce subtle problems (e.g. if you switch from conses to a sequence ADT you will lose some of conses' niceties such as being able to take the "rest" of a sequence in constant time, and without introducing memory leaks or a lot of copying). [Okasaki may have figured out that part, I should probably check it out again.]

I agree that the educational aspect is important, though I sort of rope that in with the 'socially motivated design' discussed by Leo and Sean earlier.

I have a vision of how to approach this, one I've been working towards for several years.

There are no fixed 'fundamentals of programming', just a generic purpose - ability to modify behavior of our environment. The best approach to teaching this is to support pervasive programming, i.e. such that programming is a widely accessible 'Ages 5 & Up' activity, starting with tweaking the UI and e-toys and programming the lights in one's house, and continuing seamlessly up to massive multi-player games, traffic control, and managing armies of evil robots.

The basic requirements for this vision are:

1. a programming model that scales seamlessly from e-toys and 'Hello World' to production systems and massive multi-player games, i.e. no discontinuity spikes or paradigm shifts necessary
2. a secure programming model, such that we can give young developers a power precisely commensurate with their responsibility, and such that we can integrate disparate systems without having discontinuities between administrative domains
3. a 'forgiving' model that not only resists error but is resilient to it, e.g. supporting runtime upgrades, graceful degradation, safety nets such as eventual consistency, stable state
4. A focus on communication and orchestration rather than calculation and simulation; the purpose of programming is (by definition) to control our environments and systems, not model them, though there is a relationship between the two.

The 'fundamentals' are then a function of the programming model.

I do agree the language needs a good built-in collection type - one that works well across domains, and that does not inherently capture any more semantics (such as ordering) than absolutely necessary. A lot of modern languages force developers to waste a lot of efforts converting collections as we move them between libraries and problem domains, and this is especially a problem in context of observer-patterns, live updates, and query optimization. I think well-founded sets are a good option.

I would not want built-in facilities for vector graphics. That's very domain specific, and isn't something most apps should be directly involved with anyway. I feel the conversion from data to vector graphics should be a separable concern, provided by an intermediate service, such that we can easily support multiple views of an application.

We should re-think how we support display services as a whole. Rather than an application 'pushing' a demand to a specific display or UI service, I think we should move more in the direction of browsers: applications publish services and relevant metadata to a shared registry, and these services allow a browser to 'render' the application. This structure allows greater support for bookmarking, history, secure mashups and navigation, multi-user applications, computer supported cooperative work, transparent persistence, mobility between devices (desktop and laptop), ambient exposition and sharing, et cetera.

Importantly, how we model UI will be critical to achieving 'pervasive programming' and support for education. UI is the first thing a young programmer is exposed to. If UI is not reprogrammable and extensible, we have already failed at making programming accessible.

An interesting possibility is a 'programmable hand and eye'. I.e. rather than a 'pointer', we have a 'hand' that can be extended with 'tools' - such as paintbrushes, clipboards, bags for selected elements. (Tools can be further organized into toolboxes.) Our context menus are then a function of both the tools and the target. The 'eye', in turn, can also be programmable or extensible - i.e. providing a HUD, supporting CSS-like transforms, the basis for navigation and layout. Similar to tools in the hand, we can use a stack of 'lenses' for the eye, which provide translation services, data-fusion, and x-ray vision where we want it.

The programming experience then starts with choosing sets of tools and lenses, and mashing up services. As a developer grows, they can create new tools, new lenses, and new services. There will be no distinction between 'IDE' and 'UI'. An IDE is just a hand of powerful tools and a suite of useful lenses. The learning curve - much like the programming model - should be continuous, smooth, no discontinuity spikes, and start low enough for a curious five year old.

I would not want built-in facilities for vector graphics. That's very domain specific, and isn't something most apps should be directly involved with anyway.

In view of the substantial advances in vector graphics technologies, I believe increasingly large number of apps will be interested in making use of directly manipulable pictorial data, rather than relying on external services for that.

More fundamentally, I like to think of simple geometric data – location, orientation, direction etc. – as something almost as essential (or even equally essential) as numeric data. Why should it not be so? In programming, we are used to take numbers and (text) characters as row data, then create structures of these and abstract away from those structures. But why are we limiting ourselves to these kinds of basic data? Of all things in programming languages, this is probably the sole one than never evolved, and I'd love to see that changing.

The choice of 'vector' graphics seems quite arbitrary. (Why not POV-ray? Or a volumetric model?) Language designers should avoid making arbitrary decisions, i.e. every decision should either be well justified (at least three good reasons) or delayed.

Though I grant it can be troublesome to compose different graphical models (e.g. raster and vector), I would expect developers to handle most composition with higher level data anyway - i.e. such that they can support ad-hoc semantic queries and joins.

If we are to ever have application mashups, multiple views, CSCW, user agents, and accessibility, apps will need to expose their data via a more semantic API, even if they also support a view model (via something like XSLT or CSS). The arbitrary distinction between 'service' and 'application' is something else that we should abandon. Eventually we'll have something closer to naked objects or tangible values.

IMO, translation of high-level semantic data into 'vector graphics' is simply not a task with which most applications should be directly involved. That is a job for browsers or 'lense' services and objects. Your vision - that more apps directly manipulate graphical data - is something to avoid, not to encourage.

Even if you mean 'geometric' data in a broader sense, we have quite a few heterogeneous models imposed on us by our sensors and actuators. How does a rotating laser sensor generating a probabilistic point-cloud over time while sitting on a moving platform wobbling along a gravel road without any ground-truth fit into your 'location, orientation, direction' standard? Where do the Kalman filters and Markov models get involved? What about Kinect or camera-based user input?

We need programming languages that support heterogeneous data models, and allow a high level of cross-domain composition without abandoning diversity of domains and domain models.

Basic numbers - naturals, integers, reals - are far more likely to cross domains. Even those, I feel would be nice targets for libraries (i.e. model natural numbers as Nat=z|s(Nat)) if such modeling does not hurt valuable compositional properties (such as enforcing progress or non-divergence of functions, or real-time or bounded-space properties, or meta-stability of state and dynamic algorithms).

I think you are heavily misreading my post. You are talking of detailed, very specific visualizational models, but that has nothing to do with my proposal. This is as if we shift from numbers and arithmetic as present in programming languages to discussing domain-specific libraries for numerical computations – these are completely different levels to consider.

What I propose is introducing basic data and procedural facilities for handling basic geometric notions (including visualization of simple geometric data). How that is handled in a concrete executional environment (Web browser or another graphics-enabling medium) is a separate concern (and can be controlled through external libraries and style enforcement). In much the same way, most languages' definitions introduce the notions of text, numbers, standard I/O, and files. They do so in an abstract way, relying on the ability of an executional medium with all its hardware and software components to realize the said notions.

I consider your vector graphics proposal analogous to building 8-bit sound as a primitive in a modern programming language. "But!" the analogy-world you exclaims, "It doesn't count because this is a very simple model of sound." Still, it's sound, and domain specific. It has no place as a general purpose language primitive, whether simplistic or adequate.

Can you justify your claims that geometric primitives are 'essential' and that the ones you choose would be adequate?

I also do not believe that standard I/O or files should be part of a language's definition. A language can instead just define access to modules, and how foreign services are integrated. By comparison, text and numbers are special cases because we want syntactic sugar for them, and because symbols and numbers (measures, really, and values arising from them) are truly universal across problem domains (indeed, some might reasonably argue that if you don't measure it, you don't have a domain).

I would like to see easy access to graphics, and perhaps even some domain-specific standardization. But I'd rather keep such standardization separate from the language definition. It makes a good library.

What exactly is it that you two are disagreeing about? That vector graphics should or should't come with the standard language installation?

Vector graphics (and similar arbitrary choices of geometric primitives) should not be part of a general purpose programming language's definition, nor of any formal documentation of the language's standard. And I especially don't want it to be "built-in" like Boyko proposes. I would not want a graphics service to be nailed down like that, nor would I want the language to have such a pointless, domain-specific constraint.

With regards to installation, well I've got this vision in my head of a networked language that uses DVCS for a near-global set of projects (e.g. wiki-sized) rather than one project at a time. The goals are easy cross-project reuse and multi-project refactoring. Years ago I was considering a WikiIDE, which achieves the relevant cross-project pressures based on a flat namespace and accidental linking. My ideas have evolved; I now believe import-by-name is problematic, and so pursue a model that does not rely on names - e.g. linkers as constraint solvers (which results in better domain-relevant collisions anyway due to type and pattern matching, and is more suitable for runtime upgrades).

I expect developers will have easy access to graphics when they want it, as part of a public multi-project DVCS repo that you can freely inherit from, push to, override. It just won't be, and shouldn't be, a 'standard' of the language.

I do accept that domain-specific APIs are often standardized, and I find that acceptable. (E.g. OpenGL is a standard developed for the C language, but not a standard of the C language.) I also favor user-definable syntax and extending the syntax+IDE with, say, canvases and wires has always been part of my vision.

Before characterizing anything as ‘arbitrary’ or ‘pointless, domain-specific constraint’, one should have applied some effort to actually understand it. Doing otherwise does not do a good service to the discussion, to say the least.
You said that numbers, as universally needed in programming, are really ‘measures’. This is only partially correct. If you were right, we would be content with non-negative discrete magnitudes which need only be compared. Negative, real and complex numbers would all be superfluous, along with arithmetic – all they are unrelated to measuring. In reality we do need both arithmetic and all those kinds of numbers. Why? Because computation is essential to programs, and without the whole gamut of numbers lots of computations become impossible.
Similarly, by adding values of geometric nature to the ones we already have, we can enrich the computational power of a language in a most general sense. In this respect, it is appropriate to remind the Grassmannian-Cliffordian point of view, according to which scalars and vectors, for example, are elements of the same computational domain. There is nothing ‘domain-specific’ here, and, in particular, the analogy that you see with sound is completely out of place.
As for visualizing such data, I consider this as a smooth extension to visualizing text – which we have anyway. Again, there is no analogy with ‘8-bit sound’.
Whether the said capabilities are built-in in the language or are in its standard library is non-interesting – what is important is whether they are standardly defined with the language.

My response isn't reactionary, Boyko. What makes for a good general purpose language feature is something I've spent a lot of time contemplating. As are various approaches to composition, fusion, and visualization of data.

If you were right, we would be content with non-negative discrete magnitudes which need only be compared. Negative, real and complex numbers would all be superfluous, along with arithmetic

Real numbers, and negative numbers are both domain generic: real numbers because not all measures are counts, and negative numbers because not all metrics are absolute. What is the altitude of an object buried in the Earth? Every domain has metrics and symbols. The reason real numbers have the name 'real' is because they correspond to measures of the real world.

But we don't generally need complex numbers. Indeed, their application is quite rare. Including complex numbers in the language standard is arbitrary, domain-specific constraint. Even worse, the decision between polar vs. vector representations of complex numbers is equally arbitrary, and which representation is 'better' is equally domain specific. So this is twice arbitrary.

We do need to combine numbers to create new numbers, because a lot of measurements are composite or aggregate, especially for statistical measurements. So some sort of arithmetic is necessary, once we have numbers.

In reality we do need both arithmetic and all those kinds of numbers.

We do not need arithmetic and those numbers to be standardized. It isn't as though numbers are 'cross-cutting' - i.e. we don't generally compose numbers from one domain with numbers from another because the units are usually distinct or incompatible (can't usefully add apples and mortgage interest).

The marginally good reasons for numbers to be part of the standard are (a) so that we can have number literals (like '42') in our source code, and (b) so that we have a compact, commonly understood representation of number values for serialization or distribution of code and data.

without the whole gamut of numbers lots of computations become impossible

Oh, really? Can you name even one computation that is impossible without the whole gamut of numbers? Or by 'impossible' do you merely mean 'inconvenient'?

I would posit that numbers aren't necessary for computation at all. Rather, I suspect the inverse is true - numbers are impossible without computation. There are many Turing complete models of computation that don't depend on expressing or composing numbers.

by adding values of geometric nature to the ones we already have, we can enrich the computational power of a language

This is not true by any concept of computational power I'm familiar with.

I consider this as a smooth extension to visualizing text – which we have anyway

I do not believe that visualizing text should be language standard, either. Support for symbols is important, but the idea of a global 'standard' output is an old mistake repeated by tradition.

Including complex numbers in the language standard is arbitrary, domain-specific constraint.

Numbers of many different types are needed in programming for the same reason they had been invented/discovered in mathematics: to be able to solve more varied problems, equations in particular. The concept of real numbers (that are not rational) came not from measuring (where fractions have always been sufficient for all practical needs) but from performing calculations with the measured, i.e. from (geometry and) algebra. The same holds even of negative numbers, and most obviously of complex: all they were born from the need to compute and to find solutions. It took many centuries for these numbers to be accepted by mathematicians as – in the jargon of today – ‘first-class citizens’ in computations. Regarding the name ‘real’, it was introduced not ‘because they correspond to measures of the real world’, but to distinguish them from ‘imaginary’/complex entities – a distinction needed in algebra, and having nothing to do with measurement.

Even worse, the decision between polar vs. vector representations of complex numbers is equally arbitrary […]

Conceptually, complex numbers need not be tied to any representation, so this is unrelated to their presence in a language. But since, at some point, we do need representations, both should be (and usually are) supported – no decision problem arises.

[…] we don't generally compose numbers from one domain with numbers from another because the units are usually distinct or incompatible […]

Units are irrelevant precisely because they are a domain-specific notion. Types, however, are not. Intermixing types, such as integer and real, real and complex, etc. is quite common and indispensable. (E.g., some real values can only be defined by expressions of complex arguments.) What we need in programming is better ways to unite the different types of numbers – as they are in mathematics – not separate them.

without the whole gamut of numbers lots of computations become impossible
Oh, really? Can you name even one computation that is impossible without the whole gamut of numbers?

I thought it was clear enough, but if it isn't, let me rephrase it: we need all kinds of numbers because missing any of them makes many computations impossible (or prohibitively impractical).

by adding values of geometric nature to the ones we already have, we can enrich the computational power of a language
This is not true by any concept of computational power I'm familiar with.

Not from the standpoint of formal computability, of course, but in terms of practical programming it most surely is true.

I do not believe that visualizing text should be language standard, either. Support for symbols is important, but the idea of a global 'standard' output is an old mistake repeated by tradition.

Hard to tell. In any case, I can think of several good reasons to keep standard I/O, and of no better option to replace it.

Anyway, I think this discussion is already too long, and going somewhat off the track. Clearly, we all have different ideas about what a programming language has to include, and there is no need to reach agreement.

I thought it was clear enough, but if it isn't, let me rephrase it: we need all kinds of numbers because missing any of them makes many computations impossible (or prohibitively impractical).

Your position is clear enough, but it is unjustified. I challenged you to find an example to demonstrate your assertion, and you settle for repeating yourself. I do not believe you. I am not aware of any computations that become "impossible (or prohibitively impractical)" by the exclusion of, say, complex numbers from the standard (such that we lack 'the full gamut').

If we do not provide numbers in a standard, they would be achieved in non-standard libraries, quite easily. We'd have a few common math libraries (possibly with their own standards bodies, independent of the language), and very little trouble converting between them in the few cases that we'd actually need to.

The only inconveniences I am aware of would be the syntax for number literals, and possible overheads for code distribution or serialization. Neither of these inconveniences would be prohibitive.

the name ‘real’ was introduced not ‘because they correspond to measures of the real world’, but to distinguish them from ‘imaginary’/complex

The points aren't contradictory. It is true that 'real' was introduced to distinguish from 'imaginary', but that's a shallow cause. One must still ask: why were the names 'real' and 'imaginary' chosen instead of, say, 'charm' and 'strange'? It seems to me that the name 'real' was chosen because of its connotations, because the measures in the real world are real values.

real numbers came not from measuring but from performing calculations with the measured

I consider calculations part of measurement. They always have been. Even if we measure in cubits, we still need to add them together. By nature, measurements are ratios. The notion of measurement is inextricably tied to arithmetic.

we all have different ideas about what a programming language has to include, and there is no need to reach agreement

True. But you should be able to at least justify your 'ideas' before acting on them. Only a fool believes every idea that enters his head.

no need for numbers

Sure, for a Turing or Post machine. But most programmers these days are spoiled enough to prefer a less Spartan technology.

Your position is clear enough, but it is unjustified. I challenged you to find an example to demonstrate your assertion, and you settle for repeating yourself.

No. You asked for a ‘computation that is impossible without the whole gamut of numbers’ – that is, that requires all kinds of numbers. This has nothing to do with what I am talking about, or with proving anything, and I had to repeat myself only to stress that. Are you missing the meaning of your own words in addition to mine?

I do not believe you.

Not my concern.

I am not aware of any computations that become "impossible (or prohibitively impractical)" by the exclusion of, say, complex numbers […]

What you consider impossible or impractical I do not know, so don't expect examlpes. What I do know is that a lot of widely used languages support complex numbers standardly. Some that initially didn't, acquired them at a later stage. None, as far as I know, dropped them or is going to.

It seems to me that the name 'real' was chosen because of its connotations, because the measures in the real world are real values.

Not very long before that negative numbers were also ‘unreal’, even considered absurd. Same for the irrational, let alone transcendental (mind the words!) numbers. Nowadays all they, along with the ‘complex’ numbers, are everybody's good friends, and equally real.

By nature, measurements are ratios.

True, and ratios are represented by rational numbers. No irrational number arises from measurement. So, are you also going to expel those numbers like you do with the ‘complex’ ones?

Only a fool believes every idea that enters his head.

Correct. Stick to that.

Are you missing the meaning of your own words in addition to mine?

You should ask yourself that question. Your make statements like: (a) "without the whole gamut of numbers lots of computations become impossible", (b) "we need all kinds of numbers because missing any of them makes many computations impossible". Yet, you cannot find even one example of a computation that becomes impossible or even prohibitively inconvenient. We cannot get to 'lots' or 'many' without starting with 'one'.

Not very long before that negative numbers were also ‘unreal’, even considered absurd.

Negative numbers have been around for at least 2200 years. Irrational numbers are known to have been around longer than that, at least 2700 years. The decision to name complex numbers was Descartes introduced the term 'real' around 400 years ago. What history are you using?

True, and ratios are represented by rational numbers. No irrational number arises from measurement. So, are you also going to expel those numbers like you do with the ‘complex’ ones?

Yes, but not for the reason you offer. (Irrational numbers do arise from ratios just fine - e.g. ratio of circumference to diameter.) But, for number literals in any digital language, the most we need are arbitrary-precision floats.

We can have libraries for representing polynomials and series and symbolic representation of irrational or transcendental or complex numbers. And they can even be standardized libraries. But they do not need to be part of the language standard.

Yet, you cannot find even one example of a computation that becomes impossible or even prohibitively inconvenient.

Not true, but I've already said enough on that – read my previous post(s).

Negative numbers have been around for at least 2200 years. Irrational numbers […] at least 2700 years.

Not really. There were concepts, half-mathematical and half-philosophical, of such values, but the latter were treated very differently from positive integers and rationals. The true integration of negative numbers and irrationals into the concept of number only came with the development of the algebraic methods, much later.

Irrational numbers do arise from ratios just fine - e.g. ratio of circumference to diameter.

My words were: ‘No irrational number arises from measurement’. What you are refering to is a relation between abstract mathematical quantities, not measurement.

We can have libraries […] But they do not need to be part of the language standard.

As I said, there may be different opinions – this one is yours, and I have mine. As you may have observed, this is not a reason for me to attack your position as ‘arbitrary’, ‘unjustified’, ‘pointless’, ‘foolish’ etc.

I'll make this my last post in this discussion, unless you ask a question.

For an opinion to legitimately sway others, it must be justified. I don't need to agree with the premises of the justification because you might have different principles or priorities or experiences than I do. But I should be able at least to see a marginally cogent line of argument from your principles, priorities, and experiences to your conclusions.

I am not seeing this for your arguments that languages need numbers and geometric primitives. Your entire argument consists of repeating yourself, and now a circular position that you've repeated yourself enough and I should go re-read your previous repetitions. A statement that some computations becomes 'impossible or prohibitively impractical' is something you should be able to defend with an example (or at worst a non-constructive proof), even if it means digging up your own definitions for 'prohibitive'. Complex numbers and arbitrary precision integers and vector graphics have been developed in non-language-standard libraries before, which seems to disprove any such position. Turing completeness already proves 'impossible' (in any formal, mathematical terms) to be hyperbole.

You have argued that many modern languages include a rich tower of numbers. But this is not the same as arguing standardization of the tower to be necessary or even beneficial. You stated earlier: "basic datatypes, including text, numbers of all kinds, and symbols. This is the path most beaten in the last 60 years, and yet I cannot name a single language that ‘does it right’." I doubt that putting datatypes into the standard will somehow make them 'right'; it just standardizes the wrongness - an act that is neither necessary nor beneficial. Adding geometric primitives to your GPPL only ensures you'll be stuck with poorly considered geometric primitives for a long time.

If you feel yourself 'entitled' to voicing an opinion, that goes both ways: I am equally entitled to voicing my opinion about your opinion. You are free to attack my opinions as unjustified (and therefore worth less than used toilet tissue), but to avoid hypocrisy you must first be able to justify that my opinions are unjustified. I will happily reconsider any opinion whose justification is found wanting by a cogent argument (it's something I do on a regular basis anyway).

Regarding measurement: I already told you that by 'measurement', I do include values that arise from calculating measurements - including by adding measurements together, or multiplying them to get areas or volumes, or averages and statistical measurements. This is a valid interpretation of the word according to my dictionary (the act or process of ascertaining the extent, dimensions, or quantity of something; this act or process can include such things as rough visual comparisons, or multiplying length and width to measure area, or summing objects one at a time - aka counting them). If you're going to attack my position, you must attack my meaning, not just my words. I have explained my meaning by this word twice before.

Since you keep saying my opinions are not justified:

I did provide justification that I consider sufficient for an informal discussion between colleagues in a web forum – it is there for anyone unbiased. But, it is not a purpose of life for me to try to please anyone prejudiced. Especially if I am finding many of his own statements and ideas not only unsubstantiated, but based on notions and reasoning unacceptable to me. (The apparent lack of common language then precludes me from even wanting to ask for justification.)

For example, you insist on a notion of ‘measurement’ which is not only rather vague (regarding the calculations that you want to admit vs. those you don't), but also very uncommon. Although you claim that it ‘is a valid interpretation of the word according to my dictionary’, there is no mentioning of calculation or computation in the definitions of ‘measure’ and ‘measurement’ in any of the following dictionaries: Oxford Dictionary, Chambers 21st Century Dictionary, The Macmillan Dictionary, Merriam-Webster, American Heritage Dictionary, and Random House Dictionary. (In fact, most definitions directly exclude your interpretation.)

Even more importantly, it is totally unacceptable, in my understanding, to reason about practical issues in programming languages from Turing completeness or anything like that. When I am saying that certain calculations are impossible without e.g. complex numbers, I mean it in the mathematical sense (where the truth of my statement is a well known fact), and not that the Turing machine is fueled by complex numbers or that they cannot be modelled somehow.

... for not asking a question.

The concept of real numbers (that are not rational) came not from measuring (where fractions have always been sufficient for all practical needs) but from performing calculations with the measured, i.e. from (geometry and) algebra.

This is a very strange claim. Greek mathematicians encountered irrational numbers by measuring the diagonal of a unit square. The fact that non-fractional numbers arise from such mundane considerations bewildered them!

Ancient Greeks called such quantities incommensurable, recognizing the fact that one cannot be used to measure another, i.e. that the exact measurement is, in a sense, impossible. The true concept of irrational numbers is a later achievement, a result of developing algebraic methods (composing and solving quadratic and other equations, including ones that involve geometric data). So to speak, irrationality is where measurement fails, and it took centuries to understand how to deal with it properly.
By the way, even today it is not possible to draw a line segment precisely √2 the size of another :)

Could you elaborate?
The datatypes that I can think about which should be more widespread are big int / integer with overflow exception and interval arithmetic.
Or maybe you're thinking about immutable, persistent data structures as found in Clojure?

The Lisp languages are pretty good at handling numeric data. Haskell is even better, although at the expense of some (unavoidable?) complication. But neither is particularly well equipped for working with text. Other languages excel at text processing, but are wanting w.r.t. numeric calculations. Most languages lack a symbolic datatype, some even lack Booleans … My point is that all these are too important to neglect.

Please make sure your comments have subject lines, since otherwise it is impossible to navigate to them from the tracker.

There's a clickable blank space in there somewhere.

I decided the word impossible best conveys the level of inconvenience.

I would love to see how you substantiate the claims about Lisp and Haskell being good for numerical data and bad for text. I definitely don't see it.

I can only assume Mr. Bantchev is imagining some naive approach to text processing, such as using lists and Haskell Strings, which are known to be quite inefficient.

Haskell, via libraries such as Data.Iteratee and various template programming mechanisms (like Yesod's Shakespeare), is proving quite adept at text processing and generation.

My first impression from Oleg's description is that they are fold operations over non-blocking input sources, but I don't see how that generalises to output.

I could be wrong, but from looking at their definition and usage in Haskell, I'd say 'that the recently linked paper on the yield operator provides a nice generalization of Iteratee.

An Iteratee is a generic fold over a stream. Folds do generate output. In many cases, the output is incremental, and can be composed with another Iteratee.

What distinguishes an Iteratee from the many other stream-processing techniques in Haskell is two properties:

• Iteratees make it easy to support 'chunking' or buffering of input, e.g. to process an input string 4096 characters at a time. This enables a high level of performance.
• The monadic structure of Iteratees guards against many of the subtle lazy-evaluation problems common with Haskell IO processing.

From Oleg's description:

The ugly side of Lazy IO turns up when we run the code. As the size of the input file increases, from 2.4MB, to 4.9MB, to 24.8MB, so does the peak memory use of the lazy IO code: from 43MB to 78MB and finally, 337MB. Maximal residency grows too: from 20MB to 37MB and 163MB. The program must be reading the whole file in memory. That is not acceptable: we should not need 1/3-Gigabyte of memory to count words and characters in a 25-Megabyte file.

In contrast, the iteratee-based code counts in constant memory: the peak memory use is 2MB and the maximal residency is 108KB -- regardless of the size of the input file.

Folds do generate output.

Unfolds too, in this case perhaps an output stream.

The monadic structure of Iteratees guards against many of the subtle lazy-evaluation problems common with Haskell IO processing.

My guess is that this is because the monadic abstraction allows you to ensure the user works through an algebra of well-behaved operations on them.

This does sound good. I wonder if it can be reworked with Lawvere theories.

Both Lisp(s) and Haskell have bignums and fractions, and complex numbers. Both support different modes of integer coercion/division/remainder. Haskell even allows for finer differentiation between numeric types. In this respect, they are better than most languages.

As for text handling, however, Haskell's String datatype is but a list of characters, so one has to rely mostly on the general list processing capabilities of the language. This is both space- and time-inefficient (sometimes to the extent of being ineffective). In Lisps, strings are not lists, but handling them is nevertheless limited. For example, there is no form of pattern matching for searching and replacement. In this area, many of the today's languages (let alone Snobol, Icon, and other great oldies) are much better.

I am aware of the many external libraries that exist for Haskell – some of them precisely for text processing – but they are not a standard part of the language or its environment, and I am not sure whether they work on much more than a recent version of ghc.

Do you place more emphasis on the semantics (expressiveness of abstractions provided) or the runtime properties (space/time complexity)? I am not sure. Anyway, from what you say I wouldn't conclude that Haskell (say) is not good with text, compared to mainstream alternatives, which is what I thought was on the table. Anyway, let me just say that I like pandoc...

Semantics first, but not disregarding space&time efficiency. Treating strings as lists is both semantically inadequate and resource consumptive.
Virtually all mainstream languages of today provide random access to string elements plus a form of string pattern matching (usually regexes). Haskell, the standard language, has none of these. Perhaps it could have had, say, a standard parser combinator library (the technique is almost as old as the language itself), but for some reason it doesn't.

Would it be possible in some way to reduce the need for many types of aggregate data structures, with the compiler and the runtime environment figuring out what to use appropriately? In a sense, Lua is a good example here where the "table" is efficiently repurposed for arrays, hash maps, objects and modules. You could also imagine a Prolog engine monitoring the clauses being added to the database and dynamically constructing efficient data structures depending on what it sees. In other words, could less still be more as our world gets more complex?

A related question I have is about an enduring format for procedural knowledge today. Is JVM byte code the best choice today? ... Or is it scheme/lispy plain text syntax? .. Or even JavaScript?

how can we design languages whose programs can be transformed easily?

A better starting question is "Should we design languages whose programs can be transformed easily?" I believe the answer to be a very solid 'no'.

Supporting global (cross-module) program transforms is a Bad Thing because it:

• hurts local reasoning about program behavior
• violates modularity and leaks implementation details
• hinders integration of external services, since they aren't subject to the same transforms
• consequently creates pressure for monolithic applications that replicate services
• partitions the language's community, since libraries will come to depend on transforms in non-obvious ways (i.e. developed and debugged with the transform in place, even if the dependency is unintentional) and different transforms may be incompatible
• increases complexity exposed to language users, who are now faced with a combinatorial explosion of language variations (different sets of transforms)

These, BTW, are the same reasons I dislike language extensions, fexprs, and aspect oriented programming. Local transforms - e.g. user-defined syntax per module - is much more acceptable in-the-large and almost as effective (though one might need to add a dose of explicitly modeled frameworks or interpreters to make up the difference in abstractive power).

I do believe we should create languages that can make optimizations predictable, controllable, and easy. But optimizations are a very constrained class of transform - they must preserve behavior! And even for optimizations, I favor a focus on local properties, local annotations, inductive composition properties, and partial evaluation or staging - i.e. optimizations that respect modularity.

With respect to the edit-compile-test cycle and interactivity, I believe we should aim for continuous compilation, zero-button testing, live programming, and good support for simulating or confining an environment (which implies some sort of secure composition model, whether sandboxes or capabilities). With constraint-based linkers, I also open possibility of generating an unobtrusive list of 'possibly related' modules.

It seems you are considering only "active transformations", in the sense that they have a persistent semantics. Supporting one shot-transformations (AKA refactorings) seems like a good idea, and I don't think any of your contra- bullets really apply to them (except the one about creating a bias against external services, since they can't be refactored, but I don't see that buy vs. build trade-off going away). I do agree that active transformations are probably dangerous if not limited to those that preserve relevant properties (such as optimizations), but since Andrew mentioned interactivity, my mind jumped to refactoring.

Agreed. Refactoring does not exhibit the same issues. And, to some degree, we can automate refactoring with support from the IDE or user agents.

Of course, one-shot automated code-generation (wizards, skeletons, etc.) has its own problems. To the extent we perform one-shot refactoring, it really must be directed by a human - i.e. effectively an augmented editing process.

I'm all for powerful tools to enable and support refactoring. There is a lot of low-hanging fruit still available on a per-project scale, but I'm especially interested in cross-project refactoring, such that we have greater freedom to update widely-used libraries or services in ways that are not backwards compatible. (Even simple support for backlinks would go a long way, as would support for multiple versions of a module coexisting in one application.)

Refactoring doesn't have the bias against external services, we just need to control the code for the services too (e.g. imagine live programming with a shared code repo, such that we can casually modify both the client and the server code). The buy vs. build concern is separate: there is no reason we shouldn't be able to build services just as easily as we build libraries. The difficulty with 'actively' transforming services is that services are (a) shared by multiple clients (who might each want different transforms), and (b) rooted separately from the client (so it's unclear where the transforms would apply anyway).

I believe that an 'open' or scalable language requires the following property: modules provided by external services or registries (including FFI) are semantically indistinguishable from modules provided by source code.

I believe that you are using the word "behaviour" to mean several different things in your list of points. When we consider small-scale programs it is simple to separate results (the value computed) from the method of their delivery. In this context behaviour becomes the set of observable effects (internal temporal behaviour, external synchronisation etc) and can be treated separately from results. Only telling a few small lies we can say that we care about the results and the behaviour is irrelevant.

As soon as we increase the scale of the software that we consider it becomes harder to separate results from behaviour. Your points about composability and expectations follow naturally from our inability to untangle what we would like to programs to do, from the ways in which we would like it to be done. I would disagree about what I would like an optimisation to do, although I suspect it is only a matter of terminology: I would like the behaviour to change, but the result to stay the same. Of course my choices of desirable changes in behaviour are constrained by implicit and explicit requirements imposed by external pieces of code.

So I would agree with each of your bullet points, but argue that these provide excellent reasons for why we should design languages that allow easy transformation of programs :) Firstly I would point out that in my previous post I was very vague about what a transformation should be. What I had in mind (that probably didn't make it into the post at all) was a set of transformations that preserve results (ie functionally equivalent programs) and allow easy exploration of what we would term desirable changes in behaviour, while ruling out undesirable changes in behaviour.

Although I don't know how to achieve this I would guess that clearer expression of externally visible behaviour in language semantics would be the way to go...

By program 'behavior' I refer to observed, semantically controlled consequences, including both calculated results and communication effects (including synchronization). The important bit is: You should be able to determine relevant program behavior by studying the program and language, without special knowledge of its implementation. The remaining question, then, is how faithfully the program's behavior is implemented, whether there are abstraction violations or implementation leaks (aka 'noise').

It is often feasible to express that certain aspects of program semantics are irrelevant to behavior. We do this by not observing a behavior. I.e. we can send messages to null, drop a calculated result, encapsulate temporary state, and ultimately perform all sorts of 'dead-code' elimination of subprograms that contribute to irrelevant behaviors.

At scale, we would need a language semantics and idioms that make such analysis scalable - i.e. by use of partial evaluation, staging, and compositional optimizations. (For example: dead code elimination can be compositional if we can strongly couple the sources for a result to the sinks; this allows us to propagate 'deadness' of a result to its sources.) Other than that, there is no inherent problem with scale.

I would like the [effectful] behaviour to change, but the result to stay the same.

Rather than specifying a particular behavior and asking that the language 'change' it for you, I suggest you should express constraints on behavior and ask an interpreter or agent to choose the precise behavior for you.

This is a subtle shift in metaphor, but achieves a similar level of flexibility in a manner that is much cleaner semantically. It can be systematically leveraged if you have effective staging or partial evaluation. Secure programming models - i.e. that clearly constrain access to effects (e.g. capability security or monadic towers) are promising bases since they clearly delimit what effects are available to achieve a desired consequence.

I have pursued generative grammars, matchmakers (and linkers as constraint solvers), anticipation, and various cooperative (multi-agent) shared state models with en eye towards this sort of declarative meta-programming.

I invented RDP while trying to optimize certain matchmaker patterns, and one reason I favor RDP is that it promises to be an excellent substrate for staged meta-programming models.

So we have a set of properties of programs that we call behaviour (calculated results, observable effects such as communication or shared state etc). I would claim that for correctness we would want a transformation to preserve some of these properties, the rest we would want to alter freely to optimize some criteria. You say that it is a subtle change to instead express constraints over this set. I would agree that it is subtle, as I can't see the difference :)

Equality is simple a constraint, so ensuring that part of the set of behaviours remains equal is clearly a form of constraint. I can't see the advantage in using a weaker form of constraint for other behaviours in the set. Which are the properties (behaviours) that benefit from being constrained to a sub-set of their potential values, and what are the benefits?

I did not intent to suggest you build a program then express constraints over its behavior. I meant: build a program as a set of constraints over its behavior. There is no 'transform' at all, though there is a 'search' to find an instance subprogram that meets all constraints.

With constraint semantics, you need not alter the meaning of a program in order to achieve flexibility. Semantics remain sacrosanct. Any choice for actualized behavior is subject to every constraint. Basically, you have modules that adapt to a context, but you are still able to locally reason about the limits of behavior and adaption. Layers of modules further refine and constrain meaning.

By comparison, program transforms generally alter the behavior and meaning of a program. There are no pervasive constraints. There are no limits on externally imposed transforms. Each transformation 'layer' progressively strips meaning away from the program, until nothing but syntax remains. Local reasoning about the correctness of a subprogram is not possible, since you don't know the interpretation context.

Constraints weaker than 'equality' are valuable because it results in a more adaptable, extensible, and resilient subprogram or module - one that can be used in more contexts, allows mixing in more features, and possibly supports implicit fallbacks in case a service fails. Soft constraints - aka 'preferences' - are especially valuable, since they make it easy to present 'good defaults' then later override them, making code much more reusable.

Constraint programming has plenty of problems, e.g. with respect to predicative performance, security, composition, and scalability. But these are, IMO, nicer problems than those associated with program transforms. I don't believe constraint programming makes a very good basis for programming, but is widely useful across many domains and so would make a good generic library or service. (I'd love to have easy access to SMT solvers.)

That makes much more sense, thanks for the clarification. Have you got any good papers to recommend in the areas of reasoning in the presence of transformations vs design by constraints, or language semantics derived from constraints over program behaviour?

I've not seen any papers comparing the two.

For constraint programming, you can find some introductions in CTM or SICP. If you're already familiar with constraint programming, I would recommend perusing works on Concurrent Constraint Programming (cc) (e.g. Saraswat), Temporal Concurrent Constraint Programming (tcc), and other variations (such as 'soft' concurrent constraint). And maybe add some grammar systems, since those are interesting and related.

Oh, and search for SMT Solvers as well, since that's among the more promising generic techniques for implementing constraint models.

I've covered some of this in my live programming work. Here is how we could improve things:

1. Provide lots of assistance when writing a program. Not just intellisense, but recommendations on what code to write next based on what code was written (perhaps use deep belief neural networks trained on a large corpus of existing code, but I'm not exactly sure how to do this yet :) ).
2. Compilation should be real time as you type. I'm not even talking 500 milliseconds behind, but at every keystroke.
3. The biggest drawback with theorem proving is that it requires a formal spec while many programming tasks will never get one. Or perhaps you really don't know what you want yet and you are doing exploratory programming. In anycase, we should focus on better debugging tools and faster turn around times on making changes to making observations.

Check out Conway's game of life developed in APL. When writing an APL one liner (or /bin/sh scripts for that matter) one often builds up the program a little bit at a time, always giving it some sample input. Since one can do so much in just a single line, building it up organically reduces the potential to cause massive damage.

Also check out live coding in Impromptu, Extempore etc. Here's a good example: Face to Face by Andrew Sorenson. As the code is being edited, it changes music and graphics. The editor shows what construct is being edited by changing the background of that construct, macros get expanded instantly (when used as templates) and so on.

Here a program is developed to generate live music or motion or graphics etc. Perhaps we can flip things around and use generated musical notes/graphics to indicate some properties of a program as it is being developed! Insert "test points" as it were (just like in a hardware schematic) and make it real easy to feed sample inputs. We might then be able to tell by our ear if a program misbehaves! [music => success, noise => failure, frequency => speed, spectrum => range covered etc.] One can connect to different test points and hear a different tune and soon you start to recognize characteristic sounds. We are far more capable of detecting an unusual sound than that one line of log output (out of thousands) that tell you something is wrong. Alternative ways of visualizing the same program (or test data) should also be possible.

So I think interactivity needs to involve other modes of communication. Replace something incredibly baroque like Eclipse with something that can make coding, testing and debugging fun, flexible and efficient. I would call this "live coding"!

I completely agree with this. More feedback, more emphasis on visualization and exploratory programming. Pull-back from architecture-oriented languages and text-based IDEs. Think of how programming could be fun and productive on an iPhone.

Yep. At Onward this year, there will be two talks in this area: one about TouchStudio, and mine about programming on tablets.

This is definitely a new/mostly untapped area for PL researchers.

It sounds similar but rather more focussed than the Feyerabend workshops that Richard P. Gabriel was pushing last decade. What became of them, I wonder?