Lambda the Ultimate

Ah, thanks for reminding. I should've linked to Guy Steele's post. Here's an alternate link to the same post.

I have a strange feeling that this and most of previously introduced Java5 features are there because C# has them. They are mostly basic things present in various languages for decades, but Java had to wait 10 years...

If you look at the syntax for closures you'll see constructs similar to Ruby usage of blocks.

Too bad they restrict that special construct to closures without arguments. Most usage of blocks in ruby is with arguments.

anonymous inner classes

Inner.java:8: local variable n is accessed from within inner class; needs to be declared final
          new Func(){ public void eval(){ n++; } });

reduce to amount of code a bit
You seem not to have compared the code at call sites.

new Func(){ public void eval(){ n++; } }

(){ n++; }

Thats analgous to ocaml. You have to use the java equivalent of 'ref', for example:

int[] n = new int[] { 1 };


If Java was a language without variables, which relied on array indirection to provide mutable state, you might have a point.

Java provides variables, we just can't get at them from within an inner class.

You can get at outer variables, you just can't mutate them.

It's a design choice (based on simplicity and runtime-performance) to allow only reading it. In this way its similar for example to providing seperate 'int' and 'Integer' which isn't really nice but allows for good integer performance even with a simple compiler/vm. And because there's always a workaround in the rare cases where you really need to mutate a variable from a closure I don't consider it a problem.

In cases the outer variables are nowhere assigned to, it's trivial for a compiler to infer that fact statically and use the same technique as with final variables today. Same performance.

In cases they are assigned to, the standard workaround is to put them in a mutable container. Since there is no builtin polymorphic one-value container, an array is typically used. This stores the size unnecessarily (always 1) and has unnecessary bounds checking.

One could design customized one-value containers with this overhead removed. There would have to be a separate class definition for each type, because Java 5 doesn't support parametric polymorphism over builtin types, and even using a single generic class for reference types would include unnecessary casts.

Why force programmers to write by hand what the compiler can easily automatize? It's not rocket science, closures over mutable variables were present in various languages decades ago (Lisp, Smalltalk).

The official excuse was avoiding implicit allocation without the new keyword apparent in the source. Which is pretty ironic, given that later Java has added autoboxing.

The aim at language simplicity has backfired. In the future Java will have function closures, but the old APIs with one-method interfaces will not go away, and there will be separate rules which allow conversion between the two conventions. The language will be more complex than it would be if it had supported function closures from the beginning.

If it had supported function closures, perhaps there would have been no need to add anonymous inner classes (somehow I haven't heard of another language which has them), and it would be simpler still.

A similar story is with generics. They don't play well together with arrays and have a pretty complex set of rules because generics were not present since the beginning. In effect, arrays (which were present since the beginning) archieve a similar result in an incompatible way, and the desire of compatibility with older VMs forced to compromise the design of generics.

Or perhaps it's not just failure to include important features early. C# has generics designed better even though it too has added them late.

"If it had supported function closures, perhaps there would have been no need to add anonymous inner classes"

I actually think anonymous inner classes are useful whether or not function closures exist.

Also, anonymous inner classes can access non-final data in objects, just not in the enclosing functions.

I rely on this feature quite heavily in my own messaging-based applications. I have an interface that looks like this:

public abstract class Message implements Serializable { 
   public void process(Context ctx); 
}

/* in some function */
   sendMessage(destination, new Message() { 
      public void process(Context ctx) { 
         /* perform destination work here */ 
      } });

This also allows extensibility, so that if I want to add optional members to "Message" I can just add extra variables and methods. It also has full access to the containing class's variables.

I think it's a pretty powerful feature. I want Java to have closures, but I'll still keep the anonymous inner classes.

I took the liberty of reformatting the long lines in the parent post's code, to address a page width problem. Any appearance of extra verbosity as a result is entirely coincidental. ;)

"You can get at outer variables, you just can't mutate them."

Inner.java:8: local variable n is accessed from within inner class; needs to be declared final
          new Func(){ public void eval(){ int a = n; } });

We can get to constants, we cannot get to variables.

finals aren't constants. Writing

final int x = a + b;
...

in Java is equal to

let x = a + b in ...

in Ocaml. In both cases 'a' and 'b' aren't constants so x isn't a constant either.

If you're saying that the values of 'a' and 'b' aren't known at compile time, then, yes, 'x' is an immutable variable whose value is determined at runtime.

If you're not saying that, then assuming we're talking about ints here, then whether or not 'a' and 'b' are constants is irrelevant, they are added and 'x' is set to that value. In both cases, 'x' cannot be changed.

In the case of a final reference, Java does not allow reassigning the reference once it has been assigned, but the values contained in the reference can be changed. In other words, the reference will always refer to the same object, but that object can be modified. OCaml also works this way.

Karsten, you seem to be quarreling for quarreling's sake.

It's clear that when you wrote "You can get at outer variables, you just can't mutate them" that wasn't correct for non-final local variables.

Uh... if you replace a regular local variable with a final box, doesn't that have the same effect as static in C, by causing all activations of the function to use the same piece of storage? Or do I misunderstand the meaning of final in that context?

Java final is similar to C/C++ const.

Final means const for local variables. There is no way in Java to have a method-scoped variable shared between method invocations. There are no static local variables, and inner classes cannot have static variables.

'final' in this context works exactly like 'let' in ML. If we have

void someMethod(int val) {
   final int x = val*2;
   ...
}

then 'x' has a different value for each call to 'someMethod' with a different value for 'val'. You simply can't change this value somewhere in the '...' block, you can only read it. And because 'x' can get a new value for each call to 'someMethod', I won't call 'x' a constant.

Now let's write:

void someMethod(int val) {
   final int[] x = new int[] { val*2 };
   ...
}

This time, the reference to the array (containing a single 'val*2') is created and stored into 'x' for each call to someMethod(). You can't assign to 'x' in the block '...', but you can of course change the content of the array. And because x is final, this would also work from any annonymous inner class in '...'.

I suspect most people discussing this are well aware of anonymous inner classes. Certainly, Gilad Bracha is. ;)

The syntactic overhead is a pretty severe issue for anonymous inner classes, and this inhibits them from being used in many cases for which you would naturally use closures in other languages. I've been involved on systems which have used them fairly heavily, to abstract out particular patterns of exception handling and logging, which otherwise would require something like AOP. But the syntactic heaviness of anonymous inner classes makes it debatable whether they're worthwhile, in many cases.

The examples you gave "cheated": they showed cases where the closure had already been created, and you were just calling it. You also showed a case with no arguments. The realistic situation is that you have to declare a different interface for every anonymous method signature you use, and you can't declare those interfaces inline. In addition, every different type of anonymous inner class requires two relevant names: the name of the interface, and the name of a method to call, making the word "anonymous" a bit misleading.

The minimum overhead for instantiating a closure with a signature that you use only once in a program looks like this:

interface RequiredInterface { public void notSoAnonymous(int x) };
    ...
    new RequiredInterface () {
        public void notSoAnonymous(int x) {
            ...
        }
    }

The overhead of the separate interface definition is of course amortized the more often you create instances of that type. But regardless, that's a lot of text for a feature which, in other languages, boasts conciseness as one of its major benefits. It makes programs which make heavy use of closures difficult to read. Compare this to Haskell's \x -> ... or ML's fn x => ..., or equivalents in a whole host of languages including Scheme, Smalltalk, Ruby, Perl, Python, Lisp, Javascript, Groovy.

So while technically speaking, anonymous inner classes are closures, they don't provide the benefits that closures do in other languages. Java needs "real" closures.

The realistic situation is that you have to declare a different interface for every anonymous method signature you use, and you can't declare those interfaces inline. In addition, every different type of anonymous inner class requires two relevant names: the name of the interface, and the name of a method to call, making the word "anonymous" a bit misleading.

If performance isn't critical, you can do with just one abstract class or interface and reflection.

Edit: Actually, you don't even need to use reflection. But you do need generics if you want typed method signatures.

Sure there is a code overhead, but it's quite normal for Java. The use of most Java features is very verbose which a a well known issue for many people. If you really want to takle this problem you have to discard Java altogether.

If you want to use closure you have to declare them too. But in many cases you can use generics to reduce the amount of redeclaration required. For example you can declare

interface Function1[Res, P1] {
Res eval(P1 param1, P2, param2);
}
interface Function2[Res, P1, P2] {
Res eval(P1 param1, P2, param2);
}
// etc

Delcation of closures is always neccessary in Java unless you want to incooperate type inference into the language which is the real reason for the conseise syntax of closure usage in functional and also dynamically typed languages like thoise you mentioned. Imagine for example a ML where you have to explicitly write out every type. This (type-inference or dynamic typing) is the real reason for the conciseness of the languages you mentioned. So giving Java closures wouldn't solve this problem.

Sure there is a code overhead, but it's quite normal for Java. The use of most Java features is very verbose which a a well known issue for many people. If you really want to takle this problem you have to discard Java altogether.

Saying, in effect, "that's just how it is" might be a statement of fact, but it's not an argument for keeping things the way they are, especially if Java is to remain competitive.

The point is that the overhead for this feature in Java is large enough to inhibit use of the feature, which leads people to use alternative solutions. Imagine if the overhead for declaring and using methods was large enough that people worked around it by using alternatives to methods: cutting & pasting code, perhaps. That's exactly the situation in this case - repeated code patterns are one of the results when people decide, sometimes quite consciously and reasonably, to avoid using anonymous inner classes.

This (type-inference or dynamic typing) is the real reason for the conciseness of the languages you mentioned. So giving Java closures wouldn't solve this problem.

It's one factor. But even C and C++, which have similar type systems to Java, have more lightweight syntaxes for dealing with function pointers (their lack of closure is unrelated to the type declaration issue).

The additional problem Java has is that it has no independent unit of behavioral abstraction smaller than the class. The only other major OO language that made that choice, afaik, is the original version of C#, which simply copied Java's design in this respect. I think hindsight shows that it's not such a great choice.

C#, which is by design quite similar to Java, has been actively dealing with this issue, first with "delegates", and in a future version, apparently full lambda expressions which will have "additional type inferencing features and a more natural syntax than the clunky delegate syntax" (from here). Java doesn't have any good excuses for not improving its support for this very basic kind of abstraction. I'm very glad to see Gilad bring it up.

The topic is "Closures for Java or money back" and I simply wanted to make clear that Java already have closures. And if somebody writes an article with the title "adding closures to Java" the title alone displays a lack of understanding because Java already got closures years ago.

Creating and using closures in Java is a non-problem with a modern IDE (and every serious Java programmer is using such a thing). Sure, it's verbose, but Java simply is a very verbose programming language. Thats true for nearly all language features, so whats the point with more concise closures? Adding another way of creating closures would just create redundancy.

Also Java isn't a functional programing language, so maybe closures simply aren't important for Java and maybe it's even sensible to discourage programmers of using a functional programming style.

And what about the standard libs (expecially Swing with heavy inner-class usage)? Rewrite it to a new syntax? And if not, how to decide which syntax to use, the old one or the new? I simply don't see widespread closure usage in Java only because a new syntax reduces the code by one or two lines at the call site.

C++ function pointers are ugly and also much more verbose then closure usage in a language like Haskell or Ocaml. So real closure-support requires (at least limited) type-inference, tupels (for multiple return arguments) and currying to make it's use really comfortable. Lots of stuff for something you can already do with the language. And why not simply use a language like Scala instead of Java?

C# is a typical 'melting-pot-language': Instead of deciding which way to go, the designers simply put everything into the language in hope it will be useful for somebody. I thing thats a very poor way of creating a programming language.

Anonymous classes in Java are closures in the same way structs and function pointers are in C.

On verbosity: every added syntactic construct is more code that could be a source of errors. It makes sense to eliminate any verbosity that isn't necessary, e.g. the use anonymous classes instead of closures.

No, in C++ there is no way to capture the value of locals into the closure, so you have to emulate this by putting them explicitly into some structure.

Inner-classes in Java are implemented like closures in Ocaml. Thus the 'problem' with access to non-final variables which is a non-problem in Ocaml, because there every 'variable' is 'final' (immutable).

Karsten Wagner: No, in C++ there is no way to capture the value of locals into the closure, so you have to emulate this by putting them explicitly into some structure.

True, but then you map the structure onto the stack, and there's your closure; see the Phoenix docs for details. This is a really excellent facility to have.

Karsten: Inner-classes in Java are implemented like closures in Ocaml. Thus the 'problem' with access to non-final variables which is a non-problem in Ocaml, because there every 'variable' is 'final' (immutable).

Except when you close over refs, of course.

A refs simply is a small structure with a single mutable component.

In ocaml closures are implemented (as least some years ago when I looked last at the source of the Ocaml compilers) exactly idential to javas inner classes: All outer bindings which are used inside the closure are simply copied into the a struct at closure creation time and are constant for the time this particular closure exists.

Because bindings in Ocaml are immutable this works and is much faster and less memory extensive than the 'old' method of doing closures (using heap-based frames for functions).

Java inner classes works exactly the same, but because java allows mutable variables it has the 'final' requirement.

True, but then you map the structure onto the stack, and there's your closure; see the Phoenix docs for details. This is a really excellent facility to have.

You should have look at Phoenix 2.0 instead which has full lexical closures and is much more aesthetically pleasing

Eliminate anonymous classes in favor of closures. As tool implementer, I can assure removing the various sorts of class nesting from Java would make my life much easier, far more so than adding closures would complicate it.

That's impossible because it would break the existing code-base.

I agree about the topic - it should perhaps say "Usable closures for Java..." But I think most people familiar with Java and familiar with closures from some other context know exactly what the current topic title means.

I don't agree that creating and using closures in Java is a non-problem, because even modern IDEs don't hide the extra code, so use of anonymous inner classes makes Java code harder to read. (See Dave Griffith's example elsewhere in the thread.) I've had discussions with developers about whether to use anonymous inner classes in certain situations, or just accept one of the alternatives, and we wouldn't need to have those discussions if closures were more lightweight.

Also Java isn't a functional programing language, so maybe closures simply aren't important for Java and maybe it's even sensible to discourage programmers of using a functional programming style.

And what about the standard libs (expecially Swing with heavy inner-class usage)?

Those two statements contract each other: Swing, a standard Java library, relies heavily on closures, so you can't say they're not important to Java. Further, the reason that Swing relies on closures is not arbitrary: "callbacks" of some kind are a standard approach for GUI libraries in many languages.

It's quite common in C and C++, when using callbacks, to encounter issues with the fact that function pointers don't reference closures. This shows that the need for closures arises outside the context of functional programming. Smalltalk is hardly a functional language, and it includes closures. All the scripting languages that support closures do so because they're useful for things as simple as callbacks.

The reason that closures are important, even in languages that are not ostensibly functional languages, is that the notion of a closure is just a generalization of the concept of a function (or method). Languages that have functions (or methods) but don't support closures are imposing an arbitrary limitation on abstraction.

A function is just a way of taking a block of code and abstracting (via naming) certain values needed in that block, so that the block can be reused with different values. The reasons to restrict such abstraction only to top-level blocks of code are weak at best, and mainly boil down to making a bit more work for the language implementor. But not doing that work robs the language user of a lot of expressive power.

And why not simply use a language like Scala instead of Java?

That may be a fine choice for an individual to make, but as long Java is one of the most widely used programming languages, the features it offers affect many people (including me). If its feature set is improved, that improves the programming language ecosystem in a variety of ways. [Edit: It even affects what academic research is done, since naturally quite a bit of research is devoted to Java, and academic programming language conferences often include many talks about Java.]

I'm very glad to see Gilad bring it up.

and

... but as long Java is one of the most widely used programming languages, the features it offers affect many people (including me). If its feature set is improved, that improves the programming language ecosystem in a variety of ways.

Sorry to sound so harsh, but I consider Java broken by design. The reasons for that are deeper than just the obtuse syntax (there are many informed critiques of Java on the web and I'm not going to repeat them here). Adding more convenient syntax for closures to Java will not fix the design mistakes in the semantics. As Gerald Weinberg might comment, it just serves to prolong the life of Java. I'd much rather see use of Java decreasing in favor of the more informed language designs.

I could easily give you an informed critique of Java myself, or an informed defense of it if you want to argue (not here, though...) But I earn a living in the commercial world and no amount of "truths that might hurt" will make Java go away any time soon. Adding more convenient syntax for closures will ease some of the pain of dealing with it, and that's part of what makes me glad.

I'm also glad to see important but "foreign" features being acknowledged and implemented, because it bodes well for the future of commercial language development in general. As some of Gilad's comments indicate, it's not just the language designers that are the issue, but the ecosystem that they have to operate in, and it takes time (years, decades) for ideas to permeate that ecosystem.

I don't think it's a foregone conclusion that the broader market will move towards "more informed language designs" in the form of languages like Haskell or an ML (not even OCaml). If history is any guide, it's much more likely to be a process of technology transfer between languages, so to me, the important thing is to see awareness and familiarity with important features migrating into the mainstream, because it makes it more likely that BigCompanyLang 7.0 will be less hostile to abstraction.

Well, since this is going too off-topic, I'll just say one more thing. I don't think that Dijkstra was ineffective. We need more awareness and good old fashioned debate can be quite succesful at that.

Well, I question the effectiveness of Dijkstra's "Truths that might hurt" rant, even if I agree with its sentiment. What I suspect you actually mean when you say that you consider Java "broken by design" is that Java, like other mainstream languages, was designed in response to, and succeeded in the presence of, a set of forces which you either also consider broken, or are unaware of. (If you think it comes down to your subject line quote, I think you are sorely mistaken.)

Most of those forces don't change much in the short or medium term, and of all the things that can affect them, academic rants are probably the least effective. OTOH, infecting languages with useful and important features can be a force for change: as Dave points out, with function closures or something equally lightweight, the uses for anonymous inner classes largely disappear, and future language versions might have more sensible designs.

Well, I question the effectiveness of Dijkstra's "Truths that might hurt" rant, even if I agree with its sentiment.

Dijsktra wrote more than one rant. I think that his approach was succesful enough at genering discussions. However, I didn't bring up Dijkstra.

What I was saying is that I consider adding more convenient syntax for closures to Java as makeup. It quite simply doesn't improve the language considerably. The design flaws of Java are deeper than just syntax.

What I suspect you actually mean when you say that you consider Java "broken by design" is that Java, like other mainstream languages, was designed in response to, and succeeded in the presence of, a set of forces which you either also consider broken, or are unaware of.

I meant that Java contains a number of fundamental (technical) design flaws. I doubt that fixing many of the flaws would have had any negative impact on the success of the language.

(If you think it comes down to your subject line quote, I think you are sorely mistaken.)

I disagree. I think that the subject line I used has a lot to do with Java. Java quite simply isn't an example of a language designed, promoted, and ultimately mandated for altruistic motivations.

I, too, work in the commercial world (programming in Scheme, C, and SML at the moment) and I'd love to see more commercial software development done in languages such as Scheme, Common Lisp, SML, O'Caml, Alice ML, Haskell, Clean, or Scala to name a few (in no particular order).

I meant that Java contains a number of fundamental (technical) design flaws.

I suggest elaborating this, instead of arguing about the latin quotes...

In fact, if not directly related to closures, I think it would be best to start a new thread, and continue the discussion there. Personally, I am starting to lost track of the discussion.

I suggest elaborating this, instead of arguing about the latin quotes...

That's a fair suggestion, but, like I indicated earlier, I think that there are informed critiques of Java on the web and, IMO, adding one more would be mostly waste of time.

It's not helpful to claim "Java is funadmentally broken" and substantiate it with "see internet for details". For every web page that criticizes a particular Java feature, there's another with the opposite opinion.

BTW, I happen to agree that Java is too far away from my current (very fuzzy) notion of the "ideal language" and that there's no way that Sun will make the necessary changes fast enough. I'm hoping that Scala supplants it (and I think that this could happen relatively soon, because it has a decent amount of integration with Java and its syntax is more Java-like than OCaml or Haskell).

It's not helpful [...]

I agree. In fact, I wasn't even trying to be helpful. I was rather voicing an opinion.

Anyway, for insightful notes on the design flaws of Java/JVM you could do worse than read various articles that describe the difficulties of implementing interpreters and compilers targeting Java/JVM.

Why such difficulties are relevant?

Because those are the same kind of difficulties that one encounters while trying to implement powerful abstractions and package them as libraries and frameworks. Further, compilers targeting Java/JVM do not even have to care about the syntax of Java, so the source of the difficulties is the semantics of Java/JVM.

If you read those kind of articles, you'll notice that the semantics of Java/JVM are not particularly friendly to defining new kinds of control abstractions or new kinds of data types. To implement those efficiently, or to even to be able to implement them at all, compilers targeting Java/JVM perform extensive (even whole program) transformations. Such transformations are not viable for application and library authors targeting the Java language.

Have you ever wondered why code generation tools are so pervasive in the Java world? Well, now you know why. The semantics of Java are so weak that code generation is often the only viable approach. (The same applies to reflection.)

I don't want discuss the pros and cons of closures because I know their importance. BUT: Java already has closures. Maybe with a little verbose syntax but they exist for years now. And lots of programmers obviously aren't aware of this, because otherwise you wouldn't always hear those people say "Jave needs closures, please give Java closures". So I suspect that there are lots of people who really haven't understood the concept because they aren't able to spot it only because Java hasn't called it 'closure' but 'anonymous inner class' instead.

Also from the language design point of view, inner classes are a really nice idea: They are real closures but very well in the framework of the rest of the language. And you can easily extend a closures to more then one 'call-in'-point.

In Java there are no 'functions', only classes which can have methods.
So whats the point in creating 'stand-alone' functions now? I can only repeat that I consider this really bad (I would even call it uninformed) language design.

In #comment-20547, Zen master Anton wrote:

And why not simply use a language like Scala instead of Java?

That may be a fine choice for an individual to make, but as long Java is one of the most widely used programming languages, the features it offers affect many people (including me).

What rossjudson said.

As jwz pointed out years ago:

The fact is that there are four completely different things that go by the name Java:

• A language;
• An enormous class library;
• A virtual machine;
• A security model.

I haven't written anything in Scala yet, but having read Scala by Example [PDF, 985K], I have to say it looks like an insanely pleasant language to write in for the JVM. By switching to Scala, you only give up a single layer in jwz's proverbial four-layered Java cake, namely, the language. The library, the VM, the security model and, to a large extent, the community are there for you to leverage.

What's not to like?

I agree, there seems to be little not to like about Scala, certainly in the context of Java (although for my preference, I'd say the same about e.g. SISC Scheme, or Kawa, or JScheme). But most commercial users of Java are not rushing to use some other surface language with the JVM, so the features of Java the language are still important.

Further, if closure support were added in some way beyond lighter-weight syntax for anonymous inner classes, then it would benefit JVM languages like Scala, Groovy, and the JVM Schemes, so either way, closures for Java are a win.

but I'm a realtively simple person, and the discussions about new-fangled things one can do in the language tend to involve a lot of, shall we say, "interesting" typing. I am all for more powerful type systems so we can say precisely what we want to say, but at the same time I can really see that Java got it right wrt appealing to a large population when they didn't go crazy with the type sytem. Scala requires folks to invest more brain power, and I wonder how much of a barrier that is / will be?

Add to your list:

-the extremely high-powered tooling
-the enormous recruiting pool

Sadly, you lose both of those with Scala, which makes it very problematic for enterprise use.

Right on; I so want to be able to use JML et al. with Scala... more is the pity. (Also, I suspect there are tools which on the face of it should work with Scala because they work on bytecode, but in the end I'm sure there will be pain for obscure technical reasons. Heck, regular Java code most of the time blows up if you try to use more than one tool that does bytecode injection.)

raould: Right on; I so want to be able to use JML et al. with Scala... more is the pity.

I'm about 90% sure that Scala's type system is already as expressive as JML is, so using JML with Scala would be redundant!

Not if you're interoperating with Java-with-JML code?

Phillipa Cowderoy: Not if you're interoperating with Java-with-JML code?

You may be right about this—I suppose it depends upon the extent to which JML's Runtime Assertion Checker and and Scala's dynamic semantics overlap, or whether they could be made to overlap with a minimum of user effort somehow.

JetBrains seems to be launching a project to add Scala support to IntelliJ IDEA. Assuming they go through with it, most of the "extremely high-powered tooling" issue would seem to be solved.

http://intellij.net/forums/thread.jspa?threadID=257345&tstart=0

Very interesting. Does this mean that Scala is getting adopted by Java shops? I've been watching Scala from the side-lines by reading the many great papers the Scala group has been publishing, and I must admit I had cynically written Scala off as a beautiful theoretical exercise that would never gain traction in the commercial programming world. I would love to be proven wrong about that!

As someone with no direct inside knowlege but who works a lot with the players, I'd say this is more likely to be a personal project of one of the JetBrains dev team, probably Eugene Vigdorchik, rather than driven by a view to a growing Scala market. JetBrains does that a lot, and has some great functionality that started out that way. For instance, their new COO created their new Google Web Toolkit integration as a finger-exercise to see if it could be done.

That said, it would be quite cool if this tooling could start some commercial movement in favor of Scala. Right now, I'm unaware of any large Scala projects, and it doesn't even make the top fifty on the TIOBE index. If this tooling ships, and is up to JetBrains usual quality and usability standards, I would be very surprised if that didn't change. If anything, Scala is an even better fit for IDEA-like automation than Java is.

C# is a typical 'melting-pot-language': Instead of deciding which way to go, the designers simply put everything into the language in hope it will be useful for somebody. I thing thats a very poor way of creating a programming language.

Some time ago I saw a video interview where Anders Hejlsberg was asked whether he would include more "functional-style" features in C#; his answer was that it's difficult, because for every new feature they add they would eventually have to remove some other feature to keep the language clean. So, I'd say that your claim about C# is wrong.

FWIW, the fact that Java5 added some features originally found in C# (enums, annotations/attributes, variable arguments) shows that some Java designers seem to think so, too ;-)

I believe Anders Hejlsberg that he thought this way. But if you look at the development of C# in the last years, its obvious that he changed his mind. Maybe because of pressure from above or pressure from the community. And this is happening with Java too and I suspect it will kill the language eventually making room for something new.

Putting more features in a language most often looks as a good idea in the short term but is often a very bad idea in the long term. It makes the language more complex, more difficult to learn and to use and makes the code less homogenous.

While Java wasn't everything then a beauty it had the advantage of promoting a very certain straight-forward programming style and this lead to easy code reuse (even if the code sometimes was very verbose or even ugly). But if they put too many features in the language there are stills heaps of legacy code and older progammers who continue to use the 'old' features. So the code base will get less and less homogenous and it will be increasingly difficult to read and reuse code because of the combinatorical explosion of all those new and old idioms.

And this is happening with Java too and I suspect it will kill the language eventually making room for something new.

(Might be of interest that anonynomous inner classes was a hack that was added to the language spec well after version 1.0).

I have no problem with sensible enhancements, but even for Java 5 there were some very bad design decisions. Particulary auto-boxing/unboxing and the way generics are implemented (type-erasure). And if they really implement those second way do do closures in 7, it will get even worse I've imagined.

In fact there are some nice small enhancments which would fit Java well without bloating the language to much. For example a 'using' feature (for automatically closing Closables), adding a counter-var to for-each loops etc. Those things would tackle some common uglinesses without making the language to 'soft' (= giving to many different ways to create the same thing).

Once you've accepted the fact that all PLs are a moving target - subject to change - then the next question is who's in charge of determining which changes to make and which to reject? PLs are very much about communities and stewards.

Personally, I think the biggest problem with Java (in the PL sense) is that they have been too conservative in the past. That conservatism has meant some changes that are slightly more radical (like generics) were rejected in favor of features that require minimal changes to the JVM and libraries. For example, inner classes (including the anonymous variety) were implemented with the requirement for no change to the JVM - making them name-mangled normal classes by the compiler. The JVM has changed very little in the last 15 years, and all changes to the language have had to work around that fact.

The other disadvantage of being PL conservative is that other languages determine the range of enhancements that are going to be clamored for - i.e. the 'using' feature you ask for is a borrow from C#. Of course, if all you are trying to do is keep your current throng happy, then it might make sense to pursue a strategy that is simply aimed at maintaining mind share.

I agree that the conservatism considering JVM changes is a real problem with Java. Those Java 5 generics without proper JVM support are a joke. And useful things like value-types (Sather/Eiffel terminology is 'expanded types' and C# calls it 'structs') will never happen because of this.

The problem with all those exentsions is that's its driven by the masses. The more people cry 'I want closures' the more probable it is that they get what they want (even if it already exists, just under a different name). And of course because it's implementable purly in javac. And decisions based on mass apeal are seldom sensible. But even then: Without changing the JVM it will probably still perform poorly, because every closure still has to be a separate class. And mutable parms still have to be boxed. And if closure usage explodes because it looks more 'cheap', Java programs will use more memory and are slower.

The call for 'using' is simply a result of checked exceptions. I've nothing agains them, but without something like 'using' (I wouldn't call it 'using' and reuse the 'try' keyword for the Java version instead) it's sometimes really a pain to make do it correctly.

But the real difference between 'using' and another closure implementation is that having 'using' won't change the way you use important parts of Java, it would simply make some things a bit easier and prevents some common errors. Those kind's of changes are good (like the for-each loop which is was also a good addition to the language).

But even then: Without changing the JVM it will probably still perform poorly, because every closure still has to be a separate class.

I'm pretty sure it's a bit early to be making assumptions about performance characteristics of the implementation something that was literally proposed just yesterday.

One possible reason that you might be wrong is that instances of stateless closures (which seem to be the most common use case in my experience) could be safely cached by the compiler. This would save the (admittedly slight) overhead of creating closure objects on the heap.

The bigger meta-reason you might be wrong is that closures, being a lot simpler than anonymous inners, provide the guys building JITs a lot more scope for their nefarious and arcane optimizations. Don't be surprised when they take advantage of it.

Those Java 5 generics without proper JVM support are a joke.

I assume you have a better answer than erasure, then? One that complies with all the major constraints, preferably? I'd be very interested to know what it is.

Maybe we should open a thread dedicated to discussing just that, as I don't recall one existing. Maybe that would actually bring forth something better than erasure, or otherwise shut the "fix generics" crowd for good.

I'd be very interested in a such a discussion. There certainly are a lot of smart minds around here to have it. On the other hand, it is hard for me to believe the people who came up with erasure were either just dumb or joking.

I shy away from opening that thread myself, given the age of the subject, and the fact that I can't seem to find any of the several papers I've read discussing it (to no better conclusion than erasure, I should say).

The constraint of being compatible with a poor VM and a poor old version of a language makes a good design impossible.

Good design may very well be impossible now, but that does not make the constraint of standing by the present design unreasonable.

If the language and/or the VM is unacceptable to you, then just don't use it.

Those of us that have used it with some degree of success for the past 10 years want it incrementally improved where it can be, preferably without breaking compatibility with those 10 years of perfectly acceptable and well tested code; then, if anything, we want the chance to move that old code forward gradually.

What's unreasonable, in my view, is to think we should just start from scratch every now and then and simply forsake compatibility. Everyone is free to design new, radically different and better languages, targeting the VM or not. Better languages do exist, Java is not even a favorite for me. It is only Java that I feel must stay compatible with Java - what's unreasonable in that?

It would have been possible to add new features to the VM that retained backward compatibility with all that well tested code. However, the constraint was also that all new java features had to be able to run on the old VMs (forward compatible).

I'm sorry, but that does not seem to be the case. That was a constraint for GJ, on which the JSR-14 process was inspired, but not for that JSR itself.

Java 5 features are already not accessible from older VMs, and Sun didn't just give up on that on the last minute (as some seem to believe). The fact that the VM didn't need a significant update because of generics is seen as an advantage (since bytecode manipulation code needs not be changed), but it does not seem to have been a hard requirement.

The fact that old code can run unmodified on newer VMs was in fact such a hard requirement. But that includes having code that knows nothing of generics being able to obliviously use fully generified libraries, such as the ones on the standard library. Erasure was thought to be the best way to have both generified and ungenerified code coexist peacefully on newer VMs - not on older VMs.

Other options do exist, and were considered by the JCP, such as NextGen: the website and the initial paper (finally found it). The limitations to interoperability between old and new code are recognized and discussed on Section 5.2 of the paper.

I'd just like to see the pros and cons of something like this (including the more recent developments) discussed in a constructive way, rather constantly seeing people simply dismiss erasure as a bad joke.

In comment-20945, Nuno Cruces wrote:

Maybe we should open a thread dedicated to discussing just that, as I don't recall one existing. Maybe that would actually bring forth something better than erasure, or otherwise shut the "fix generics" crowd for good.

This has been discussed a couple of times. In no particular order:

• Java's Generics, behind the scenes by Alan Eliasen
• A much misunderstood issue by Neal Gafter
• Feedback by javac developer ;) by Martin Bravenboer
• Why Type Erasure is Wrong by Sam Tobin-Hochstadt
• Generics: The Importance of Wildcards by Martin Bravenboer
• Generics Considered Harmful by Ken Arnold

I believe Anders Hejlsberg that he thought this way. But if you look at the development of C# in the last years, its obvious that he changed his mind.

Is it really? I'm using C# 2.0 for work about 70% of the time, and although I'd often rather use ML/F# or Python, that's definitely not because of C#'s "bloated". In fact, I think C#'s design is still quite clean and new features, like anonymous delegates or nullable types definitely fit into the existing design nicely. If you took away a significant feature (like generics, or delegates) you'd know somethings's missing in day-to-day work.
In contrast, when I look at C++/CLI, I see the mother of all melting-pot-languages: You could strip away more than half of its features and still have a language that has richer semantics than C#.

The additional problem Java has is that it has no independent unit of behavioral abstraction smaller than the class. The only other major OO language that made that choice, afaik, is the original version of C#, which simply copied Java's design in this respect. I think hindsight shows that it's not such a great choice.

I think that the reason why classes are the smallest unit of abstraction (at the VM level at least) is so that you can have the all important class-loader. Its the class-loader that makes Applets, JSP's, network code transfer over RMI, Jini, and a lot of the current AOP frameworks possible. I don't really see the problem with representing closures or anonymous inner-classes (or enum's) as "classes" for the sake of generality and simplicity.

The additional problem Java has is that it has no independent unit of behavioral abstraction smaller than the class. The only other major OO language that made that choice, afaik, is the original version of C#, which simply copied Java's design in this respect. I think hindsight shows that it's not such a great choice.

Just a sidenote: C# 1.0 already supported delegates, and they were used throughout the framework form the start. Only the "anonymous delegate" feature, which allows you to to declare a delegate "closure-style" came in C# 2.0. Before that, you could only create delegates that pointed to methods.
[Edited a typo]

This (type-inference or dynamic typing) is the real reason for the conciseness of the languages you mentioned.

Agreed.

Consider an inner-class in Java:

new Functor(){public Object f(Object x){ ... }}

versus a lambda statement is Scheme:

(lambda (x) ...)

But the difference is almost all related to typing. If you removed the type information from Java then you would get:

new (){f(x){...}}

which is only one character longer than in Scheme.
Conversely, if you added type information (somehow) to Scheme, then you would get something about the same size as Java:

Functor (public Object lambda (Object x) ...)


It is interesting how a difference of only a few characters can have such a dramatic affect on the programming culture and conventions of a language.

it's just called 'anonymous inner classes'.

And it has an awfully verbose syntax, making it unusable for heavy usage of higher order functions.

Imagine expressing a monadic parser combinator library in Java, or CPS, where there is approximately one anonymous function per "statement" (one less than the number of statements in a sequence actually). For each such statement Java requires about two extra lines and three levels of nested }}), and a return keyword even in a simplest function body, and repetition of types (the result type must be written explicitly in the method signature and it's also implicit in choosing the appropriate interface).

Oops, CPS is unreachable anyway because of lack of tail call optimization. Poor Java.

Also, passing an already defined named method as an anomymous inner class requires to wrap it with code as heavy as for new functions (repeating all parameters along with their types). And they have a pointless restriction of being able to access only final variables.

Also, higher order functions are quite often polymorphic (parametrically). In Java 1.4 this looked even worse, see an example which combines anonymous inner classes with lack of genericity, yielding the classic sample of Java ugliness. Java 5 makes this better wrt. types, but still distinguishes between int and Integer, arrays are still monomorphic, there are still too many casts done at runtime, and the anonymous inner classes is still an ugly substitute of an anonymous local function.

Also inner classes are the better and more 'java-way' to do closures

I can't see what makes them better, and if 'java-way' means anything then it must be a synonym with 'ugly' or 'verbose' or 'inconvenient'.

You got it: Java is no functional programming language. To use it like one is verbose and frustrating and the result will probably be a slow and memory intensve program. But why would you? If one really want's those program features why not simply use a language like Haskell or Ocaml? Why do some people always try to turn every language into their favourite one?

Does anybody really expect that all the Java programmers suddenly start to use monadic combinators and similar stuff only because Java got a more concise syntax for closures? And why? Monads are used to create something like mutable state in a referentially transparent language - but Java isn't referentially transparent, so why even bother using monads?

As you pointed out: To get all those nice stuff you want Java needs much more than just a simply syntax for closures. So I think it's better to let Java stay the way it is and phase it out in the long term and use a real better language instead, because Java simply is beyond repair in too many points.

Something I do every day:

executor.execute( 
    new Runnable(){
        public void run(){
           System.out.println("Hello, world!");
        }
    }
);

will become


executor.execute{System.out.println("Hello, world!");};


Now I'm no fan of terseness for terseness sake, but this strikes me as a serious win in terms of clarity and readability.

It should also be noted that there is something in the proposal which can't be done with the anonymous classes: non-local control transfers. Whether this is a win or not I leave up for debate.

The "deep returns" are just syntactic sugar for exceptions, semantically speaking. In fact I believe I read that they will even be implemented via exceptions.

The problem with using Exceptions for flow-control is that they are pretty expensive because of the need to generate the stack trace. Even worse, if debugging is enabled (but not necessarily being used) then doing so will cause your method to be un-JIT'ed. This isn't normally a problem when your Exception is a rare exception, but when it becomes the rule, as in the case of using it for flow-control, it can bring your system to a crawl. The solution is to add the following method to Exceptions that you wish only to use for flow-control:

public Throwable fillInStackTrace()
{
return this;
}

This avoids the overhead of creating the stack trace and prevents your method from being un-JIT'ed.

They can be designed and implemented well.

The stack trace should not be materialized until it is needed. It should not be materialized at all if the exception is caught and handled by the program.

I don't know if there are Java features which would disallow that. I know that it's a practical design because I did this for my language Kogut and I'm happy with the result. Catching the exception dowesn't physically unwind the stack, finishing an exception handler does. The strack trace is materialized (by scanning the stack) if an exception falls of the main program, or on explicit demand.

I didn't mean that Exceptions were slow in general, but rather, specificially in Java with debug enabled and when used a lot (such as for flow-control). Java actually has two stacks: the real execution stack and a source-code stack so that in the event of an exception it can give you a nice stack-trace which actually relates to your source code. The second stack isn't actually used unless you create a stack dump or debug is enabled. They could/should probably fix their debug mode so that if a debugger isn't actually connected then it doesn't incur this overhead.

Yes, if that's all there is to closures then it isn't such a problem. But there is more to closures than that.

Also you can make the argument that for loops are "mere" syntactic sugar for goto statements.

Is writing

int i;
begin:
do something
i = i+1;
if(i

instead of
for(int i=0;i do something
}

really such a problem? I can't believe this.
Obviously there isn't much of a problem in this simple example,but
when your program grows in complexity, the simple syntactic sugar of for loops saves you from the (now little known) horror of spaghetti code .

Sure, there could be better support for using closures. But with IDEA it's already really simple:

Createa closure declaration and something which uses the closure (I used '[]' instead of '<>'):

    interface MapFunc[R, T] {
      R eval(T val);
    }
    static[R, T] List[R] map(Collection[T] list, MapFunc[R, T] f) {
        List[R] res = new ArrayList[R](list.size());
        for(T e : list) res.add(f.eval(e));
        return res;
    }

And now at the callsite you can simply type:

List res = map(data, new 

and hit alt+space and return. And IDEA creates

        List[String] res = map(val, new MapFunc[String, Integer]()
        {
            public String eval(Integer val) {
                // todo: write implementation 
                return null;
            }
        })

You only have to fill out the body. Sure, the resulting code looks a bit ugly, but it doesn't take more time then using a closure in a language like this Ocaml. Also refactorings etc works like expected.

To create complete closure support in Java there must be lots of changes in the standard libs and you would need additional language features (for example tuples). And afterwards you would have lots of redundant code and always to choose between iterators and maps etc. I simply don't see the point in that. Closures are really nice, but you need comprehensive usage in all tools, libs etc. This simply isn't possible for Java anymore because of the big existing codebase.

And thus making it readable is more important than making it writable.

Personally, I like the verbosity of Java! I think people who like terse languages are aiming for exactly the opposite: writability over readability.

If you sit down to work with a codebase in a dynamic or type-inferred language, the first thing you need to do is spend a long time figuring out the types of...well, everything. And if someone's passing around a closure, you have to figure out where the code for that closure is really located.

Here are some things I like about the explicit MapFunc class:

- I don't have to reconstruct type information that was in mind of whoever wrote the code. I can see every aspect of the type in front of me, and I can quickly navigate to the definitions or documentation of String, Integer, map, or MapFunc within my IDE. If I've never heard of any of those types before, no problem. If I'm familiar with them, I can immediately see how they're being used.

- A good Java IDE can enumerate the list of all implementations of MapFunc on the system. This can be helpful either to see how a MapFunc should be implemented or during a large-scale refactoring.

- We all agree that abstract data types (such as classes) are important for their ability to flex and extend as the codebase evolves. You wouldn't use the "int" type throughout a large program to represent Money, or an array reference type to represent a priority queue. You would define a specific money type and a priority queue type. Then why is it such a good idea to use a raw function type whenever the interface to some functionality currently resembles a single function?

To some degree I'm playing devil's advocate here. What I really want is to have a choice depending on what coding I'm interacting with and what I'm doing. I want to be able to configure in my IDE what level of type information is displayed inline with the code so that I can be looking at something terse or something with rich typing information depending on my needs at the time.

An IDE for a type-inferenced language could do exactly what you're asking for, just by filling in annotations (as "phantom" text, though having the ability to make it actual text would be useful too) containing the relevant types.

[Copied from a posting on the IntelliJ IDEA user forums, where they are also talking closures]

The way I see it, there's good redundancy and bad redundancy, and confusing the two is a key error in programming language design.

There's nothing wrong with redundancy or verbosity if it actually provides context, allows cross-checking, or acts as documentation. We know from information theory that redundancy is necessary for automated error detection, and I'm all about automating error detection. Having interfaces in Java, for instance, is unnecessary redundancy. You could program just fine without them, and some people do. What interfaces do is provide context, documentation, and cross-checking. The dynamic language people would argue that having any explicit types at all is unnecessary redundancy, and they are right, but in practice explicit types can be an amazing tool for enhancing program readability through context, cross-checking, and documentation.

On the other hand, there are a lot of sorts of redundancy that don't add anything to comprehensibility. For instance, all the various helper classes that each EJB 2.0 business class required didn't add anything to readability. They just repeated information that was in the business class, without adding anything. It didn't take very long for their creation to be automated out of existence with XDoclet and similar tools. That's not useful redundancy, that's just noise, and doesn't do anything to provide context, cross-checking, or documentation.

Simulating closures with anonymous inner classes requires a lot of boilerplate, which would be fine if that boilerplate actually added anything to program understandability. It doesn't. There's really no way that

executor.execute(new Runnable(){
public void run(){
System.out.println("Hello, world!");
}
};

provides any more insight into design intent than

executor.execute {System.out.println("Hello, world!"); };

In either case, what the developer really wanted was a closure, even if he didn't know the term. Making him simulate the closure with an anonymous inner is just busy work, and doesn't make anyone's job easier. Since the vast bulk of anonymous inner use fits that pattern, eliminating the busywork will be a big win.

That said, I do appreciate your points about class flexion and class searchability. IDEs will have to grow to accomodate those. I'll add "Migrate Closure Argument to Class" to my list of refactorings to have added to IDEA, and give some serious thought to how one searches for closures.

All I would add here is that Java is a bad example with which to make this perfectly valid point, because so much of Java's verbosity really does seem to come down to being boilerplate, and because its type system is sufficiently weak that the required annotations often just cause eye-rolling rather than information (at least for me).

I see that Phillipa has responded in kind regarding IDEs for type-inferred languages. To this I would just add by way of reiteration that O'Caml and its EMACS modes almost get there: if you compile your code with the -dtypes flag, O'Caml mode and Tuareg mode will both let you ascertain the type of the expression at the point. The downside, obviously, is that you have to compile your code. But when all else fails, you can still use the toploop to experiment with expressions and their types, and this is dirt simple to do with these modes.

Quite. Readability is a virtue; verbosity isn't.

Ada uses verbosity to enhance readability, Java as far as I can tell doesn't. Both languages are rather more low-level than the functional language we often discuss, so perhaps comparing them makes more sense.

I always liked Eiffel for its efforts to put readability first, and certainly it is more verbose than some languages, but the results in terms of clarity seem worth it. Most additions to the language seem very carefully thought out in terms of readability. Interestingly closures in Eiffel are one of the exceptions to that - the agent syntax is surprisingly clunky (particularly when it comes to calling syntax) all things considered. Still, it's better than what Java currently offers.

F# and the F# extensions for Visual Studio also offer a rather nice example of this. By hovering the mouse over expressions you get the type of that expression in a little "tooltip." But you can still get the benifits of type inferrence.

In a Boo project in the SharpDevelop you type:


l = [1,2,3,4] // creates a list
a = (1,2,3,4) // creates an array


Hover your cursor over the l and you'll get a little popup saying "local variable l as Boo.Lang.List". Do the same for the a and you get "local variable a as (int)". (int) means array of ints

Note, you don't have to compile the code to get the popup. It has an incremental parser working for you.

If you sit down to work with a codebase in a dynamic or type-inferred language, the first thing you need to do is spend a long time figuring out the types of...well, everything.

Why won't a good IDE do this for you?

I am not into dynamically typed languages, but I don't have the impression that even they spend that much time figuring out types.

And if someone's passing around a closure, you have to figure out where the code for that closure is really located.

How is this any different? You need to find the caller anyway, and there you find the closure, whatever form it is in.

You would define a specific money type and a priority queue type. Then why is it such a good idea to use a raw function type whenever the interface to some functionality currently resembles a single function?

It depends on the consumer of the closure. For things like sorting lists, which everybody needs, the lowest common denominator is a binary predicate. Not having concise syntax for this is a burden on everybody. For other things where the interface may naturally become more complex, a closure may not be the answer.

Why won't a good IDE do this for you?

How many dynamically typed or type-inferred languages have a good IDE? Zero? One?

How many dynamically typed or type-inferred languages have a good IDE? Zero? One?

For the most part, type-inferenced languages can have all the goodies that Java enjoys. Dynamically-typed languages are another story.

Wouldn't that depend quite a lot on how we define "a good IDE"?

Wouldn't that depend quite a lot on how we define "a good IDE"?

I never brought up the "goodness" of an IDE, but made a general statement about IDE support for dynamic and type-inferenced languages. The "goodies" I referred to are the IDE features that you would find in Eclipse, IDEA, etc..

Did you mean to respond to the person I responded to?

"The "goodies" I referred to are the IDE features that you would find in Eclipse, IDEA, etc.."

And you've still failed to say what specific features you are talking about, so no one can look at IDE X and say it does or does not have those features.

A reasonable cut what constitutes a good IDE nowadays would have to include completion, navigation, formatting, syntax-highlighting, run/debug, inline error-checking, automated refactorings at the small-to-medium scale, build integration, test integration, VCS integration, app-server deployment, and possibly some variety of GUI generation tooling (either web or desktop).

WRT the statement that "Good IDEs don't exist for dynamic languages", I'd say that that is a lot less true now than it was a year ago. Eclipse/IDEA extensions for JavaScript and Python are getting very close to satisfying the reqs above.

And if you say which specific languages you are talking about we can avoid the usual Smalltalk and Lisp IDEs did that years ago commentary.

I don't know when all of those things became available in high-end Smalltalk and Lisp systems, I have some idea when I used them in low-end Smalltalk systems:

• code completion is relatively recent in Smalltalk IDEs, here's a demonstration
• navigation across classes and methods, implementors/senders - vanilla Smalltalk-80, so most Smalltalk implementations (I used in ~1989)
• formatting - vanilla ObjectWorks/Smalltalk (I used in ~1993)
• syntax-highlighting - Smalltalk doesn't have much syntax to highlight, but I seem to remember comments being shown in italic before color displays were widely available (I used ~1989)
• run/debug - vanilla Smalltalk-80, so most Smalltalk implementations (I used in ~1989)
• inline error-checking - in most Smalltalk IDEs we don't edit a whole file of stuff, we edit a single method and save that changed method definition or we edit a single class definition and save that class definition. The definitions are compiled and error-checked when they are saved. That isn't the same as error-checking while you type-in code, and it isn't the same as error-checking a whole file of stuff - it's somewhere in between. (I used in ~1989)
• automated refactorings - the Smalltalk Refactoring browser and Rewrite Rules (pdf) became available in the late '90s.
• build integration - there's a whole range of deployment options, some map to a build process and some don't.
• test integration - Kent Beck was writing about SUnit in 1994, and as most Smalltalk IDEs are written in Smalltalk (and the source code is provided), unit test was integrated into IDEs by development groups long before the vendors bothered.
• VCS integration - tightly integrated, fine grained, version control systems were available for most Smalltalks by '91 (I used in ~1991)
• app-server deployment - not sure what that would mean outside the J2EE toolchain
• GUI generation tooling - iirc the first Smalltalk programs I wrote explicitly built the GUI (I used drag&drop GUI builders in ~1991)

Which is just a long winded way of saying Good IDEs existed for dynamic languages years ago.

I'm not sure how much sense it makes to talk about "a Good IDE", we can talk about a good Java IDE or a good Smalltalk IDE or ... We want the IDE to support our working practices, and I think working well in a different language means working differently.

Among some things I always found interesting about Smalltalk was (1) the ability to instantaneously browse object instances of any class that I wanted - throw them all in a collection and trundle through them at will; and (2) the image serves as a lightweight persistence mechanism - like capturing a snapshot of the run time. Or as one might say, the image is a place where objects live and breed.

I suppose one can mimic these capabilities - like Visual Age for Java which is an attempt to bring IBMs Smalltalk philosophy to Java. And that's the problem with PL wars - there's too dang much fraternization taking place on the front lines of the battle - being a real distraction to those who just like to concentrate on advocacy. :-)

Thanks to LtU I'm a little more knowledgeable than I used to be, so I hesitate to say Smalltalk invented something that afaik might previously have existed in some Lisp implementation :-)

1) In Smalltalk, that piece of the IDE is called an Inspector - there's a context-sensitive menu-item Inspect it, which opens an Inspector on the currently selected text. Or we can send the message inspect

Array allInstances inspect

There seem to be 59,210 Array objects in the image I just looked at... the 401st Array object has 2 elements (both instances of class Symbol) #(#Graphics #Rectangle)

2) There's no distinction between development-time and run-time, development-time is just run-time when the user is a developer. (Yes, sometimes a distinction is created by stripping out the development tools.)

The Smalltalk image is a snapshot of the objects that currently exist. Smalltalk compiled code is represented by Smalltalk objects, so saving the image saves whatever methods were written and classes edited since invoking the image. And all the windows are Smalltalk objects, so saving the images saves the GUI state. And all the Processes are Smalltalk objects, so saving the image freezes Processes (and unfreezes them when the image is invoked next time). ...

Most of the VisualAge IDEs (including VisualAge for Java) were written in IBM Smalltalk. Later VisualAge Micro Edition was implemented in Java (and thereafter begat Eclipse).

Quote from Isaac: "Karsten, you seem to be quarreling for quarreling's sake."

Isaac, most of us know what the "goodies" in Eclipse and IDEA are. So maybe you should follow your own advice, because you have a bad habit of doing exactly what you didn't like Karsten doing.

most of us know
That's just an assumption

Isaac, your standard template of asking vague questions in order to make some unknown point is troll-like at best. You do it here and other places.

My response was not to you, but someone else that is well aware of the feature sets of the top Java IDEs. And I know you are well aware of these features too. If you have a point, make your point, and stop the trolling.

You made "a general statement about IDE support for dynamic and type-inferenced languages". Was it true for commercial Lisp IDEs? I don't know I haven't use any, maybe you have. Was it true for commercial Smalltalk IDEs? How much of Dave Griffith's list of "what constitutes a good IDE nowadays" has long been available in Lisp and Smalltalk IDEs?

Maybe you were just talking about Python and Ruby, and maybe there are interesting questions about the kind of things Python and Ruby have been used for that would allow productive work with fairly simple editors.

"...the feature sets of the top Java IDEs. And I know you are well aware of these features too."
I've never used intellij and I haven't used eclipse for the last 18 months. Even if I currently used both of those IDEs that still wouldn't be enough to tell me or anyone else which IDE features you thought were important.

How many dynamically typed or type-inferred languages have a good IDE? Zero? One?

For the most part, type-inferenced languages can have all the goodies that Java enjoys. Dynamically-typed languages are another story.

Since type-inferenced languages are statically typed (at least the definition I'm going by), things like code completion and little hover popups on variables are as doable as in a manifestly typed languages like Java.

An IDE for a dynamically typed language won't be able to always get the code completion or the type for a variable because of the nature of dynamic typing (How can you know what a method returns for example?)

That is the answer to his question. It wasn't a response to some unknown Isaac Gouy question.

Since type-inferenced languages are statically typed (at least the definition I'm going by), things like code completion and little hover popups on variables are as doable as in a manifestly typed languages like Java.

Almost. The thing about type-inferenced languages is that the stuff you mention works fine if the type-inferencing algorithm can run to completion. If it can't (because the program is ill-typed), the IDE will have a lot less context information to help fight it's way out with. That means that in-line error messages are likely to be a lot more cryptic for ill-typed programs in type-inferenced languages, and functionality like completion and navigation is likely to fail a lot more often. Now you can start using heuristics and approximations to help get you around such issues, but then that's exactly the sort of thing you have to do with a dynamically-typed language.

It may seem like a small thing, but if your program only works well when people do something perfectly, then your program simply doesn't work well.

You can likely improve the in-line error messages significantly by tracing back to where type checking fails and including a link with the error message. This doesn't approximate or use a heuristic, it just tells you what to fix to get more info.

I don't mean to be inflammatory, but if someone tried to suggest workflow like that in an IDE I was building, it would almost certainly be red-lit for being confusing, producing too many error messages, and being aggressively unhelpful. Better to punt to dynamic-typing-ish heuristics than shower the user with "there's a problem over there!" links. This is particularly true since 'where type-checking fails' can be far distant from where the user error occurred.

Explicit typing can be wordy, but it has the advantage of strictly containing the scope of type errors, thus allowing better error-detection and -correction when they (inevitably) occur.

If I'm going to use type inference, I want those links - right now without an IDE, the equivalent pattern is to start nailing type errors from the top of a file downwards and hope my dependencies weren't too circular. Certainly I want it in the case "you use identifier foo here, but it's bound to something I can't type so I can't type the rest of this" - and preferably when there's an entire binding group that's unresolved it should show me the binding group. Possibly links only via a tooltip or a separate tab/list or similar, but definitely some sort of dependency graph available if I want it. IIRC there's a known algorithm for doing this for Hindley-Milner, I don't know how hard it'd be to make it work with common extensions.

The heuristics would irritate me no end by forcing me to calculate exactly the info just mentioned above.

I guess the short version is that I'm being asked to be shown how the typechecker drew its conclusions, and to have useful links into the typing so I can navigate it at will.

It may seem like a small thing, but if your program only works well when people do something perfectly, then your program simply doesn't work well.

Of course that all depends on the strength of an IDE parser. I know that the programmers of the Boo and Nemerle compilers have exposed and implemented certain functionality specifically for code completion and will have a certain tolerance for errors.

Now I've never seen something like the eclipse compiler that gives you syntax problems on the fly for a type inferenced language, so that might be another whole problem.

The type inferenced languages I'm used to deliberately make sure syntax is independant of typing - though I have to admit to seeing using a declared identifier with no known/correct type as being a similar class of error to using an identifier with no binding in scope. But it shouldn't produce any new problems at a syntactic level. The existance of binding groups should help get as much information out of type inference as possible - frequently much of a module's typeable even when the module as a whole isn't.

Epigram's sheds seem a nice idea, I wonder how hard it'd be to map something similar onto source in a language like ML or Haskell?

An IDE for a dynamically typed language won't be able to always...
That's right!
Function and method signatures in statically checked languages have more information so IDEs can discriminate more sharply. That's noticeable for code completion and method rename refactoring.

Actually using the code completion and method rename refactoring provided by Smalltalk IDEs shows how useful they are in practice. Refactoring was integrated into Smalltalk IDEs back in the mid-90's. It's great that the capabilities of dynamic language IDEs finally became available in popular IDEs.

An IDE for a dynamically typed language won't be able to always...
That's right!
Function and method signatures in statically checked languages have more information so IDEs can discriminate more sharply. That's noticeable for code completion and method rename refactoring.

Yes....and that's why I made the distinction to the original parent of the subthread.

Actually using the code completion and method rename refactoring provided by Smalltalk IDEs shows how useful they are in practice. Refactoring was integrated into Smalltalk IDEs back in the mid-90's. It's great that the capabilities of dynamic language IDEs finally became available in popular IDEs.

And the Smalltalk history that I'm already aware of is relevant how? Oh, it's not at all, and irrelevant to the distinction made between completion and type info for dynamic and statically typed languages.

"For the most part, type-inferenced languages can have all the goodies that Java enjoys. Dynamically-typed languages are another story."
For the most part dynamically-typed languages can have all the goodies that Java enjoys.

Your notion of "the goodies that Java enjoys" seems to have narrowed from "the IDE features that you would find in Eclipse, IDEA, etc.." to what MS brand as Intellisense.

Dave Griffith seems to have a broader notion of what makes "a good IDE". Define "the goodies that Java enjoys" how you like but remember that its your definition and others may have different definitions.

Your notion of "the goodies that Java enjoys" seems to have narrowed from "the IDE features that you would find in Eclipse, IDEA, etc.." to what MS brand as Intellisense.

No it hasn't

Dave Griffith seems to have a broader notion of what makes "a good IDE". Define "the goodies that Java enjoys" how you like but remember that its your definition and others may have different definitions.

I never made a definition. David Griffith already made a defintion that would be widely accepted and would be the inference made by the parent poster.

But you really can't help yourself from trolling can you Isaac? It's here, it's over at the Cincom Blog where you're a constant nusiance troll, it's over at Artima. It's basically everywhere. What exactly is your problem?

Quote from Isaac: "Karsten, you seem to be quarreling for quarreling's sake."
Once again Isaac, you need to follow your own advice.

"For the most part, type-inferenced languages can have all the goodies that Java enjoys. Dynamically-typed languages are another story."
For the most part dynamically-typed languages can have all the goodies that Java enjoys.

For the most part dynamically-typed languages can have all the goodies that Java enjoys.

For the most part, but not the whole part, and it is another story for dynamic languages when "the story" is pretty much the same for manifest typing and type inferencing. There's another "story" for refactoring in dynamically typed languages too. I believe Dave Griffith (who has done some work on IDEA plugins) has touched on these issues in a past post.

There's another "story" for refactoring in dynamically typed languages too.
I've already stated that, in general, dynamically typed languages do not allow automatic, safe, rename method refactoring, there isn't enough information to distinguish call-sites of polymorphic methods. That is one specific refactoring.

If there isn't a "Loss-less round trip Ruby parser" that's a specific issue for Ruby, it isn't an issue for dynamically typed languages - it's nearly a decade since the creation of a Smalltalk parser for refactoring.

If there isn't "Runtime call graph analysis" for Ruby that's a specific issue for Ruby - runtime type recovery has been available in Smalltalk for years The Analysis Browser.

If there isn't research into concrete type inference for Ruby that's a specific issue for Ruby - the research has moved on to developing algorithms that are effective for >100,000 LOC Demand-Driven Type Inference With Subgoal Pruning (pdf).

I've already stated that, in general, dynamically typed languages do not allow automatic, safe, rename method refactoring, there isn't enough information to distinguish call-sites of polymorphic methods. That is one specific refactoring.

That's where the other story comes from. The static model and the dynamic model have different requirements for the refactoring. Completion can't ever be totally known because type inference won't help with you all cases. Dolphin's completion is a case in point. But if it was anybody else, I would have guessed that it was just a misunderstanding on what "another story" means, but given your reputation for trolling, it was most likely not the case.

If there isn't a "Loss-less round trip Ruby parser" that's a specific issue for Ruby, it isn't an issue for dynamically typed languages - it's nearly a decade since the creation of a Smalltalk parser for refactoring......

I'm well aware of the Smalltalk environment capabilities.

A reasoned (and perhaps impassioned) argument about the relative merits of a particular PL, or class of PLs, is appreciated. However, this somewhat passive LtU participant does not like the introduction of personality conflict into what would otherwise be an informative exchange.

It's unfortunate that Isaac would rather play his little question troll game here and other places, instead of having an informative discussion. If it was anybody else, I would pass it off as a misunderstanding, but Isaac does this on other forums as well. And it's also unfortuante that someone has to point it out it to him, and further distract from the conversation, but he apparently doesn't get it.

Please keep these baseless personal insults off LtU.

The above should be modified to...

Please keep ALL personal insults off LtU.

The personal insults have no place on LtU.

His response to me was a troll in a class fashion. It's his standard template of asking a vague question in order to bog down the discussion, instead of making his point - which he finally made later down the thread (days later) with his Smalltalk IDE feature list.

Isaac, if you felt it was a sleight on Smalltalk, then you should have just made your Smalltalk IDE features list post as a response to me. By the way Isaac, you jumped the gun. I wasn't dissing Smalltalk at all. In fact, I have Dolphin and something called Vista Smallscript in front of me right now.

So Isaac, if you want to make a point or you feel your favorite language/(language-type) has been dissed, just make your direct point and don't bog down the discussion with the senseless troll question that you are well aware of that you do.

I guess I could have played the endless vague question game with Isaac, but then I would have been participating in his game.

Let's end this exchange, please. It is good to bring things out to the open, but continuing this thread of personal accusations is unhelpful.

Thanks.

What's your point? It's clear that such languages can have good IDEs - see e.g. DrScheme, or the various Smalltalk IDEs (from which Eclipse was derived!) The fact that such languages don't have good commercial-grade IDEs at the moment seems to have mainly to do with commercial backing, more than any other factor, and that may very well change. So what conclusion should we draw here?

I was just speaking in practical, here-and-now terms in response to the question:

"Why won't a good IDE do this for you?"

A good IDE won't do that for me because in the situation under discussion, I probably wouldn't have a good IDE.

I certainly don't dispute that these languages can have a good IDE.

David Weitzman's complaint was actually about what would happen to a Java programmer if Java got type inferencing. Since the feature is not in Java yet, there is no IDE that provides any support, so I don't understand how in practical, here-and-now terms this question is relevant because the feature doesn't exist yet. However, if the feature does make it in, I bet many Java IDEs will provide support for it very quickly.

How many dynamically typed or type-inferred languages have a good IDE? Zero? One?

I don't think that's a problem with the languages (at least not for type-inferred languages), more with their populatrity. F# (an OCaml dialect) has an excellent Visual Studio integration, with intellisense and tooltips showing you the types of everything. Definitely better than Visual Studio's support for C++. Something similar has been done for Haskell (google for "Visual Haskell"), but I think the project isn't continued.

...are there any real use cases left for the anonymous inner syntax?

In the project I'm working right now we use this:

public Collection someQuery(final int x, final String y) {
    return new CollectQuery() {
         public String getSql() {
              return ...;
         }
         public void setParameters(PreparedStatement stmt) {
              stmt.setInt(1, x);
              stmt.setString(2, x);
         }
         public void forEach(ResultSet row, Collection collected) {
              collected.add(new Foo(row.getInt("id"), row.getString("name")));
         }
    }.execute(this.connection);
}

Someone could argue that we should use a ORM tool but this usage (i.e. anonymous inner classes to implement a strategy/template method hibryd) is really nice to use (modulo no type inference and excessive syntatic overhead). Also "setParameters" is optional and in other places we have additional hook methods for other templates.

If I had to change this to using just functions I would become this:

public Collection someQuery(final int x, final String y) {
    return collectQuery("sql", void (PreparedStatement stmt) {
              stmt.setInt(1, x);
              stmt.setString(2, x);
    }, void (ResultSet row, Collection collected) {
              collected.add(new Foo(row.getInt("id"), row.getString("name")));
         }, this.connection);
}

Which is harder to read and we lose descriptive names because Java doesn't have named parameters.

Java already has methods. The method is a form of abstraction that is distinct from classes. Function pointers would make this existing abstraction a first-class entity, possibly even improving the orthogonality of the language.

Adding support for structural subtyping, however, would add significant complexity if you want anything deeper than simple syntactic sugar. Maybe the complexity is worth it; I don't know yet. But it's not fair to rule out a feature based on the claim that "two distinct forms of abstraction is worse" (which seems to me a misapplication of the generally accepted "more complexity is worse" rule-of-thumb).

Methods are part of the abstraction provided by classes, neither of them stand alone. You don't have to decide between a method or a class. When you add function types you have to decide if you will use a function or a class, and with classes we can have more than one operation:

public void foo(void(void) run, void(void) undo) {...}

foo({/*do stuff here*/}, {/*undo stuff here*/});

vs.

public void foo(Command cmd) {...}

foo(new Command() {
    public void run() {
        /*do stuff here*/
    }
    public void undo() {
        /*undo stuff here*/
    }
});

Both operations (i.e. run and undo) are related so it's natural (in OO languages) to place them in the same class. With function types we have to separate them and explain that the two parameters are related in the documentation. In a classic discussion the issue is clearer: we can have an unfold with three function parameters or an unfold receiving a record of three functions and, as long as the functions are related, I believe they should be packed together.

In PLT simplicity and orthogonality are values that shouldn't be understimated, specially when you have a (already) complex and large language as Java.

Java is so syntactically verbose that even 4th year CS students can successfully invent a shorter syntax for it.

Could we see more?

So far I've known that only Smalltalk has such a semenatics of return (^) from a block. And of course, C preprocessor macros allow something similar. Any other language that has that kind of break/continue/return in closures?

Any language with first-class continuations can do this, e.g. Scheme, SML/NJ, Stackless Python, and the Javascript implementation which supports continuations. You could use one of these new Java closures containing such a return to fake a first-class continuation. Actually, it's not even faking exactly, but it is a sort of backwards way to do it, essentially allowing Java's built-in use of continuations in return/break/continue to be abstracted via closures.

The proposed captured control operations for Java have dynamic extent, rather than an unlimited lifetime You can't call a function that does a break after you leave the loop, so you can't use it to implement call/cc.

Honestly, given that Java already has exceptions, I don't see the point of this feature.

Yeah, when I posted my response, I hadn't noticed the sentence which Vesa quoted.

Honestly, given that Java already has exceptions, I don't see the point of this feature.

I assume they were just trying to give a reasonable semantics to any Java operation you could perform inside a closure, and erred on the side of not restricting things. I agree it seems questionable.

Well, the proposal says:

If a break statement is executed that would transfer control out of a statement that is no longer executing, or is executing in another thread, the VM throws a new unchecked exception, UnmatchedNonlocalTransfer.

Which, I assume, means that only exit continuations are supported. Those can be simulated in other languages with exceptions. Here is an example in SML:

fun withReturn f =
    let
       exception Return of 'a
    in
       f (fn v => raise Return v) handle Return v => v
    end

withReturn (fn return => ...) passes a return procedure to the given function. Calling the return procedure from the function causes the call to withReturn to return (immediately). withReturn has the specification:

val withReturn : (('a -> 'b) -> 'a) -> 'a

Here is an example using withReturn:

val SOME 6 =
    withReturn
       (fn return =>
           (app (fn x =>
                    if 0 = x mod 2 then
                       return (SOME x)
                    else
                       print (Int.toString x))
                [1, 3, 5, 6, 7, 9]
          ; NONE))

The effect of the above code is to print "135".

If the return procedure is called outside of the execution of the withReturn call, then the program (or thread) is terminated with a note on an unhandled Return exception.

Outside of a withReturn, only a wildcard exception handler may handle a Return exception raised by calling the return procedure.

Edit: Also see the following note.

In fact, if I were to design it, I would prohibit calling a closure after its enclosing function has ended its stack life.

Why? This restriction would make no sense to me.

Consider types. We don't really like overusing types locally (inferring types is better), but types ease the pain of specyfing and maintaning "contracts" between "modules" (let us call them APIs), often developed by different people.

You know that in Java 1.4 there is no way to specify a non-null reference type, and that is bad for APIs. That specification aspect ends up in API comments, but not when you needed (according to a Murphy law).

With the proposed closures, a runtime exception UnmatchedNonlocalTransfer may be thrown. But it happens only with closures containing break/continue/return (let's call them nlt closures), and when passed to a module that would store them for long-time use (let's call this a long-life closure consumer). So, there is an analogy between passing an nlt closure to a long-life consumer, and passing a (maybe-)null variable to a (probably-)not-expecting-null method - however, this could be harder to detect / workaround in existing code.

Therefore, a well-mannered API designer would end up commenting on each closure consumer, whether or not it expects/allows nlt closures. So, to promote the well-manneredness of APIs across the vast amount of libraries that will be written in JavaExt language, I'd promote the difference between short-life closures and long-life closures (a.k.a. anonymous inner classes with better syntax) to - at least - a verifiable annotation.

Well, that's the sort of point I had in mind when designing that restriction.

If a closure can't outlive the function which has created it, each closure must be syntactically written at the point it is passed down: you can't wrap creation of a closure in a function.

The fact that it outlives that function can't be detected statically, so the compiler would have to insert explicit "closure killing" code at the end of a function which creates closures, so from this time the closure starts throwing exceptions instead of working. In effect the code is not only strictly less general but slower.

This rule would complicate the language: it would be an explicit restriction which needs to be described and learned and worked around.

Dynamic extent closures that throw exceptions when called after they go out of scope are hard to implement unless you continue to allocate at least part of them on the heap, in which case you might as well have indefinite extent closures. Whereas dynamic extent non-local exits from closures are not.

That's because there's a fixed number of the latter but not of the former, so detecting when closed dynamic labels variables go out of scope is easy as long as you can force a return path that can update a hidden variable shared with all closures that close the given exit label.

Conversely, rendering all references to stack allocated memory that go out of scope invalid is difficult -- you can't do it when the enclosing block exits, and you can't do it where the stale references are accessed.

http://gcc.gnu.org/onlinedocs/gcc-4.1.1/gcc/index.html#toc_C-Extensions

Nested functions, for example, are closures allocated on the stack that start with a bit of trampoline code to set up the frame pointer and jump to the right location.

There's no way to make sure that stale references to such closures don't remain when the block that creates them returns -- this is C, you don't even have a GC.

For long-life call-backs (like gui event handlers), the Java approach of anonymuous inner classes work better and good enough.

"Good enough" I'll buy, although they seem to be overkill for the task. "Better"? I'm really not sure why that would be. In most cases anonymous classes are just used to (verbosely) simulate closures, so I don't really see how they could be "better" than closures.

So a closure could be used in functional-style-like map, apply, folds, etc. But it would be pointless to store a closure for long-term use.

This would only allow for eager combinators. Lazy combinators wouldn't work with this restriction. That quickly cuts the scalability of functional-style filter/map/reduce constructs, to no good purpose.

In #comment-20691, Marcin Stefaniak wrote:

So far I've known that only Smalltalk has such a semenatics of return (^) from a block. [...] Any other language that has that kind of break/continue/return in closures?

Since Ruby has borrowed so heavily from Smalltalk, I don't even know if it qualifies as a different language within the scope of your question, but here you go:

#!/usr/bin/ruby -w

def foo
  p = Proc.new do
    puts "enter"
    return "non local exit"
  end
  p.call()
  puts "exit"
end

puts "result: #{foo()}"

This produces:

$ /tmp/foo.rb 
enter
result: non local exit

I think a lot of languages with mature support for closures have something like this. With Scheme you have CALL-WITH-CURRENT-CONTINUATION. With Common Lisp you have the BLOCK and RETURN/RETURN-FROM special forms. And of course it can be implemented as syntactic sugar on top of an exception mechanism very easily (which is the road Java appears to be taking).

In comment-20737, Per Vognsen wrote:

I think a lot of languages with mature support for closures have something like this.

Depends on your measure of likeness. Unlike call/cc, exceptions, Lisp/Ruby-style catch/throw, the proposed feature has a very lexically delimited feel to it. Since you can't jump too far, you can reason about the call flow transfer locally. With truly non-local call flow transfer constructs, all sorts of interesting problems arise.

The proposed three types of non-local transfer -- return, break, and continue -- are almost like Tcl's return -code return, return -code break, and return -code continue. These work like so:

#!/usr/bin/tclsh

proc line {} {puts [string repeat - 40]}

proc foo1 {} {
    puts "enter foo1"
    bar1
    return "exit foo1"
}

proc bar1 {} {
    puts "enter bar1"
    return -code return "exit bar1"
}

puts [foo1]
line

proc foo2 {} {
    puts "enter foo2"
    foreach n {1 2 3 4 5} {
        puts "$n: [bar2 $n]"
    }
    puts "exit foo2"
}

proc bar2 {x} {
    if {$x == 4} { return -code break }
    return [expr $x * $x]
}

foo2
line

proc foo3 {} {
    puts "enter foo3"
    foreach n {1 2 3 4 5} {
        puts "$n: [bar3 $n]"
    }
    puts "exit foo3"
}

proc bar3 {x} {
    if {$x  4} { return -code continue }
    return [expr pow($x, 3)]
}

foo3

This produces:

enter foo1
enter bar1
exit bar1
----------------------------------------
enter foo2
1: 1
2: 4
3: 9
exit foo2
----------------------------------------
enter foo3
3: 27.0
4: 64.0
exit foo3

By the way, even though Ruby has callcc, raise/rescue, and catch/throw, it also provides the "lexical" non-local return construct as described in comment-20725.

As far as I understand it, the meaning of Tcl's return -code depends on how the caller's code is organized into functions. In order to jump to a particular place, you must know how many stack frames are between the source and the target of the jump. This is insane.

Statically scoped jumps like proposed for Java are fine.

Tcl has the same problem with emulating closures with uplevel and upvar: you must know the number of stack frames between variable definition and closure application. This is insane.

In comment-20859 Marcin 'Qrczak' Kowalczyk wrote:

It's not at all like in Tcl

Tcl's upvar and uplevel take some getting used to, I'll grant you that :) But calling it insane is just another way of saying that using these features takes some discipline. But what doesn't? Exceptions?

If you squint hard enough, you'll see that Tcl is actually lexically scoped :)

There are known cases of people exposed to Tcl in their tender formative years who went on to implement upvar in Python, thereby freaking out the Python establishment.

But I digress...

You can't deny the similarity between Java's proposed return, break, and continue and Tcl's return -code return/break/continue. These are short-range non-local transfer mechanisms. In Java, they are (going to be) lexically delimited. In Tcl, they only allow you to skip a single stack frame. The net result is very similar: you can't jump too far.

Plus ça change... (Obscure reference to a comment in Paul Graham's On Lisp about object orientation.)

IOW, what are we discussing that's new? Nothing.

Closures, continuations, non-local exits, of dynamic and/or indefinite extent, etcetera -- nothing there is new.

FWIW, I think Java should get closures and continuations of indefinite extent, but first some research would be needed to decide on how best to implement continuations. Stack copying will do, but heavy users of continuations won't like it. Stacklessness will do too, but without some optimizations for the common case where continuations are not accessed then it won't do for most users.