Lambda the Ultimate

My only problem with finally is that it is oblivious of the possible exception that was thrown and it may hide it by throwing it's own exceptions

That's why in the appropriate construct in my language Kogut an exception from the closing function is ignored if the body has failed.

Ignoring an exception is generally bad, but we have to ignore one or the other anyway. I felt that aborting the program (like in C++) would be too drastic, and replacing the exception with some “compound exception” would break code which expects to see the exception from the body.

...replacing the exception with some “compound exception” would break code which expects to see the exception from the body.

In Java 1.4+ every exception* can have a cause (i.e. another exception*) so you can chain exceptions. You just need to extend this concept to another field, let's call it hidden, so we don't lose any of the exceptions.

Now we can define that exceptions are extensible variants of records and every exception has at least two fields (i.e. cause and hidden) of type List Exc. If finally throws an exception (fexc) and there's already an exception thrown by the body (bexc) we just reconstruct bexc and append fexc to its hidden field.

OTOH if the finally clause can fail the caller shouldn't be suprised if it was the other way around (i.e. fexc hiding bexc) because even if the body succeeds the finally may fail anyway, so its exception should be known by the caller.

_{*Actually it's for any type of throwable, but the reasoning is the same.}

If finally throws an exception (fexc) and there's already an exception thrown by the body (bexc) we just reconstruct bexc and append fexc to its hidden field.

I'd just construct fexc with bexc as the value of the "hidden" field (which no longer needs to be appendable). No need to "reconstruct" bexc and the causality relation would then actually make sense (bexc is raised first and then causes fexc).

I would like to attach an else clause to any control structure that may never be executed (e.g. for or while)

I've actually included that in a language I've been developing. It has an inline "iterative" conditional, similar to a C style inline conditional with the difference that it will iterate the "true" target while until the initial condition turns false, and it will only execute the "false" target if the initial condition started off false:

  count < 10 ?? ( print(count); count++ ) : print("Error, count was to large")

I was thinking I could do the same thing with this Squeak Smalltalk snippet:

[count < 10] ifFalse: [self error]; whileTrue: [self step. count := count + 1]

but in point of fact this produced the error "Cascading not expected" on the semicolon. I wonder why??

The best way I found was to add this as a new control construct by adding a new method to the BlockContext (i.e. closure) class:

ifFalse: falseBlock whileTrue: trueBlock
   ^self value ifFalse: [falseBlock value]
               ifTrue:  [trueBlock value. self whileTrue: trueBlock]

Then I can send the ifFalse:whileTrue: message to a closure with two other closures for arguments and it will do what I want:

[count < 10] ifFalse: [self error] whileTrue: [self step. count := count + 1]

which reads as "my dear closure [count < 10], if your execution returns false then please ask [self error] to execute, otherwise continue asking [self step. count := count+1] to execute for as long as you return true."
Smalltalk rocketh! See The Early History of Smalltalk for background on this kooky idea of writing your own methods on classes like Block, True, and False.

I wonder how best to write this as a Lisp macro?

but in point of fact this produced the error "Cascading not expected" on the semicolon. I wonder why??

It is most likely because ifTrue:ifFalse are heavily inlined in squeak. In theory, a boolean expression should cause a boolean object to get instantiated, but in practice I believe it is inlined directly to the statement that will run (so it ends up being equivalent to if/else in C). And since no object actually gets instantiated, you can't send cascade messages to it.

This was my guess too but it's disappointing. I'm all for the inline'able case being inlined but it's bad to disallow non-inlinable cases in the process.

Self has the same implementation of ifThen:Else: but doesn't compromise on it and (unsurprisingly) does reduce to a C-like if-then-else in the standard case without any magic.

The recent bump in this thread led me to ponder this:

I wonder how best to write this as a Lisp macro?

How do you define "best"? A straightforward solution in Scheme might look like:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (if cond
         (let loop ()
           (if cond
               (begin body ... (loop))))
         (begin alt ...)))))

(let ((count 0))
  (while (< count 10) 
    (display count)(newline)
    (inc! count)
   (else (error "blurk!"))))

But in the above version, the condition is evaluated in two different places. At first I thought to myself, hey, maybe I can complicate this with some delimited continuations (FP has a reputation to uphold!) along the lines of this Oleg post. But to my great disappointment, a mild dash of CPS does the trick:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (let loop ((exit (lambda () alt ...)))
       (if cond
           (let ((result (begin body ...)))
             (loop (lambda () result)))
           (exit))))))

Only one test, woohoo. :)

But if you really want the best, here it is. Naturally, "best" == "uses call/cc":

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (let-values (((loop exit) (call/cc (lambda (k) (values k (lambda () alt ...))))))
       (if cond
           (let ((result (begin body ...)))
             (loop loop (lambda () result)))
           (exit))))))

But in the above version, the condition is evaluated in two different places. At first I thought to myself, hey, maybe I can complicate this with some delimited continuations (FP has a reputation to uphold!) along the lines of this Oleg post. But to my great disappointment, a mild dash of CPS does the trick:

Ahh, but anything you can write in CPS can be straightforwardly written in direct style with shift and reset:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (reset (begin
              (let loop ()
                (shift exit
                  (if cond
                      (let ((result (begin body ...)))
                        (reset (begin (loop) result)))
                      (exit #f))))
              alt ...)))))

We've drifted pretty far from why-we-need-finally-in-try country, but I should demystify the delimited continuation version of Anton's second solution. You don't need to be Oleg to program with delimited continuations (though I'm sure it helps), and Luke likes to see code :)

Anton's original continuation-based solution, slightly modified (expanding named let into letrec, making continuations take an ignored argument, and returning an "undefined" value from the body to match the behavior of the first solution) is:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (letrec ((loop
                (lambda (exit)
                  (if cond
                      (begin body ...
                             (loop (lambda (ignore) (void))))
                      (exit 'ignore)))))
       (loop (lambda (ignore) alt ... (void)))))))

The function loop has "type" (A -> Void) -> Void, ie, it's a value in the continuation monad with answer type Void (which is as "undefined" as any other value). Thus, as per Filinski, we can write it in direct style using the monadic reflection operators reflect and reify for the continuation monad (with answer type Void) with types

reflect : ((A -> Void) -> Void) -> A
reify   : (Unit -> A) -> (A -> Void) -> Void

In short: loop will now have type Unit -> Void (it's a recursive value so we make it a thunk), where we previously explicitly constructed a value of type (A -> Void) -> Void we now use reflect, and where we previously explicitly applied a value of type (A -> Void) -> Void we now use reify:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (letrec ((loop
                (lambda ()
                  (reflect
                    (lambda (exit)
                      (if cond
                          (begin body ...
                                 ((reify loop) (lambda (ignore) (void))))
                          (exit 'ignore)))))))
       ((reify loop) (lambda (ignore) alt ... (void)))))))

The definitions of reflect and reify for the continuation monad are:

(define (reflect f)
  (shift k (f k)))

(define (reify c)
  (lambda (k) (reset (k (c)))))

Inlining them and a fair bit of simplifying (beta_v and eta_v reduction, writing (let ((ignore E0)) E1) as (begin E0 E1) as per the original Lambda paper, lambda-dropping the definition of loop, and reintroducing named let) yields:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (reset (begin
              (let loop ()
                (shift exit
                  (if cond
                      (begin body ...
                             (reset (begin (loop) (void))))
                      (exit 'ignore))))
              alt ... (void))))))

How do you define "best"?

I was mostly thinking of the name. The Smalltalk ifFalse:whileTrue: selector tells the story pretty well. Your while-else is quite a bit better (and Schemier) than the ideas I had for a Lisp syntax. When I have more time I will see if I can actually understand the code. :-)

Code on LtU again, very good! :-)

The first implementation looks really nice, but for the repeated test that you pointed out.

The second is mostly nice but not being a real Schemer I'm conscious of my poor intuition. Are the continuation-closures heap allocated? in your implementation? in my implementation? am I supposed to care? am I allowed to know? (It isn't that I'm an efficiency nut but that I like to understand what's going on.)

The third I have not taken the time to understand, sorry. :-)

Here is how I'd write it in Common Lisp:

(defmacro while (test &body body+else)
  "(while TEST BODY.. (else ELSE..))"
  (assert (eq 'else (first (last-element body+else))))
  (let* ((body (butlast body+else))
	 (else (rest (last-element body+else))))
    `(if ,test
	 (loop do (progn ,@body) while ,test)
       (progn ,@else))))

I find this quite easy to understand and have no qualms about the generated code. But it's not as clean-looking as define-syntax and I'm surprised that I had to write this myself:

(defun last-element (list)
  "Return the last element of LIST.
.. as opposed to LAST which returns the last cons cell."
  (car (last list)))

BTW: I was interested in Olin Shivers' Danfest talk that he lamented the need for awkward longhanded looping in his Scheme code. I too share this objection to his Scheme code. :-)

Are the continuation-closures heap allocated? in your implementation? in my implementation? am I supposed to care? am I allowed to know?

That would vary by implementation, but any Scheme that can't optimize the use of closures well would be a little suspect.

That said, the other solutions are inherently less efficient than the first one, not for any Scheme-specific reason but because the first solution moves the first occurrence of the test out of the loop, avoiding any switching overhead inside the loop. A very smart compiler might perform such unrolling on the CPS version, but I wouldn't count on it. The only reason I gave the other solutions was because I thought you might be getting at clever ways to avoid doing the test twice, as would have happened if the Smalltalk chaining example had worked.

BTW: I was interested in Olin Shivers' Danfest talk that he lamented the need for awkward longhanded looping in his Scheme code. I too share this objection to his Scheme code. :-)

In Scheme, it's common to favor a functional looping style over mutating imperative loops. In that sense, the macros I gave are somewhat un-Schemely, because they require that the condition rely on mutable variables — there's no other way for the body of the loop to affect the condition. To avoid mutating your loop variables, you'd need something more like the R5RS 'do' or named 'let', where a variable like 'count' would be part of the environment managed by the loop mechanism, and a fresh environment would be created on each iteration. That might make the resulting macro less concise to use, or more confusing (see e.g. 'do'). If you don't want that, you could always just risk your immortal soul and use mutating loops...

The only reason I gave the other solutions was because I thought you might be getting at clever ways to avoid doing the test twice, as would have happened if the Smalltalk chaining example had worked.

Well there is a semantic difference too, if the test expression has side effects. I thought that was the reason to reject the first one.

True. I wasn't taking the problem seriously enough. :) I appreciated the delimited continuation version, BTW.

But wouldn't this have worked too, then?

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (if cond
         (let loop ()
           body ...
           (if cond (loop)))
         (begin alt ...)))))

Yes. That's essentially Anton's second version, except he set himself the challenge of only writing the test once. Transform what you wrote by lifting the definition of loop above the test:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (letrec ((loop
               (lambda ()
                 body ...
                 (if cond (loop)))))
       (if cond
           (loop)
           (begin alt ...))))))

There are two tests of cond, so abstract them into a function, thus winning the game:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (letrec ((loop
               (lambda ()
                 body ...
                 (test loop (lambda () (void)))))
              (test
               (lambda (body exit)
                 (if cond (body) (exit)))))
       (test loop (lambda () alt ...))))))

The first argument of test is always loop, so we can specialize test, then inline loop, then remove its definition because it's dead:

(define-syntax while
  (syntax-rules (else)
    ((while cond body ... (else alt ...))
     (letrec ((test
               (lambda (exit)
                 (if cond
                     (begin
                       body ...
                       (test (lambda () (void))))
                     (exit)))))
       (test (lambda () alt ...))))))

Voila.

Python has them (a while statement and a for statement with "else" clauses), but they mean something else altogether -- something like "... and if no 'break' statement was evaluated ...", which is one of the very very few things about Python that I've always found utterly non-intuitive. Given this baffling precedent, I'd suspect that any language would be better off without the feature. Or at least should not use the "else" keyword for it, since it clearly has different meanings for different people.

In the case of the 2e language, I had no choice but to put an "else" type clause in there, if I wanted the iterative (looping) inline conditional to be part of any expression. In the case of a normal inline conditional, it will always retrun a value (either the true or the false target). With the iterative version, I had a delima -- it would normaly return the "last" value of the true target, but what if the condition started off false? So I had to throw an else in there so that the entire phrase would still return a value.
Of course, I could have done a better job of picking the symobls to use, and I still don't have ones picked out for a break / continue, but perhapse I should start another thread on that topic. But I'm new here, so I don't want to start off any topics untill I figure out what all is appropriate.

There are two big reasons that your code might need a finally block, at least in some languages.

1. In Java, as with a variety of languages, there are both "checked" and "unchecked" exceptions. There are a lot of "unchecked" exceptions that a lot of code could generate (say, NullPointerException or ArithmeticException that could potentially be generated in a lot of places, but you don't have to put "throws" statements for.) There are also objects of type Throwable which is the superclass of Exception and Error. Errors include things like ThreadDeath or AWTError. Just catching "Exception" would not clean up after these different types of error.

2. Your try block may contain a return statement (or in some languages, a jump or goto or something nasty) that breaks out of the block without calling your cleanup code. In this case, it's simply safer belt-and-suspenders programming to ensure that no following programmers bung in a "return" statement into your logic and forget to clean up. In this case, the finally block saves you. It's thus a pretty good pattern when you realize that not closing or freeing something would be rather costly or destructive, and want to do all you can to prevent that from happening. It's a very useful and safe pattern whenever you work with programmers less perfect than you. :)
   

This seems to miss the case where the resources you're using come from a pool and need to be reset before returning to it. e.g. in Java we often use JDBC connection pooling, and the connection will still have references to it when we exit - because it's intended to be used again - but needs to be cleared before it can be used again.

It also misses the case where the threads themselves are pooled, as is quite common in large Java applications. We often have per-thread resources that need to be completely (and reliably) deallocated without killing the thread.

Good point. I should have caught that, as I spent much of today working on something where thread pooling was crucial if we didn't want our application server to die horribly. :-)

In this case the thread-local resource is really a "reference to an object from the pool" and not the actual connection itself. My point was that this property is to be declared at the time a resource is acquired rather than in a disconnected piece of code. It is object meta data. If such objects can be specified with a standard interface, then there will be no need to take care of them in finally blocks.

Pooled threads also have clear entry/exit points at which such cleanup can happen automatically, granted that Java as it stands might not provide such facilities right away. Imagine placing a single "finally" wrapper block to release all resources tagged as "thread-local" whenever a thread is returned to a pool.

Java's had thread local storage since 1.2. Not the easiest things to work with, but useful. Java's concurrency support is one of the strong points of its standard library.
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/InheritableThreadLocal.html

As far as finally, I much prefer C++'s RRID idiom (Resource Release Is Destruction -- the other half of RAII)

You can always use destructors in C++:

#include <iostream>

void demo(bool do_throw) {
  struct finally { ~finally() { std::cerr << "finally\n"; } } finally;

  if (do_throw) {
    std::cerr << "throw\n";
    throw 0;
  } else {
    std::cerr << "return\n";
    return;
  }
}

int main() {
  try {
    demo(false);
    demo(true);
  } catch (...) {
    std::cerr << "catch\n";
  }
  return 0;
}

Edit: Below is a version of the demo fn using FINALLY implemented as a macro:

#define CONCAT(l, r) CONCAT_DELAY(l, r)
#define CONCAT_DELAY(l, r) l##r
#define FRESH(x) CONCAT(CONCAT(x, _generated_identifier_), __LINE__)

#define FINALLY(...)                            \
struct FRESH(finally) {                         \
  ~FRESH(finally)() { __VA_ARGS__ }             \
} FRESH(finally)

void demo(bool do_throw) {
  FINALLY( std::cerr << "finally\n"; );

  if (do_throw) {
    std::cerr << "throw\n";
    throw 0;
  } else {
    std::cerr << "return\n";
    return;
  }
}

BTW, the code

try {
  block1();
  block2();
} catch (...) {
  block2();
  throw;
}

doesn't have the same semantics as with finally. If block2() inside the try block throws an exception, then block2() will be executed a second time. The correct translation is:

try {
  block1();
} catch (...) {
  block2();
  throw;
}
block2();

Do you use much the "success" scope?
I am asking because I did not introduce it in dodo yet.

Also, what does D do if there is a return or an exception occurs inside the failure scope?

Walter has described them as a list of delegates attached to each block. One per type of scope guard. At all exit points of the scope the appropriate guards are called(in reverse lexical order).

Explicit goto or throw is not allowed in a scope guard.
For the implicit exception, I haven't tested but I would assume all current delegates to be called in the outer block are still called.

http://www.digitalmars.com/d/statement.html#ScopeGuardStatement

I haven't used the scope(success) much.

I suppose it could replace some esoteric code, such as gotos out of nested loops, or perhaps used with fallthroughs in switch statements.

Explicit goto or throw is not allowed in a scope guard.
For the implicit exception, I haven't tested but I would assume all current delegates to be called in the outer block are still called.

From the description it looks like the exception is simply ignored. Which is equivalent to say "all current delegates to be called in the outer block are still called", except if the scope guard itself contains more than one instruction and one throws-- do the next instructions in the guard execute? I think they should not.

Also, is the caller aware at all that the delegate encountered an error? That could be done by chaining the exception to the original exception, supposing there was one in the first place.

You might be interested in the article Exceptional Syntax by Benton and Kennedy.

A good read, indeed. Thanks.

I see that I can express try... in... unless... using, logically, continuation passing style in dodo. However the imperative syntax does not support this.

Judging from the JVM workout, I think that a valid solution is to use labels and label matching in the catch block-- I was thinking of labels to support resuming anyway. Not as pretty as the author's proposition but it should do the work.

    String catpartial(String[] list) {
       String n list[1]; String[] ns list[2+]
       #try
(1)    File f FilePath(n).open(fRead) ==> f.close
       #in
       return readIt(f) + catpartial(ns)
    <catch error="e" at="loc">
       #unless
       match (e.type, loc): IOException, (1)
           { return catpartial(ns) }
       .
    </catch>
    }

In a finally block you need to check if file is null, but not so with the scope method. If the open method throws the scope statement will never be executed.

You do?

file = new File( 'foo.txt', FileMode.Read );
try {
  // ...
} finally {
  file.close();
}

I'm not saying this is better than your version - it's not. But you don't need to.

file = new File( 'foo.txt', FileMode.Read );
try {
  ...
} finally {
  file.close();
}

I observed that pattern too, that is why I chose the following syntax for my language:

file = new File( 'foo.txt', FileMode.Read );
try {
  ...
}
file.close();

Disadvantage is that the finally block has no explicit boundaries, advantage is that the cleaning-up (close file) is at the same level as the resource-taking (open file). I like the symmetry, but I am not sure developers will be comfortable with implicit finallys coming from Java etc.

What if the 'new File(...)' throws an Exception?

Then file.close() is not run, obviously, as you haven't even entered the try block yet. This is, as you yourself stated, the intended behaviour: if the file wasn't even opened, there's no point in closing it.

Again, your version is no doubt better, in the sense it's more readable: quoting Digital Mars "It places the unwinding code where it aesthetically belongs." But they are behaviourly the same (as you can see if you read through the linked page).

My (minor) point was the exception thrown would not be caught in your code.

Then, I just don't get it. It wouldn't be caught in your version either, would it? You'd need a try-catch for that. So my version would need a second try block. Ugly? Yes. I've never actually used it though, nor have I used the null checks in finally you have suggested.