Lambda the Ultimate

muse-v0.2cp-darwin-ppc.tar.gz (50kB)

Just updated the processes branch with porting instructions for MacOSX PPC. So added a release build for MacOSX PPC users.

I've moved the above downloads to the new Google code Downloads page.

I had the option of implementing continuations to be invokable across process boundaries. In fact, its hardly a line of C code or two to enable that. The semantics weren't very clear to me though and I hope to study it a bit more. I also think that message passing between processes satisfies most of the situations for which cross-process continuations might be useful. So there's an alternative there from a design perspective.

As for linking processes for fault tolerance, I'm considering that but haven't implemented it 'cos its easy to add support for it at the scheme level using closures and the exception mechanism. For example,

(define with-supervisor
  (fn (sup thunk)
    (fn ()
      (try (while (not (thunk))) T) ; Run the thunk till it evaluates to T
      (sup 'SlaveDied)  ; Notify the sup that we died.
      T)))

Transforms a thunk that you need to evaluate in a separate process into a thunk that will send a message to a given supervisor upon death. You can then spawn a thunk with a supervisor like this -

(spawn (with-supervisor sup thunk))

Anyway, the current implementation in muSE is more like a playground for me to explore these concepts and I thought others might benefit from it as well. As is noted on Termite's post, it takes very little code to implement these things :)

To me invoking a continuation from process A in process B would mean that B would become a copy of A at the moment the continuation was captured. Except perhaps that it still returns its value to the same point. And I'm not sure if it should copy A's pid or continue using the old pid.

Yes, unless your going into distrubution there it could be a hardvare error in the new process, which wherefor couldn't be handled in that process.

Othervise I must say muSE seems like an nice lisp dialect. Especially using { and } for read time evaluation.

On cross-process continuations, I went through the same thought process and decided that I wasn't clear enough. For example, if you have the continuation invocation complete in the calling process, then the calling process needs to distinguish between a cross process continuation and an in-process continuation invocation. That wasn't a consistent enough model for me. So I've (sort of artificially) forbidden cross-process continuations.

On supervisors, I agree that what I wrote in my post is no substitute for direct exception propagation (as in Erlang). muSE doesn't check unix signals yet as well, to trap hardware errors. So its a bit too early and I make no claim to total fault tolerance.

Thanks for the note on the {} notation. It came about almost by accident, but it has proven to solve many representation and optimization problems rather elegantly.

You might be right there. I was stuck at the problem of having to decide which pid to use. The captured continuation might have a reference to the original pid whereas the invoking process will have another. I couldn't figure out (at the time) how to make them all consistent (A or B) when B starts executing like A. Hmmm .. will revisit the topic this weekend. Thanks for the tip.

Are you referring to define, try, case, let, etc? Yes they are all first class as well. Its valid in muSE to do -

(define using let)
(using ((a 10) (b 20))
   (write (+ a b)))

Basically, there are no "reserved symbols".

[edit]
However, I usually refer to them as "sort of first-class". Though you can play with the symbols as you like (in the manner above), there is a restriction on the constructs that introduce bindings within sub-expressions - the only 3 of them being fn, let and case. When they are used inside closures, they must be known at closure capture time. For example -

(define a 5)
(define b 6)
(define joke
  (fn (mylet)
    (mylet ((a 100) (b 200))
      (write (+ a b)))))
(write joke)

will print a definition of joke that you probably didn't intend -

(fn (mylet)
    (mylet ((5 100)
            (6 200))
           (<prim:18c3> (<prim:d3b> 5 6))))

If you do want to plugin different definitions of let into the body of joke, you could use fn: instead of fn in its definition. That causes joke to be defined using dynamic scoping. If you do that, you get what you want -

(joke let)

will then print.

300

Yeah thats tricky to. I have been thinking about seperating mailboxes from processes, but im not sure of that.

A mailbox is essentially a FIFO queue of messages. So there is no difficulty there. I guess your doubt is whether a separate queue implementation is useful (design-wise) when each process comes with its own queue. Am I right?

Pattern matching does make the queue different. I'm not very experienced in this matter, but how useful is the facility to dispatch messages out of order in general? I chose to experiment with FIFO dispatch for the moment in muSE because I wanted to control concurrency using atomic blocks. With out-of-order dispatch, when I do

(atomic (procA 'take 10)
        (procB 'give 10))

there is no guarantee that some other process won't interfere with procA and procB's mailboxes after exiting the atomic block before those messages get processed. With FIFO dispatch, this guarantee is fairly easy to provide.

I didn't understand your comment about "reificating the mailbox as a closure". Can you please explain?

If you want to use processes as active objects as in Erlang I think its rather necessary as the process might be in a state there its not ready for a certain kind of message, but expects annother kind of message, and if it would then get a message of the first type it would stall forever, if it has to take care of them in order. And you could still access the messages in order by not patternmatching on them.

But i dont know, you might be using a slightly diffrent concurrency modell so it might not be a problem. Have you tried using muSE for any substansiall concurrency programing?

Not really, but will soon be. It is basically the same concurrency model of another system that I'll be moving over to muSE. The processes I need to support have the property that each can accept any of a set of message types at any time. There is no "sub-protocol" with these processes as they largely work in streaming mode.

Time to do more research I guess.

I checked the meaning of "reify" and now I understand what you mean :)

In muSE, I've generally chosen to avoid passive identifiers whose sole purpose is to, well, identify an object. For many types of objects, there is an important function they serve that makes it worth presenting them as a closure. For example, vectors are functions from indices to objects, hashtables are functions from keys to values, ports are functions whose sole side effect is to transport their argument message across, pids are functions which send their arguments to the process' mailbox as a message. It makes the language feel cleaner. Though there is no attempt to unify ports and processes, the fact that pids and ports are both functions that send messages lets you treat them without worrying about whether you're dealing with in-process communication or across the network.

It makes higher order functions more powerfull to. I really think more languages should do this.