Lambda the Ultimate

An interesting post. I'd like to see lift. At the moment I'm particularly interested in your OR mapper, as we're spending a lot of time on ours at the moment (we use Scheme).

WRT performance I think it is attributable to two things:

1. Scala is a better match for the JVM
2. All that JVM technology really does work.

Ruby is quite well known for being dead slow.

I find the Jetty continuations amusing. Sun seems dedicated to keeping the JVM stone-age, but people just find ways of working around it regardless. MS seems much more open to innovation in this respect.

Groovy can also deliver the closures, Java-like syntax with fewer keystrokes that you mention, and it has Grails, which implements the RoR model. I have yet to use Scala (due to time limitations) so can't comment any other comparative aspects.

I hesitated before posting, for what it's worth. I feel somewhat vindicated by the fact that Dave Pollak noticed the post and replied to it. I think his post is a good hook from which to hang our PLT hats if we wish to do so. I confess, though, to also having decided to go ahead with the post due to my belief that LtU is a place where theory and practice are supposed to meet. If this post is heavy on the practice and light on the theory, welllllllll... hopefully that balance will shift at some point in the future. :-)

But I must apologize to our hosts if my judgment here was inappropriate.

No, this is all goodness. Gives insights into Scala and additional info on the runtime constraints of Ruby. Would like to see more such posts.

"I feel somewhat vindicated by the fact that Dave Pollak noticed the post and replied to it."
Why? Isn't it ordinary for a blogger to direct traffic to their blog?

I think if an expanded version of what Dave Pollack talked about here had been in the original blog entry then there would have been far more reason to post a link here - "What (in detail with examples) makes Scala a great environment to write a web framework in?"

It's hard to disagree with "So, yes, I'm pimping my own framework,..."

Dave made some claims, based on his experience porting an app from Rails to his framework, on his blog, that anyone with any exposure to Scala could reasonably interpret, IMHO, to be of interest to LtUers. Since Scala even has its own "Focus on" section here, the mere fact that a web application framework has been written in it is sufficient to make it of interest.

As for "why," I suppose because not everyone would feel comfortable joining such an obviously theoretically-focused site and discussing their work. So it was gratifying when Dave did, and his comments, I think, confirmed my belief that lift is of legitimate interest from a PLT perspective. If you want more detail, why not ask Dave for it?

Finally, what, exactly, is wrong with him pimping his own framework?

Finally, what, exactly, is wrong with him pimping his own framework?
Absolutely nothing - it's a fine thing for him to do on his blog - is it front page news?

(afaict there really are an awful lot of web frameworks that haven't been mentioned here.)

...is, of course, that he has direct experience in migrating from Rails to a framework that gives him a roughly comparable level of ease of development/concision with the added benefits of greater scalability and type safety than Ruby offers. It supports a level of interest in extremely expressive type systems and provides a data point with respect to their pragmatic value. The fact that he used Rails for 18 months first should tell us something, too: Dave isn't some wild-eyed static typing proponent who wrote his framework "because he could." He appears to have written it because, 18 months in, he still had problems that he wanted to solve.

It's true that there are an awful lot of web frameworks that haven't been mentioned here, perhaps because most of them really don't add anything to the PLT discussion. But continuation-based web frameworks have been discussed many times. haxe likewise. I could add more. But lift is written in Scala. Scala seems to be of interest to LtUers, so I thought I'd post a story about it.

Once again, given that Ehud has offered his opinion that the intersection of theory and practice is consistent with LtU's purpose, I have to ask why we're having this conversation.

Dave isn't some wild-eyed static typing proponent who wrote his framework "because he could." He appears to have written it because, 18 months in, he still had problems that he wanted to solve.

Note that the issues mentioned aren't necessarily purely static typing issues (unless you want to nitpick about all static checking being type checking, but that doesn't affect the point anyway).

For example, in Schemes with decent module systems, typos in procedure names are caught at compile time (or more generally, macro expansion time or module elaboration time). Luke's comment elsewhere in the thread similarly points out the static error checking performed by Erlang ("unbound variables, unused variables, calls to undefined local functions, unreachable code, etc.") The point being that it's possible to do a lot of static checking without resorting to a static type system, and some languages do exactly that.

In the Ruby case, the problem is presumably that since its methods are looked up at runtime, it's not really possible to perform the kind of non-type-based static checks that Scheme and Erlang can do. So in that sense, there is a static typing issue here, and Ruby is probably similar to Smalltalk in that respect. However, you can't generalize this to all non-statically-typed languages, any more than you can draw negative conclusions about static typing in general from the type systems of Java or C++.

If I were to critique Dave's points, I would also suggest that "catching typographical errors" isn't a very compelling example of a benefit of static typing, for all of the reasons that you, Luke, and others have pointed out. AFAICT, every Common Lisp, about half the Schemes, and apparently most other non-Python/Ruby dynamic languages will at least tell you when you attempt to refer to an unbound variable!

What I find more compelling about Dave's comments, of course, is the observation that he needed about 95% test coverage in Ruby not to get bitten by something—typos or otherwise—and that the lift checkins had, as of that writing, resulted in only one defect. Now, naturally, the devil is in the details: how many of those defects were due to Ruby's lack of warning about typographical errors and would even have been caught in, e.g. PLT Scheme or SBCL? OTOH, would, e.g. PLT's web framework or UnCommon Web provide the same level of concision as Rails or lift? These strike me as questions very much worth asking, although answering some of them would involve rewriting Dave's application in each language and framework to get a realistic assessment. So again, what we have here is a data point, no more—but also no less.

Once again, given that Ehud has offered his opinion that the intersection of theory and practice is consistent with LtU's purpose, I have to ask why we're having this conversation.
Probably for the same reasons that you initially "hesitated before posting".

Let me be explicit about my tone - not critical of you or Dave Pollak, just mildly puzzled. There probably are a couple of stories waiting to be told about his experience - a story about how the framework actually performed, and a story about what aspects of Scala made writing the framework so much easier - but in my opinion the blog entry lacks content. (I wish I could find the 3 part story about converting a PHP bulletin board system with hundreds of thousands of users to Rails, but I can't - they told a good story, lots of content).

I don't recall knowingly posting any "theory" material to LtU - I certainly agree with Ehud's opinion that the intersection of theory and practice is consistent with LtU's purpose. However, I think that the number of occasions on which Ehud expressing an opinion was a strong signal that others should cease to express their opinions has been thankfully few.

My opinion certainly isn't meant as law. It is my opinion. Still, since I've been here awhile, I think my opinion may be relevant.

Two things are worth noting as regards the issue currently under discussion: First, LtU was always a mix of theory and practice. This approach seems to have served us well. Second, the idea behind having contributing editors is that they each bring his or her own perspective. I trust their judgement and appreciate what they contribute. I have yet to regret following up on the items chosen by the LtU editors.

I think we should all be grateful to the editors, and encourgae them to post more... If an item seems less interesting to me, I simply move on. I think that at the current rate of new posts, and the overall quality of them, this makes the most sense.

Thanks, Isaac, for the cogent and reasonable response. I must apologize if my question seemed insincere. I suppose my own puzzlement was that, once Dave had posted something about his experiences, the connection to properties that Scala has from a PLT perspective were more apparent than they might have been from the blog post alone. Of course, that in itself is a subjective judgment that might or might not be shared by others.

If we're all of an accord that the intersection of theory and practice is appropriate to LtU, though, my puzzlement over others' puzzlement only deepens. If a post says that there was a system written in X, and that system was rewritten in Y, where X and Y are known to have quite different characteristics from a PLT point of view, but the post itself says nothing about them, isn't it still rather obviously a pertinent post, if only because the person doing the rewriting has perceived some value in these quite different characteristics, and we readers might be able to articulate a set of questions about how those values map to the PLT characteristics? And let me be clear: if I had found a post that said "I had a system written in O'Caml, but I rewrote it in Ruby and it's much better along dimensions X and Y," I'd post that, too; the point isn't PLT partisanship (hard to believe in my case, I realize), but just that someone has, for a change, a very specific story to tell.

In any case, my apologies again for leaving the "Ehud wrote it; I believe it; that settles it!" impression. :-)

I think it's as front-page worthy as the article about Erlang being used in Vendetta Online. Scala clearly is an interesting language from a PLT point of view, and seeing it used in production systems is great. I presume you don't actually want all the good languages to stay academic :-).

I remember this one from my childhood. (I might even have been young enough to confuse it with directions, which makes it really baffling.)

Two wrongs don't make a right.

I'm guessing that you missed Ehud's post on the subject. It'd be nice if we could move past the (resolved, as far as everyone else seems to be concerned) issue of whether the post was appropriate or not and perhaps discuss whatever lessons might accrue from Dave's experience. He's used Rails for 18 months, learned Scala, and is developing a web application framework in it—surely there are questions of a PLT nature that we can arrive at, given that?

That's what I was aiming for—not to go around the dynamic/static merry-go-round again on the basis of theory, but to put something concrete on the table because, let's face it, hands-on experience going from one platform to a very different one in a production context is rare. I think Dave has a number of interesting things to say about how Scala met his needs, in this case, better than Ruby did, and some of those reasons have a PLT flavor to them.

Paul: I confess, though, to also having decided to go ahead with the post due to my belief that LtU is a place where theory and practice are supposed to meet.
That's my opinion too.

Chris: The relevant question should be whether the story has something interesting to say about Programming Languages

Yes, that's a good question to use in deciding what's worth discussing on the home page.

the real world (also known as the JVM ;])

It's interesting how many different Real Worlds there are in computing. The one I live in doesn't have any Java at all and everybody is driven by having fun and creating things. There's a lot to be said for finding and moving to a better world even if it's a bit less populous than others. Happily most of the common objections (e.g. monetary) are strawmen.

I don't have the eloquence to express this idea as well as I'd like. Hopefully Paul Graham will write an essay on the topic one day.

It's interesting how many different Real Worlds there are in computing.
It really is!

When I see "real world" mentioned in discussions I begin to wonder if their real world has much in common with my real world.

Once I get past my own vanity, I do find it fascinating how many very different kinds of computing are out there.

(otoh "real world" just seems like bad sampling.)

Luke Gorrie: Hopefully Paul Graham will write an essay on the topic one day.

He's written several, with Beating the Averages and How to Do What You Love being the most obviously relevant.

But if you watch the Scala Experiment video, you'll be reminded that the Scala experiment is an experiment along at least two dimensions: yes, the language design experiment, but also the language adoption experiment. It's not hard to create a language. It's only marginally harder to create a language that occupies some niche for some number of years, especially if you're willing to define "niche" as "me and my colleagues." But to get 100,000 people using your language—that's much harder. I don't know whether Scala is there, or will get there. But it's certainly an easier sell when it runs anywhere Java runs, integrates trivially with existing Java code, can be developed in Eclipse, IntelliJ, etc.

It's true that, strictly speaking, popularity isn't all there is to the "real world." But there seems to me to be a certain petulance, not to mention lack of realism, to the insistence that people can merely choose a smaller "real world" when popularity matters to them and they use the phrase "real world" as a shorthand that everyone understands to mean "popular."

While I like Mr. Graham's ability to stir things up, I have to wonder if he's gotten past the "if it bends it's funny" into "if it breaks, it's not funny."

There are many, of course ... I've dipped a toe in quite a few over the years. I am quite encouraged by the current trends in language development towards well-defined VMs, and being able to call out into them from new languages. Scala is quite successful in this regard, as are a number of other languages (Nice, F#, and many more)...

I vividly remember when I switched from Java to Emacs Lisp being plagued by typo bugs and shocked that people would actually program in such a way. But quickly I adapted and these days it's a non-problem. I think the main reason is that I unlearned the subconscious habbit being sloppy and relying on the compiler to check my spelling. I also think that completion features like dabbrev-expand are a great contribution. Just think -- how often do you misspell a filename in the Unix shell? Hardly ever if you habitually tab-complete to check it.

Erlang reports a LOT of errors/warnings compared with Emacs Lisp: unbound variables, unused variables, calls to undefined local functions, unreachable code, etc. Programs like xref and Dialyzer do much more checking but for me at least they're too inconvenient to bother using (i.e. batch-mode separate from normal compilation).

Erlang programmers do occasionally waste a lot of time chasing a bug caused by e.g. a typo in an atom being sent in a message and then curse the lack of static checking for the next week. I have heard of projects where each atom is ?defined as a macro so that the compiler will detect typos, but I wouldn't do that myself.

The story with Smalltalk is similar - the programmer does the work to avoid typos - use the browser tools to check the method selector name, use copy/paste instead of typing characters, use completion - and the compiler warns about undefined method names.

And of course some programmers are more diligent than others ...

I wasn't kidding when I said that you could have been me writing under a pseudonym. I chose to create a front-page story at least in part because I was struck by the parallels in our experiences (OK, I have yet to write my own web framework). I can also characterize myself as a "recovering mutability junkie," find Haskell too rich for my blood (but enjoy O'Caml a lot), and am extremely pleased to find something as sophisticated and powerful as Scala living on top of the JVM.

Since LtU is somewhat self-consciously a "programming language theory" site, it has too often degenerated into the standard flame-wars (of which I'm ashamed to admit to having been among the most vociferous participants!) regarding static vs. dynamic typing. One difficulty with this, er, "debate" is the rarity of real-world conversion experiences. Another difficulty is that if you introduce real-world experience, you're no longer talking about PLT. So the structure can end up being self-defeating, or at least self-limiting. Since our host was gracious enough to ask me to be an editor and I was crazy enough to accept :-) I chose to exercise that discretion in creating the story, to some extent for good and proper reasons that have been discussed here at more length than I actually feel is necessary, but yes, also for more personal and questionable reasons of feeling a bit vicariously vindicated and wanting to escape that self-limiting trap of being challenged to produce practical evidence in support of my views and practical anecdotes not being the sort of thing we discuss on LtU.

Finally, I'm very glad to have made the choice that I did, because it brought you here, and you do such an excellent job of describing your experience and choices in a cogent, balanced, and patient fashion—qualities that, again, are often sorely lacking in such discussions. Welcome to LtU.

Unless you got lucky (and usually you wouldn't know that until later) and started on a language like Haskell, everyone is either a mutability junkie in denial or a mutability junkie in recovery.

"qualities that, again, are often sorely lacking in such discussions."

Not nearly as much on LtU as most places.

Scala's Actors are critical to lift. I do not need continuations in order to implement the ask/answer paradigm that Seaside implements with continuations.

I'm too lazy to read Scala's documentation, but if its Actors are the same as other languages they are a lightweight concurrency mechanism. You can fake continuations with threads by spawning a new thread every time you want a new continuation. Is this what you're doing? If not, how to you handle the back button? Do you require the programmer to do a manual CPS? Is this what the state machine thing you describe is for? Mucho curious! Thanks!

I'm not using Actors to deal with the back button. I'm working on versioning model and session state by using immutable objects to store state (e.g., session variables, instance variables) and then rolling back to a previous version.

Seaside uses continuations for its Ask/Answer paradigm. You can see an example at http://liftweb.net/lift The "enter your name" form is a "question" asked by the chat application. Until the question is answered, the view and controller is the "AskName" controller. Once the question is answered, the chat controller (which is the controller defined in the view, and lift is "view first" where the view defines the layout and the controllers) then becomes active. The relevant code is and a more detailed description can be found at:
http://liftweb.net/lift/chat

The state machine is for extra-session state. For example, a user signs up with the service and receives a confirmation email. How does one describe the timeouts to send reminder emails and ultimately terminate the account? Here's how:

  def states = List(
      State(Initial, To(NewAccount, {case _ => })),
      State(Terminal) entry {terminate},
      State(NewAccount, After(3 days, FirstNewAccountNag)),
      State(FirstNewAccountNag, After(7 days, SecondNewAccountNag)),
      State(SecondNewAccountNag, After(14 days, EndNewAccountAccount)),
      State(EndNewAccountAccount)
      )

  def globalTransitions = To(Terminal, {case Terminate => }) :: Nil

  override def transition(from: StV, to: StV, why: Meta#Event): unit = {
    to match {
      case FirstNewAccountNag =>  user.obj.sendFirstNagMail
      case SecondNewAccountNag => user.obj.sendSecondNagMail
      case EndNewAccountAccount => user.obj.disableAccount; this.processEvent(UserNag.Terminate)
      case _ => // ignore
    }
    super.transition(from, to, why)
  }

The "machine" takes care of persisting the states, transitioning the states at certain times or on certain events. There can be multiple states and state types associated with a given instance of a model.