Lambda the Ultimate

Sun did not receive Phase III of the DARPA HPCS contract. After the end of Phase II, the rest of the Sun HPCS effort was folded into their regular product groups, but Fortress was maintained as a Sun Labs project.

I expect there is no objective fact of the matter at this point as to whether it will be maintained (see Oracle's recent confusion about SPARC, a much larger product).

Also, Guy Steele heads up the Fortress effort (the real structure of the group is difficult to describe in a comment box).

[A former intern in the Fortress project]

Guy Steele said in an interview (I believe it was in "Coders at Work" a few years ago) that in hindsight, they should have designed Fortress with a pure-functional core like Haskell. Now might be a good time to do that. :)

I think fortress was started with what was, in retrospect, a bogus theory of what kind of language design efforts get commercial support. Sun commercially supported the effort, sure, but also basically failed. Sun was (by corporate culture) a hopeless romantic in that sense.

My *guess* would be that Steele is rich enough to retire and "just (as in only)" make art. He should, I wish to suggest, do so. I'm less interested in what Steele can do with a funded team than in what he can do if he has to keep it on a scale he can do mostly alone and without funding. I don't mean that I wish hardship upon him or his colleagues or anything like that. He just shouldn't be so worried about "products" right now -- rather he just went with what seemed personally interesting and fun.

Even with your qualification near the end, I can't help but read your comment as astonishingly crass. "Retire, old man?" Wow. Just wow.

I don't read it like that at all.

I also felt it was a bit overboard, but decided that the emphasis is on "just (as in only)" make art. And being called an artist is a great compliment in my book.

Art or not, positive or not, I find that kind of career advice for an individual to be bad form. It's one thing to suggest that a group should focus on this or that technical problem or area, but this seems overly personal to me.

(but it won't happen).

Art or not, positive or not, I find that kind of career advice for an individual to be bad form.

I can see that perspective and will keep it in mind but I think people are reacting oddly. I was tempted to go over the various - and divergent - ways people have "read" my comment as wrong or insulting or whatever but, fine.... people are taking in those ways regardless of what I say. I consider the readings pretty ungenerous and needlessly fussy but it isn't worth a long discussion.

For the record, I didn't read it that way at all.

How much of Sun's budget did this project really cost??

At least. The DARPA funding ran out four years ago and Sun continued to finance the Fortress group to the tune of 6 full-time staff members--quite senior people at that--since that time. I don't know how long Oracle will continue the project.

What other languages/systems are in the works or on the table that can accomplish what Fortress set out to achieve? And if anything, the demand for that type of high-performance alternative to Fortran and C is growing. For various reasons, I don't see mainstream managed languages (C#/Java) filling that gap.

...so one could imagine that X10 is the alternative.

Regardless, all three languages have roughly the same approaches. Fortress was simply too radical to win. It was too clever by half to catch the eye of DARPA.

What other languages/systems are in the works or on the table that can accomplish what Fortress set out to achieve?

Why does there need to be an alternative? Are you saying Fortress can't succeed without a corporate sponsor?

Are you saying Fortress can't succeed without a corporate sponsor?

No

24 * $200,000 US = $4.8 M US.

I suppose that is quite a waste of money, especially to somebody frugal and scrutinizing like Larry Ellison.

What theory do you think Fortress was started under? I think Fortress, at least implicitly, espouses or has espoused, the following theories (and others):

• That we should design languages that can grow. This is the most central idea.
• That language design did not end with Java.
• That better programming models than MPI were and are possible.
• That bringing language much closer to specification is a good idea.

I'm sad that these ideas didn't win the DARPA Phase III funding (which I don't think was mostly about PL at all), but I'm still hoping for them overall.

What theory do you think Fortress was started under?

I mean theories about who could/should/would pay.

I know what you meant, but I don't know what theory about funding you were referring to.

Briefly, that big titles for researchers in corporate R&D labs were more like tenured academic posts than they actually are.

+1, at least from my experience. But I'm sure someone like Guy Steele has enough star power to start projects that are riskier than the ones we could start.

First, I don't think that Guy has the mistaken theory which you attribute to him.

Second, as someone who these days works on a large, academically-developed, software system, I think the funding situation for developing a real language in an R&D lab is better, not worse, than in academia. It's very hard to get credit in academia for writing and (especially) maintaining software.

First, I don't think that Guy has the mistaken theory which you attribute to him.

Here is an example of what I meant by Sun's corporate culture having had what is today a romantic view:

from The Register, writing in 2005

Sun showcased a selection of its research efforts at the Computer Science Museum in Mountain View last week. Established by Ivan and Bert Sutherland and Bob Sproull in 1990, Sun Labs has an enviable reputation for pursuing difficult problems, and investing in long-term research.

Serious computer research has been under pressure in recent years from two areas, finance and the media. In Wall Street's short-sighted view, systems companies have no business doing systems research, which is adequately covered by Intel, Microsoft and favored start-ups. But research has also been under pressure from what we might call 'Disneyfication", which reflects the media's obsession with novelty and a belief that technology works well enough already and can only get better.

The article goes on to explain how Sun Labs bucks that trend, but only while struggling to do so. It concludes with something pretty interesting for having been written in 2005. They write of some unimpressive demos of sensor-focused robotics by the lab. They snidely wrap up (it is "The Register," after all):

Seeking what deep philosophical thinking informed the sensor work, we rushed to one talk that offered to explain the "magic" at work, only to catch the very end of the presentation. The speaker talked of reinventing TV by putting the web, the chat room and email into a high definition TV, with "... Sun sensors which are wearable, that track body telematics that allow Magic Wands, that have Bots that have presence, and connection into the virtual. Television at that point becomes much more like a renaissance fair!"

And then we noticed that a member of the audience, who subsequently turned out to be from a "Cyber clothing" company, was dressed in a medieval jousting suit - as if he'd stepped out from a game of Dungeons and Dragons. Clearly the Rise of the Machines™ is more pervasive than we thought, and we must investigate further. Even at Sun Labs these days, the serious work wrestles for attention with the fatuous. ®

One historical point of note is that, then, apparently Sun was trying to invent and impress with Kinect, Wii, and Google or Apple TV. As were lots of people. See what that implies for the sustainability of such a lab's business model?

When I read an article like that what I mainly read is how the classic commercial research labs are challenged mainly by issues of financing. I don't mean that they have trouble competing for grants. I do mean that they came to be widely perceived as often being bad investments.

They are perceived as being bad investments not because they fail to innovate and create value, but because they take too long and it is too hard for the sponsors to capture much of that value. The spreading awareness of how these labs looked to private capital formed over the course of the 1980s and 1990s and the social and economic arrangements that prevail changed accordingly. Many commercial labs, rather famously, got gutted. Most got warned to change their tune and work harder on shorter term, demonstrable returns. To a first approximation, our society put the breaks on long term computing systems research. (To a second approximation, we kept alive a few areas of such research for various reasons.)

Incidentally, the big academic labs all got the same pressure. DARPA, NSF, etc. are ultimately state security tools. Their "returns" are measured in overall economic growth and strategic advantages for the sponsoring state. A lot of what was once called systems programming research is irrelevant to those aims, just as it is to the aims of private capital.

Within firms like Sun, and my impression is that Sun was itself like this, the top and more or less tenured staff (e.g., the "Sun Fellows") partly came to to earn their keep as public and customer facing technical experts who were also key advisors to top management and the board. And that's terrific! Smart people with sufficient social skills and training should do exactly that, quite often. It is just that grafting that advisory and PR role onto the old-school notion of how labs were supposed to work translates into larger-than-small payrolls for research support teams and long-term projects (like Fortress). Those payrolls are a problem if they don't produce ROI curves with sufficient area under the curve in sufficiently short amounts of time.

Fortress may well live. Prbly will in some form. It's emphasis on parallelism and scientific computing make it an interesting fit for a database brand and "vertical solutions" company, especially these days.

Yet one has to ask the business question: what is our forecast for Oracle's books if the project continues well funded and on its own natural course vs. how do they look if it gets cut way back or morphed into something barely recognizable? Under the old-school model of labs, that's a valid but not central, time-critical question. These days, it's the primary question with some urgency.

The entire remainder of Sun Labs must survive a thorough combing over and examination by Oracle's control systems, both to see how well it integrates into the management power structure (if at all) and, more importantly, how its operations help rather than harm returns from the acquisition of Sun.

Such an outcome for the lab was inconceivable when it was founded, especially in such a short time period. You ask Ivan Sutherland to create a home and guest house for researchers when you are digging in, putting up a large fence and guard towers, and trying to establish a small strong-hold ... a base of operations that you expect to last a long time. A fortress, if you will :-). You hire some multiple of 10 people to work on a long-term project in language design when you think the environment is suitably stable to make that a responsible move. As it turns out, all those beliefs that justify creating such a lab and such projects turn out to be somewhat romantic hopes rather than necessary truths. The aristocracy is, so to speak, increasingly ambivalent and economically disempowered for finding stable seats for favored researchers in these areas.

Spam?

There doesn't seem to be obvious spamming intent, two guesses:

• A poorly written unprocessed brain dump from a budding (if somewhat confused) language designer.
• Somebody is testing an automated weblog posting tool, like the ones that are used to automatically write papers that are then accepted for publication.

My hunch is the second, perhaps with a goal of creating a more intelligent spam bot down the line.

I of course meant spam in the most general sense of the term...

Nah, looks like an experiment in HCI; double posted, of course, to generate the correct amount of interest. ;) The generalization from SeeLisp to ending the war in Iraq is bit far-fetched. I guess it's okay in the sense that if some people believe a C compiler is an endofunctor he's entitled to his opinion too.

IMHO, he should just have posted a new topic.

OK then.