Lambda the Ultimate

inactiveTopic CLR Hosting
started 2/22/2004; 1:19:39 AM - last post 3/5/2004; 3:44:53 PM
Ehud Lamm - CLR Hosting  blueArrow
2/22/2004; 1:19:39 AM (reads: 10022, responses: 20)
CLR Hosting
Lisp oriented weblog you say? It seems like we are turning into a .Net weblog...

Chris Brumme is on a roll. This gargantuan entry concerns hosting the CLR in such environments as SQL Server and ASP.NET.

My background as a system programmer and DBA makes me especially fond of the issues involved in hosting language runtimes, and Chris's discussion makes for very interesting reading.

Two side notes:

The vast majority of execution inside SQL Server was via Transact SQL or TSQL. Any application written in TSQL is inherently scalable, fiber-aware, and robust in the face of resource errors. Any computation in TSQL can be terminated with a clean transaction abort.

Unfortunately, TSQL isnít expressive enough to satisfy all application needs. So the remaining applications were written in extended stored procedures or xprocs. These are typically unmanaged C++. Their authors must be extremely sophisticated, because they are responsible for integrating their execution with the unusual threading environment and resource rules that exist inside SQL Server. Throw in the rules for data access and security (which I wonít be discussing in this blog) and it takes superhuman knowledge and skill to develop a bug-free xproc.

In other words, you had a choice of well-behaved execution and limited expression (TSQL), or the choice of arbitrary execution coupled with a very low likelihood that you would get it right (xprocs).

This gives a nice perspective on DSLs vs. general purpose languages.

From time to time, readers of my blog send me emails asking if there are jobs available on the CLR team. At this moment, we do. Drop me an email if you are interested and Iíll forward it to our recruiter and our hiring managers.

If you are looking for a job in industry at the moment, I suggest taking a close look at this opportunity.


Posted to cross-language-runtimes by Ehud Lamm on 2/22/04; 1:23:01 AM

xeo_at_thermopylae - Re: CLR Hosting  blueArrow
2/22/2004; 2:20:59 PM (reads: 519, responses: 2)
Lisp oriented weblog you say? It seems like we are turning into a .Net weblog...

Becoming? I though LtU was a SPAM site for .NET. FWIW I wish I were joking.

Franck Arnaud - Re: CLR Hosting  blueArrow
2/22/2004; 2:43:33 PM (reads: 511, responses: 0)
If you are looking for a job in industry at the moment, I suggest taking a close look at this opportunity.

It makes me more want to change profession and come back in twenty years to see if the mess is sorted. With all the new kinds of unsafe stuff in these environments, I suspect that when people are finished with exploiting buffer overflows :-) they will probably try to trigger locking or finalizer or caching bugs by sending the right sort of input to JVM/CLR machines. It's not obvious one wants to help build the weaknesses of tomorrow's IT infrastructure. Anyone doing deterministic computers?

Chris Rathman - Re: CLR Hosting  blueArrow
2/22/2004; 7:01:08 PM (reads: 483, responses: 1)
I though LtU was a SPAM site for .NET.
Easy way to fix that. Become an editor and submit whatever subject interests one.

Only thing I dislike about the .Net literature is it's silence on those who have tread this terrain before. Oracle put the JVM inside the database server a version or two back. PostgreSQL has supported multiple languages for some time now. Why nary a mention of the experiences of these two platforms?

From my own perspective, I still do quite a bit in T-SQL and while I've seen some gradual improvement in the language, I'll say flat out that Microsoft needs to put more effort in improving T-SQL not by passing the problem off to a JVM. There's a host of things that could be done to improve the language, and from what I see, they are not doing anything but placing a bandaid on the language - instead chasing the latest marketing push of xml or .net goodies.

I wish that you could pass tables (i.e. resultsets) around as parameters to functions and stored procedures. T-SQL 2000 allows you to declare a table variable but doesn't allow you to pass that variable outside of the confines of the current procedure. Really effects modularization when the main data type of a relational database can't be passed around.

Other stuff comes to mind as well. But I'd settle for this one for now.

Ehud Lamm - Re: CLR Hosting  blueArrow
2/23/2004; 12:29:09 AM (reads: 468, responses: 1)
Sorry guys, I thought these posts were worth a look.

Andris Birkmanis - Re: CLR Hosting  blueArrow
2/23/2004; 12:54:35 AM (reads: 469, responses: 0)
Ehud: Sorry guys, I thought these posts were worth a look.

In fact, I enjoyed the out-of-the-trenches report of naive multithreading inadequacy to solve all concurrency problems. It would be interesting to compare different control handling schemes not only by expressiveness, but also by friendliness to the current mainstream hardware architectures. Anybody aware of such reports?

Neel Krishnaswami - Re: CLR Hosting  blueArrow
2/23/2004; 8:33:20 AM (reads: 405, responses: 0)
Ehud, there is no need to apologize: these posts ARE definitely worth a look. There's basically a countless number of possible theoretical approaches, and which paths we choose to go down should be informed by practical problems.

Ehud Lamm - Re: CLR Hosting  blueArrow
2/23/2004; 8:53:13 AM (reads: 401, responses: 0)
Let me try to explain why I think this issue is important. It's much wider that CLR, Microsoft or .Net.

One of the things I try to discuss here concern the use of language based approaches in software engineering. An important aspect, which may seem obvious to many here, is the notion of embedding DSLs, scripting languages and the like inside applications.

Quite often when you propose these approaches to programmers in industry you get one of two classic answers (sometimes both): (a) there is no need for specialized languages, either because they add no expressive power or because embedding is equivalent to writing library code, using DLLs etc. and (b) embedding is difficult and the rewards are slim.

The discussion of embedding the CLR is SQL Server, as opposed to just doing stored procedures in TSQL, can help clarify both issues. First, we see that a specialized language can help the surrounding application manage user code additions ("Any application written in TSQL is inherently scalable, fiber-aware, and robust in the face of resource errors.") This is far from obvious for most programmers I met.

Next, we see that embedding though sometimes quite easy (recall Lua, Rexx etc.) can create tricky problems. By attacking these problems inside the language processor or runtime we address the issues once and for all, and are able to manage the complexity. A classic example is handling security policy, eliminating the possibility that user contributions break security. Obviously, as in any other type of system building, we may decide that changes are required. Centralized handling of these issues will help with this problem as well.

Since I find these issues to be important, and all too often neglected, I think they are worth the attention of LtU. The CLR example is just that, an example. The issues are there whenever you consider language embedding, language coexistence, and DSL design.

Brent Fulgham - Re: CLR Hosting  blueArrow
2/23/2004; 2:56:56 PM (reads: 364, responses: 0)
I found this on the web: From Why's (Poignant) Guide to C#:

This little volume is about a language called C#. Think of the C as a Community of little programmers, producing amazing constructs of fabulous complexity and beauty. Think of the '#' as a giant hammer, being used by Microsoft to smash those programmers' prior art into smithereens. You could call it, "C-Smithereen" if you wanted to.

The '#' is like the Borg, from that old Sci Fi TV show called Star Trek. Imagine the # as a borg starship latching onto a semicircular planetoid full of craftsmen and engineers. The # assimilates the work of the craftsman, leaving no attributions behind.

"YOU ARE ALL PART OF THE BORG" the # bellows happily. "NOW REJOICE THAT WE SHARE OUR BOUNTIFUL CREATIONS WITH YOU."

Now weep quiety.

xeo_at_thermopylae - Re: CLR Hosting  blueArrow
2/23/2004; 3:32:21 PM (reads: 357, responses: 0)
Chris Rathman wrote on the immortal scroll:
I'll say flat out that Microsoft needs to put more effort in improving T-SQL not by passing the problem off to a JVM. There's a host of things that could be done to improve the language, and from what I see, they are not doing anything but placing a bandaid on the language - instead chasing the latest marketing push of xml or .net goodies.

Amen, brother. But Microsoft didn't write the core of SQL Server, Sybase did, and Microsoft still doesn't understand it. If you don't understand something completely you cannot safely extend it.

It is also impossible to acquire a technology if you don't understand it. Consider Microsoft's collaboration with Unisys on multiprocessing technology: Unisys got everything they wanted but Microsoft didn't get even what they needed.

I suppose the good news is that money doesn't buy everything and that in business as well as education "there is no royal road to mathematics".

GordonWeakliem - Re: CLR Hosting  blueArrow
2/23/2004; 3:52:09 PM (reads: 347, responses: 0)
One reason that MS doesn't talk much about others experiences is that there's not much published on that. I doubt that Oracle engineers spend much time exchanging technical tips with MS engineers. So I doubt that Chris could have much to say about Oracle's experiences, Also, culturally MS seems to be sort of insular and through NIH or just being under deadline pressure, they don't always spend a lot of time on prior art. Personally, I think that it's awesome that people like Chris Brumme are peeling back the covers on this stuff. I don't know of any similar quality resources on the JVM, for instance.

Franck Arnaud - Re: CLR Hosting  blueArrow
2/23/2004; 5:22:53 PM (reads: 337, responses: 2)

Ehud, these posts are definitely interesting. Something can be both interesting and sad. In my own comment, I was only whining about the somewhat disappointing state of the art of concurrency handling in such mainstream systems, in the sense that they do not manage the complexity as we wish they would.

programmers in industry [say] a) there is no need for specialized languages, either because they add no expressive power or because embedding is equivalent to writing library code, using DLLs etc. and

A reduction in expressive power is a very desirable feature for a DSL, it helps ensuring that it remains applied to its domain -- that good modularity is enforced. It may become a problem if the DSL is frozen and the domain is not, but on the other hand DSLs that expose their general purpose host language may be fragile in that respect.

(b) embedding is difficult and the rewards are slim.

Useful DSLs may not be embedded at all, when all the DSL code is compiled into the deliverable software for instance. Perhaps strangely, this is the kind of DSL I usually think about, although of course DSLs executed within the application are certainly also essential for some uses.

Ehud Lamm - Re: CLR Hosting  blueArrow
2/23/2004; 10:44:06 PM (reads: 324, responses: 0)
A reduction in expressive power is a very desirable feature for a DSL, it helps ensuring that it remains applied to its domain

Quite right. Of course, to mind, this can also be regarded as added expressive power...

Ehud Lamm - Re: CLR Hosting  blueArrow
2/23/2004; 11:01:31 PM (reads: 324, responses: 0)
Useful DSLs may not be embedded at all

Of course. Embedding is important when you want the ability to script applications on the fly. When embedding isn't needed (or can be avoided), there are good reasons not to embed.

Mark Evans - Re: CLR Hosting  blueArrow
2/25/2004; 2:48:05 PM (reads: 183, responses: 2)

Now weep quietly.

Indeed. Microsoft is always suspect as a case study in language design and implementation, for example:

"As poignantly described in Microsoft's pricing strategies for its upcoming VJ++ 6.0 Java development suite, the 'strategic objective' of its new toolkit is to 'eliminate/contain cross-platform Java by growing the polluted Java market,' 'migrate and lock Java developers to Win32 Java,' and ultimately 'kill cross-platform Java by grow[ing] the polluted Java market.'" The pricing strategy was described in an internal slide presentation at Microsoft...

With due respect to LtU's DSL interest, always remember this caveat: the Microsoft way isn't always the right way, or even a half-baked try at it.

Andris Birkmanis - Re: CLR Hosting  blueArrow
2/26/2004; 1:28:35 AM (reads: 162, responses: 1)
the Microsoft way isn't always the right way, or even a half-baked try at it.

Yes, some initiatives of corporations do not pursue better sales through technical domination, but something else (better sales through hype/FUD, lowering sales for competitors, strategically growing/shaping market for the future, etc.).

But does it mean their solutions are always technically crippled?

Ehud Lamm - Re: CLR Hosting  blueArrow
2/26/2004; 1:33:28 AM (reads: 163, responses: 0)
My basic assumption is that LtU readers are both highly critical and well educated in the field, so I don't see any harm in discussing MS technologies. The key word here is discussing.

I am fully aware that many people distrust and even hate MS, and I am not a fan of them myself: but I don't see why that should make technical discussions impossible.

Patrick Logan - Re: CLR Hosting  blueArrow
2/26/2004; 8:26:37 AM (reads: 150, responses: 0)
But does it mean their solutions are always technically crippled?

Not necessarily. The industry is a combination of business, psychology, sociology, and technology.

I think it helps to just bear that in mind, try to understand the distinctions, and how they influence ourselves and the community at large.

With or without Microsoft, or some other dominant corporation in its place (you know there *would* be one), our discussions here could *never* be purely technical, because there rarely is such a thing as a "purely technical conversation".

The only examples I can think of is when the discussion is about a formal syntax or semantics definition. Most discussions are not about that.

Mark Evans - Re: CLR Hosting  blueArrow
2/28/2004; 1:56:23 PM (reads: 100, responses: 0)

My statements about Microsoft carry factual citations in an effort to inform. The attitude is not disembodied "distrust" or "hate" but factual investigation. We should all remember that MSFT was/is under legal investigation by the US feds, the US states, the European Union, and Japan. Cited above is an example of MSFT marketing considerations crippling a technical design on purpose. It comes from a lawsuit against MSFT. So LtU discussions of MSFT products should keep a wary eye on monopoly tactics that impact technical designs. My impression (which could be wrong) is that LtU discussions are very deficient on that count, even though market control is the main driver of all MSFT technical activity - trumping every other technical consideration by leaps and bounds, as witness the citation above. We don't have to become Slashdot to discuss how the salient facts of monopoly life affect MSFT technical designs. MSFT technical designs emerge from meetings like the one quoted.

The statement that MSFT "solutions are always technically crippled" was never made, so it's a bit of a strawman. Come to that though, they are always crippled on non-Windows platforms - in most cases existentially. What I said is that Microsoft is always suspect, which is not to say always guilty.

Getting back to the subject matter: Maybe the problem lies in the closed-world assumptions of the languages themselves. So the remedy is not a CLR with attendant problems of vendor lock-in and lowest-common-denominator functionality, but better language design.

Mark Evans - Re: CLR Hosting  blueArrow
3/1/2004; 1:25:06 PM (reads: 88, responses: 0)
Sometimes MSFT behavior is comical (yet still scary).

Mark Evans - Re: CLR Hosting  blueArrow
3/5/2004; 3:44:53 PM (reads: 62, responses: 0)
Another day, another Microsoft market innovation.