Lambda the Ultimate

I always run into the problem, because of the world where I am paid to provide solutions, that none of these can be used to build great GUIs nor can they deal with RDBMSs very well. Thus for all their potential, they do not do well in the "typical corporate solutions" environment.

Or is the system only compatible in the sense that all languages are translated to a common intermediate language, a la CLI?

Even in this case, there are interesting questions, concerning for example type safety and run-time type checks between interlanguage procedure calls.. How are these handled? I suppose I'll have to dig the documentation :)

(update: Ok, it's through an intermediate language. But I still can't seem to find information about how they handle the typing issues in the VM. In other words, how is Poplog significantly more interesting than Scheme on the JVM or F#...?)

http://www.cs.bham.ac.uk/research/poplog/doc/pmlhelp/mlinpop

However it's been many years since I used Poplog seriously so things may well have changed.

Whether it's any more/less interesting I don't know. Its certainly an environment I enjoyed playing with a great deal when I was using it (disclaimer: Many moons ago I used to work on the Poplog development team :-)

The incremental compilation bit would be one thing I've not seen with something like Scheme on JVM, F# and friends. You've got an environment much more like Smalltalk where you're continually compiling bits into an image in various different languages. Poplog users tend to "live" in Poplog much the same way Smalltalk and EMacs users do.

Anyway, I'm reminded of Sjoerd Visscher's comments about the standard library in multi-paradigm environments. He claimed that the style in which the standard library is written is inevitably the one in which programming will be most natural. For example, most Schemes support OO of some form, but the libraries (say, the SRFI's) are functional-bent, and so OO is unnatural. I guess it's the same with Smalltalk: Smalltalk has closures, hence functional programming is possible, but not necessarily natural. What are your thoughts about that? Since you've worked on the dev team, you have a pretty valuable point-of-view there.

PS: I most definitely do not want to tackle issues like "Ok, define natural, then": my best answer would be that I have programmed in languages like, Scheme and Smalltalk, and I feel the impedance mismatch, so to speak. It may be just my incompetence, though :)

[edit: that was unclear. typed is a subset of untyped, but with a good type system the difference should be small; for a sufficiently expressive type system you could argue that any difference is an error in the untyped code but that's a confusing side issue that i should have avoided. in any case, the conflict between using untyped and typed is of a different type - some cases are excluded from a common core - to that between oo and functional - where there is no common ground. hence the earlier discussion about ocaml where paul said no-one used the oo features...]

Alex: I always run into the problem, because of the world where I am paid to provide solutions, that none of these can be used to build great GUIs nor can they deal with RDBMSs very well. Thus for all their potential, they do not do well in the "typical corporate solutions" environment.

It's funny that you mention Oz in this context. Here are some database interfaces for Oz:

<http://www.mozart-oz.org/lists/oz-users/5184.html> (MySQL)
<http://www.mozart-oz.org/lists/oz-users/5183.html> (PostgreSQL)
<http://www.mozart-oz.org/mogul/info/mahmoud/oztclodbc.html> (Anything with an ODBC driver>

Oz also has very nice GUI-building facilities:

<http://www.mozart-oz.org/documentation/mozart-stdlib/wp/qtk/html> (Using Tk)
<http://www.mozart-oz.org/download/view.cgi?action=rpm> (mozart-gtk and gtk-canvas, also available for Mac OS X)

If that's not enough, Oz is comparatively straightforward to extend in C or C++:

<http://www.mozart-oz.org/documentation/foreign>

Basically, Oz seems like an excellent counterexample to your point.

Thanks for the references.

I am familiar with the Oz facilities for GUI and that is why I added the qualifier "great". You cannot use the commercial components (ActiveX or .NET), only the simpler Tcl/Tk widgets.

I am also familiar with the Oz ODBC connectivity. Again I added the qualifier "...very well". ODBC is useful, but pretty low level.

As for "interfacing with C/C++", again this is pretty low productivity stuff. I prefer COM or .NET for better leverage of other people's work.

I'm saying this as someone who wishes the very best for Oz (and other modern languages for that matter). I don't find the counterexamples you offered very compelling.

Of course there is growing adoption of MySql and Postgresql, but I think it is still safe to say that these are not widely used in the (as Alex put it) "typical corporate solutions" environment. It's great to have interfaces to these databases, but until you support MSSQL or Oracle (preferably both), you're not supporting the majority, IMHO.

I know the ODBC interface is intended to do this, and it's a good idea - you kill a lot of birds with one stone by getting an ODBC interface. Unfortunately this one requires a external product in another language (tcl) to work. This raises doubts in my head as to performance. ODBC rarely performs as well as the native interface (unless you're using MSSQL, of course) to begin with, so that's two doubts I have about performance now.

Since the link to that product from the Mogul entry you provided is broken, I went to Google, which took me to a sourceforge site that hasn't been updated in 3 years. If I'm a potential corporate user, than at this point, the purported ODBC interface goes down in my book as "does not exist/unmaintained".

As for user interfaces, Oz is certainly providing usable alternatives, but not compelling ones. Anything that can't look native (and look nice) on a Windows machine is probably going to be a tough sell. Tk widgets tend to lag well behind the times wrt. native look and feel on Windows, in my experience. Maybe that's just my predjudice, but when I see "via Tk" in a GUI library description, I think "Ugh!" and move on. As for Gtk, it's great for Linux, but terrible for Windows.

A good C FFI is definitely a requirement, but only so I know that other people with the time to write extensions can write them well. I definitely don't have time to write bindings to complicated C/C++ libraries, and the complicated libraries are probably the ones I need.

I can't speak for Alex (or anyone else), but in my view the libraries around Oz (and many others - not just Oz!) need to develop a bit more before I'd consider them ready for the "typical corporate solution" market. Some folks may be looking for a Linux/Gtk/MySql sort of "corporate solution", and it looks like Oz would fit the bill there. But from where I'm sitting, "typical corporate solution" means Windows, native look and feel, and either Oracle or MSSQL. Or, it means modern web browser, web server, and Oracle/MSSQL. But that's a whole different story.

I don't mean to hurt anyone's feelings, and I wish Oz all the best. I just want a realistic appreciation of the problem.

Carlos remarks "Ok, it's through an intermediate language" but this is very much a half-truth. Yes, all four languages are implemented through a single virtual machine (which produces native machine code) and, if you squint a bit you could see that as an intermediate language. Sort of. The core language is Pop-11, in the sense that the virtual machine is most closely related to its high-level concepts, and the other three languages (Prolog, Lisp, SML) are called subsystems.

However, that's not the level that programmers work at when they are doing a mixed language project. Each subsystem is expected to provide facilities for working with the core language, Pop-11. One subsystem pair (i.e. Prolog & Lisp) also provides their own direct specialized facilities. The best developed interface is between Pop-11 and Prolog; that's actually a pleasure to work with.

Carlos asks a more specific question: ... there are interesting questions, concerning for example type safety and run-time type checks between interlanguage procedure calls.. How are these handled? I suppose I'll have to dig the documentation :)

The answer depends on the mixed-language facilities provided by the programmer. There are no general rules. Unfortunately SML is less well integrated than either Prolog or CLISP and so this issue is not dealt with in the "obvious way" - I'll explain what I think that is below. Instead the interface is really aimed at letting system programmers extend the SML subsystem using Pop-11 without compromising efficiency. Which it does nicely but is of little interest to Carlos.

So, in the interesting case of calling an SML function from a Pop-11 procedure, alas, the onus on the programmer. Basically, from Pop-11 you use ml_valof( NAME ) to get hold of the function and then you call it, making sure that you have got the type of the argument correct. e.g.

ml_valof( NAME )( ARGS_MUST_BE_RIGHT ) This rather unsatisfactory interface is really just a reflection of the immaturity of the SML subsystem. There is a distinct pecking order of maturity of the subsystems: Prolog, CLISP and SML. The "obvious" interface would be to arrange for a dynamic type-check of the arguments. You would use something like

ml_typechecked_valof( NAME )( ARGS ) From what I remember of the code of the SML subsystem, this would be quite a middling kind of effort - a week or two of an enthusiastic programmers time.

In my experience, Poplog's multi-language integration is very successful. You can write entire projects in Prolog, very comfortably, thinking in Prolog all the way and then add just a little Pop-11 to do (say) multi-threading of complex queries. Or you can take Common Lisp library and then juice it up with some Prolog. It feels completely natural. I've worked with teams of Prolog programmers, who found Poplog to be a congenial environment, to add in GUI integration using Pop-11, for example.

The key, it seems to me, is that the mixed language facilities have been worked and reworked many times to avoid the nightmare situation where using one small part of the "standard library" requires pulling in vast amounts of code. If you find that you Prolog library is missing a bunch of predicates for (say) doing database calls, but Pop-11 has an ODBC library, then you can just add the predicates as a separate library _or_ you can just call the procedures you need. It isn't difficult. So as a programmer you feel in control.

I asked about the dynamically-typed to statically-typed integration because that's something I have never seen before. As you say, the SML integration falls behind the other languages'. This is a good thing to hear, because I misunderstood the documentation to mean that the 'dynamically-typed calls statically-typed' situation was inherently dangerous.

Sorry for the misunderstanding.

So "great" is defined as "Win32?" I guess the answer there is all going to revolve around how you extend Oz to support .NET, then. The point was that in response to your general assertion, there is a counter-example, and I think the counter-example stands: you can take advantage of the work surrounding QTK and the concept of declarative layout and apply it to the layout of visual elements in .NET if you wish to.

Alex: I am also familiar with the Oz ODBC connectivity. Again I added the qualifier "...very well". ODBC is useful, but pretty low level.

Compared to what? "Dealing with RDBMSes very well" typically means "connecting to them and allowing me to pump SQL to them." There are other desirable things to be able to do, of course, but it's not clear that they have much to do with dealing with relational databases. For example, serializing language values: Oz does that well, too. It might be a nice thing to be able to serialize language values to a relational database table, but that has little to do with interacting with relational databases well. A better goal there would be to ensure that there was a trivially simple mapping from the relational features of the language to relational databases. But as other languages are no better at this (most don't even have "relational features" as Oz does), I have difficulty believing that this is what you meant.

Alex: As for "interfacing with C/C++", again this is pretty low productivity stuff. I prefer COM or .NET for better leverage of other people's work.

It depends upon what your goals are. Since one of the Oz teams' goals is portability, I find their interfacing with C/C++ to be much higher productivity than COM or .NET would be, since I'm not developing on Windows. Would there be value in providing COM or .NET bindings for Oz? Most definitely; it's just not clear that it's a failure of the Oz developers to provide them, nor does it tell us anything useful about the language or its libraries to observe that it lacks them.

But it's obviously valid to observe that that's the nature of some corporate environments, so it's a good question as to how much effort would be involved in providing, say, Mozart-Oz 1.3.0 bindings to .NET, and perhaps some nice modules to address specific application domains such as pickling to and from databases or an equivalent to QTK around .NET widgets. I'd applaud an effort to provide an answer to such a question. But if no one who does Windows development makes the effort to provide those bindings, what should we say about that? I don't know what the answer is, but I do know that it isn't "Oz can't be used to build great GUIs nor can it deal with RDBMSs very well."

I think it's fair to say that these things are desirable, but unfair to criticize the Oz team for not providing them. What they do provide is some nice, portable infrastrucure and an incredibly capable language. What it sounds like is needed is for a Windows developer to step up to the plate and take advantage of Oz's C/C++ integration capability to provide bindings to .NET and write some nice modules. As I wrote to Alex, it's a good observation; it just doesn't say anything at all about how good or bad Oz is for mainstream business application development.

ok, i'm confused. as you observe (i think) oz isn't tightly integrated with win32. in a lot of people's books, that has to mean it's not suitable for "mainstream business application development".

this isn't a question of "criticiz[ing] the oz team", but a simple observation that applies to a whole pile of open source languages, for understandable reasons.

i think it's possible to recognise how cool oz it, see that it *could* be integrated more closely with Win32, *and* find it not suitable for business apps until someone does that integration.

Certainly, and once it became clear that that was the yardstick, I think I offered a reasonable response. But lest we forget, here's what Alex wrote:

Alex: I always run into the problem, because of the world where I am paid to provide solutions, that none of these can be used to build great GUIs nor can they deal with RDBMSs very well.

"None of these can be used," not "They tend not to have tight Windows integration out of the box." And I didn't see anything at all about the pluses or minuses of the language that might or might not motivate someone to provide the bindings being requested, or any acknowledgement that a savvy Windows C/C++ programmer could address the issue using Oz's C/C++ integration. These kinds of critiques strike me as adding up to "Tool X isn't Visual Studio .NET and its frameworks," which, to my way of thinking, has the simultaneous disadvantages of being both unfair and uninteresting.

Again, just for the sake of crystalline clarity: I do see the value in such bindings, most definitely. And I'm convinced that if such bindings for Windows technologies existed, Windows developers would find Oz a very powerful tool.

With that thought firmly in mind, then, perhaps folks interested in multiparadigm languages with excellent Windows integration should look at <http://tech.motion-twin.com/ocamole>, <http://caml.inria.fr/camlidl>, <http://tech.motion-twin.com/?&node=41>, and <http://tech.motion-twin.com/?&node=32>. As most Lambda readers probably know, O'Caml integrates functional, imperative, and object-oriented programming, and seems to be well-supported by numerous libraries.

It is only a criticism in the general sense. There are some wonderful languages out there, Oz included, that are not productive in the world of "typical corporate applications".

Take the GUI issue. Perhaps you are not familiar with the richness and breadth of available components? Take a look at http://www.componentsource.com/. Windows is on most desktops (about 93% by last measures) and definitely in the corporate world. The users demand GUIs comparable to commercial apps. And to do that in QTK is, shall we say. less productive.

On the RDBMS front, just see how much code you write on ODBC to do a parameterized query (and put the results into an editbale grid (table)). Now do that with ADO, or ADO.NET and bind to an off-the-shelf grid (or if you like an edit form). You will immediately be struck by the difference in productivity.

So I want to build better user interactions, faster. That's all.

P.S.: If the PL designers are really interested in GUI toolkits, I'd suggest taking a hard look at wxPython as a better example of things done right.

I am aware, and my point is that you're conflating two things: ability to support native UI controls, and the mechanisms by which effective UIs are assembled and presented. In other words, the issue with QTK isn't the Q, it's the Tk. That's fair. What remains is for someone who's familiar with the Windows APIs to do QWin32 or something like it.

Alex: On the RDBMS front, just see how much code you write on ODBC to do a parameterized query (and put the results into an editbale grid (table)). Now do that with ADO, or ADO.NET and bind to an off-the-shelf grid (or if you like an edit form). You will immediately be struck by the difference in productivity.

Again, I'm very much aware, and this is the type of thing that I was driving at when I mentioned how nice it would be to have an effective mapping of Oz's relational computing features to RDBMSes. That, coupled with effective means of interface building, would provide the level of integration that you see with ADO[.NET] or more.

In short, you continue to make my point for me: you tell me how Microsoft does it and point out that no one has done the same level of Win32 integration in these non-Microsoft products. Stipulated. It still doesn't tell us anything useful about what these better (than C, C++, C#, or Visual Basic, taken as languages apart from their deep embedding with Win32, that is) non-Microsoft tools can or can't do in the hands of someone familiar with the desired integration context. In other words, there's nothing magic about Visual Studio .NET and its languages and frameworks. The same things can be done using these other tools, and probably more easily, relative to the source code for Microsoft's tools. It's just that no one has yet in Oz's case. I remain curious, though, as to Windows' developers thoughts on some of the integration that's been done with Objective Caml, another quite nice multiparadigm language, which I referred to earlier.

Let me be clear: I'm not looking for Visual Studio or its relatives, and I doubt Alex is either. I can get along perfectly well with just an Emacs mode, and if the language is smart enough, I'll choose the language over the tools (debugger, IDE, etc.) any day. But there are some libraries I need, and don't have time to write. Any language that doesn't provide these can't be used in my work. It's no criticism of the language, but until those libraries exist, I can't use the language for work.

I don't particularly care about integration with .Net or ActiveX, though those are certainly nice. What I need is a GUI that looks attractive and native, and database access that is both easy and efficient, and supports the DBMS that I want to use. Oh, and internationalization (Unicode, etc.) support. Nobody's conflating libraries and language - we all (I think?) freely admit that (modulo opinions about static typing, VM languages, etc.) Oz would make a fine language for typical corporate solutions.

The question is, given that the language is spiffy, can it actually be used in the corporate arena? Does it have the right libraries? Wishing that advanced language implementations would provide these things isn't saying that language designers aren't doing their jobs, it's a hint that wider adoption might be spurred by actually providing the tools people need.

Oz-the-language is great. But it's not enough. And that's a shame. That's all. It's not just Oz's problem - lots of advanced languages suffer the same troubles. It's also not a death sentence - these things improve over time! I think Alex was just lamenting the fact that making the libraries that corporate developers need seems to come long, long after the introduction of the compiler. You see a great language, think "Gee, I wish I could use that <sigh>," and then you wait a few years to see if it ever grows to meet your needs. As language enthusiasts, we want to use the best available language. But if the best language doesn't support our needs, we have to use the second (or third, fourth, ...) best language instead. But we're not happy about it.

So we make complaints like Alex's: "I always run into the problem, because of the world where I am paid to provide solutions, that none of these can be used to build great GUIs nor can they deal with RDBMSs very well. Thus for all their potential, they do not do well in the "typical corporate solutions" environment."

You state the issues far clearer than I can.

- We have one funded person, David Bouvy, who is working on an Oz port for .Net (in concert with the CETIC lab and a company called Bizzdev). We also have some volunteers who expressed interest in making a Windows-like IDE. Basically, we rely a lot on volunteers to do the extra stuff that we don't have time to do as researchers (anybody is welcome to contribute).

- Regarding the DB interfaces: another point to make w.r.t. to the points already made is that for many small DB applications (up to, say, 100,000 tuples) the symbolic computation abilities of Oz make an external DB interface unnecessary. One possible future route would be to go the way of Erlang: build an advanced DB inside the language (like Erlang's Mnesia), using the transparent distribution ability of Oz when needed.

- Donatien Grolaux tried a couple of years ago to add a 'Q' to Dynamic HTML, so that any Web browser could be used as a GUI toolkit. It worked, up to a point, but fell down for two reasons: (1) Web browsers did not implement the Dynamic HTML standard (the best was IE, but even it documented stuff that was not implemented), and (2) GUI performance was terrible. Evidently, we were pushing the GUI toolkit aspect too hard (the "suffocating the gerbils" problem). Maybe we will try this again some day. (BTW, there is a Gtk interface, but without a 'Q'.)

- A final point w.r.t. Oz, Prolog, and multiparadigm programming. The CLAES laboratory (Central Laboratory for Agricultural Expert Systems) in Cairo, Egypt specializes in making expert systems for agricultural purposes. They mostly develop them in Prolog (SICStus I believe). Translating one of their big expert systems to Mozart resulted in big gains in performance, mainly because Oz lets you use the right concepts in the right place (e.g., using dictionaries instead of Prolog's in-memory database).

i'm probably saying the obvious here, but i'm in a job where i write fortran 66 much of the time and there are major internal political battles over moving to java (why bother when fortran 66 is turing complete?). maybe that kind of conservatism is extreme, but i'm pretty sure that sql would be an easier sell than an internal database for the mass market.

i'm not saying it's better, just that anything familiar is more likely to be accepted.

which raises a related point - is there any value at all in trying to target a language like oz at the commercial market? yes, there are a few people who might use it, but i would guess the chances of oz being the next java/visual basic are tiny, whatever you do. ocaml is perhaps the biggest commercial success story (and paul has made it pretty clear that it has the integration :o), but even that is insignificant.

maybe it is better to spend resources developing an "academic" language in the hope that some of the features of oz will gain credibility in academic circles, trickle down through engineering/compsci students, and eventually be picked up by the next bill joy.

is that what you (peter) have concluded? or is the direction pretty much decided by external factors (like what you can get grants for)?

(ps before anyone points out that fortran 66 ain't that bad, it's used with a hack that involves the c library memory management, so we have all the memory problems you get with c programs too - memory management and integration with 3rd party products are the main reasons for moving to java).

is that what you (peter) have concluded? or is the direction pretty much decided by external factors (like what you can get grants for)?

Personally, I am interested mostly by language expressiveness and distributed programming. I am not so interested in commercial acceptance. To do forward-looking research, I strongly believe one should do what is right in an absolute sense and not care about the commercial side! For example, Oz resurrects an idea which was discarded by the mainstream, namely transparent distribution. The mainstream is correct, if one is limited to using a mainstream language. But if part of the research is to design the language, then transparent distribution actually works and is a much better solution than the usual distributed objects like they are implemented by Java or Corba (see chapter 11 of CTM, in particular the introduction).

But most Mozart researchers are also interested in system building: the proof of an idea is that it works in practice. (And we are sticklers for making real systems, not just toy prototypes!) So potentially, there is always the possibility of using our work in a commercial setting. The distance is not so great that an enthusiastic user could bridge it somehow. We encourage this kind of leakage, and we help where we can.

Regarding funding, so far we have been lucky in being able to piggyback Mozart development on top of funded research.

That's not a bad way to go in some cases, but in many corporate apps (as Andrew pointed out) there is more of a need for standardization of the database. Many companies will expect to be able to use Crystal Reports, for instance, and a report engine is high on my list of things I don't have time to write. Also, many companies tend to be more conservative with their DBMS than with anything else, as well they should be - their data is an important asset. If you could provide an ODBC driver, a JDBC driver, SQL-92 support and be significantly faster than Oracle/MSSQL (or offer compelling features that those don't have) you might have a shot at convincing people to use it.

Without all those (or a large marketing budget), it's not likely to sell well. It can be done, though - the K language supports its own database, kdb, which is (allegedly, I've not tested it myself) significantly faster than mainstream DBMSs and seems quite popular among large finiancial firms. In general however, I think that most research languages are better off integrating with existing DBMS products.

To do forward-looking research, I strongly believe one should do what is right in an absolute sense and not care about the commercial side!

I agree, please don't stop!

Andrew: which raises a related point - is there any value at all in trying to target a language like oz at the commercial market? yes, there are a few people who might use it, but i would guess the chances of oz being the next java/visual basic are tiny, whatever you do. ocaml is perhaps the biggest commercial success story (and paul has made it pretty clear that it has the integration :o), but even that is insignificant.

That's a question I often think about as well. We should perhaps just be happy with Java and C#, knowing that great minds are working on languages that will eventually cause mainstream languages to grow. I have two problems with this view, though. One it personal - I just want to use great languages! Mediocrity in programming languages drives me crazy. The other is more practical - I think the right programming language can be a competitive advantage in the marketplace. To get a competitive advantage though, you do need some libraries, and a certain critical mass of users. Ocaml and (lately) Haskell seem to be getting close to this point now, so there's hope.

Of course, languages like Nice sidestep the library problem entirely, by providing seamless access to all of Java's libraries, but that is a double-edged sword too - when you don't control the VM your language targets, it limits the sorts of directions you can explore. I suppose that's why we don't have all these advanced languages running on the same VM.

Peter: Is what I wrote above really true, though? Or is there a realistic possibility of ever having a common VM for research languages? As a researcher, do you need to have intimate control of your VM? Alice and Oz both run on the same VM, could other languages run there as well? Why might they want to (or not)?. What about Oz on the (by most reports) blindingly fast Ocaml VM? What about a common bytecode, which could be run by different VMs that wanted to make different performance tradeoffs? Just trying to think of ways to increase the sharing between different research languages, so that every language implementation doesn't have to invent its own Gtk binding, sockets, etc., unless it has something new to offer there.

Paul: I think we are in vehement agreement Ah, good. :-) I'll take a look at the Ocaml/Windows integration stuff you mentioned, I'd not heard of the stuff at Motion Twin.

just noting old things here, so peter can give the interesting new stuff...

i understand that most vms are too specific to the language they support to be generally useful. both java and .net make broad assumptions about oo-repated functionality that makes some functional languages difficult (slower?) to implement, i believe. parrot might be better, although it's not yet obvious to cynical me that perl 6 is going to be a success.

aiming more directly at this problem is llvm - http://llvm.cs.uiuc.edu/ - and c-- - http://www.cminusminus.org/ - which have both been discussed here before (c-- isn't a vm, but solves (many of) the same underlying problems from the pov of the language implementor; it also has run-time bindings, unlike (i believe) llvm, so you get more control over gc; in contrast llvm has more bells + whistles).

As for Gtk, it's great for Linux, but terrible for Windows.

GTK on Windows can use the GTK Windows Impersonator, the Mac OS X Aqua theme, or the WinXP theme.

-Chris

Hey, that's great! It looks a lot better than it did a year or two ago. Progress! Just out of curiousity, do any of them respect the user's font and color settings and so on?

Thanks for those.

Peter, is porting the whole Oz system to .NET the only way to make it play nice with .NET? By definition I would not call that an "integration", more a port of Oz to a different platform.