Lambda the Ultimate

Some background to my question: I run a small bespoke software business, most of whose work is in Pascal and C++, with the Pascal automatically translated into C++ (using little beyond ANSI C actually). We are reaching a crisis of scale and I am looking around for something better for the larger and more critical systems we need to produce. Eiffel and Ada are the two main contenders because they have the commercial support that we want. The main drawback to Sather is that it looks moribund and may go away in a few years. Another nice point to Eiffel and Ada is that it would be quite easy to move the Pascal code over automatically (I've already started playing with pas2eiffel.pl), but that's something Eli could tackle, I'm sure.

Ada looks OK but its 95 incarnation has the OO bits rather bolted-on (the 83 incarnation does not have any form of closures at all, as far as I can tell, thus disqualifying it from consideration) and Eiffel is, well, OK, but if it's so good then why is EiffelStudio so unreliable ?

We need to be able to produce MS-Windows 32 applications, as well access to MS-SQL, Postgres and (later) one or another GNU/Linux based GUI.

This is a rather unusual question for LtU as it comes from a commercial rather than academic point of view, but I am convinced, from our increasing struggles with the Pascal|C++ leviathon, that a rigorous academic approach to the problem will yield the best commercial result in the long term.

I have no real knowledge of Sather, so I'll just breifly comment on what you say about Ada.

It takes a bit of getting used to the way inheritance is handled in Ada95 -- no denying that. But it is definitely not "bolted-on". In fact, I think that aside from the lack of "interfaces" (something that will be fixed soon, I hope) the Ada model is cleaner than the C++/Java approach. The main difference is that modularity and inheritance are handled by seperate language features, instead of haing a class construct which is used for both purposes.

There are plenty of messages in the comp.lang.ada archives about this (and you may also want to check the Ada95 Language Study Notes).

I suggest you raise your concerns on comp.lang.ada. It is usually a vrey helpful newsgroup (esp. if you don't troll them about the "bolted-on" thing...)

We are reaching a crisis of scale and I am looking around for something better for the larger and more critical systems we need to produce

Why are you finding C++ unacceptable in large-scale projects? Seems to me C++ was more-or-less designed with large-scale projects in mind, and indeed quit a few such projects exist. In addition, on the Windows platform there are some very nice IDEs for it, and lots of library code. Where is C++ failing you? (I am aware of some of the limitations of C++, in fact I have mentioned several such limitations myself in previous posts. I am wondering though where you find yourself blocked.)

with the Pascal automatically translated into C++

I am assuming that the Pascal code is mostly legacy. Otherwise, why do you go through this translation process? Also, how do you debug, or do you only rely on trace statements and such? Why haven't you considered Delphi?

Eiffel and Ada are the two main contenders because they have the commercial support that we want

You haven't considered Java or C# at all? Again, I am aware of at least some of the limitations of these languages, but am wondering why you rejected them.

Bottom line is, you seem to be putting a lot of thought into this, for obvious reasons. But I'm not sure what your criteria of selecting/rejecting a programming language are. I would certainly like to know, as I occasionally need to make similar decisions myself.

I mean that standing back and looking at what I do, a lot of the time I am faffing around with issues that I should not need to be bothered-with. There are just so many little examples over the years: 1. Memory-management; looking after objects; who's going to free them ? 2. The confusion and mistakes caused by automatic promotion of Booleans and characters to integers. One particularly stupid example was an overloaded name with a combination of Boolean and integral parameters, some of them optional, where different compilers resolved calls to different methods. I don't use optional parameters now. 3. Missing a `break' off a switch branch. 4. Preprocessor tomfoolery behind my back; what you see is not necessarily what you get. 5. Lack of numerical sub-typing (bounds and checks; Ada is strong on this: Eiffel is not). Bounds checks on subranges have caught many problems over the years on the Pascal code, but C++ just waits for you to use the out of range number. Yes, I know you can create classes to do all this, but it seems so basic to me, it should be provided (as it is with Ada).

<< Why are you finding C++ unacceptable in large-scale projects? Seems to me C++ was more-or-less designed with large-scale projects in mind, and indeed quite a few such projects exist.>>

It's the combination of automatic promotions and cantankerous syntax (shorter is not necessarily better). Large projects ? Well, yes, Solaris and MS-Windows are written in C, after all,n'est-ce pas ? But look at the pace of evolution ! and in the case of MS-Windows, their unreliability ! The only real large-scale feature is that each class can be separately-compiled, but it's that feature that leads to the fragile base class problem, because the virtualness of each method is determined (in isolation) from the declaration of the class alone. I've read (most of) John Lakos' book and in my opinion it provides the best argument against using C++ for large projects, in that it really just says use the tired old hacks more rigorously: #define guard code, one .C file per class etc. I have built what I think are slightly more imaginative solutions myself, but there is no perfect system. One innovation is that I've dispensed with #include in source code entirely, and I use a formal comment markup system to enable the automatic generation of a list of required #includes for each source. I've also worked as a contractor on larger C++ systems, and the problems are the same there, including half-day recompilations. Stupid. I now have about 140,000 lines to maintain. I just find that the system is fragile. Maybe I'm a bad workman blaming his tools, but I never had the problem when the system was Pascal-only.

<< In addition, on the Windows platform there are some very nice IDEs for it, >>

All the source is on Solaris, I won't trust a wintel box any further than I can spit. I also chop and change environments; currently I'm on KDE 3.0.3 + Kate + SCCS (yes, SCCS).

<< (I am aware of some of the limitations of C++, in fact I have mentioned several such limitations myself in previous posts. [...]) >>

Who isn't ? and still the language marches on.

<< I am assuming that the Pascal code is mostly legacy. Otherwise, why do you go through this translation process? >>

No, the Pascal is still compiled by a Pascal compiler for DOS-based customers (yes, really), and translated to C++ for the MS-Windows sites.

<< Also, how do you debug, or do you only rely on trace statements and such? >>

Yes. Trace statements.

<< Why haven't you considered Delphi? >>

Proprietary lock-in.

<< You haven't considered Java or C# at all? Again, I am aware of at least some of the limitations of these languages, but am wondering why you rejected them. >>

Java repeats the screwed-up syntax of C and C++ (WHY ? FOR GOODNESS' SAKE WHY ? Sorry for the strong language, utterly inappropriate to a technical forum, but I just cannot understand what was going through Gosling's head when he thought of reusing all those braces and punctuation marks, as well as the fall-through on the switch statement).

C#: Proprietary. No support on Linux, Solaris etc.

<< Bottom line is, you seem to be putting a lot of thought into this, for obvious reasons. But I'm not sure what your criteria of selecting/rejecting a programming language are. I would certainly like to know, as I occasionally need to make similar decisions myself. >>

I can't really put a finger on it, but I think the best summary is that I want a language that allows you to specify more checks and constraints, so that you can say ``this is what should be happening at this point''. Ada provides them as very strong type checking, and Eiffel as pre- and post-conditions, but in C and C++, it's roll your own, which is never quite enough, and very hard work. Of course, I have a simple assertion checker which flags up an error if the assertion fails; but I have to remember to call it. The language could do that for me.

The American psychologist William James said that people are attracted to one religion or philosophy above another as much by aesthetics as anything, and I think it's the same over programming languages. I've been messing with microcomputers since 1979, and I think that my generation likes C and C++ exactly because they *are* low-level: we feel we can imagine what code the compiler will produce. I think that to get on and produce some useful work, we need to let that go.

I have no special knowledge about it, but AFAIK the Sather project is no longer active.

You say you are looking at Eiffel and Ada because they have commercial support, but since you asked about Sather and seem to have some confidence in languages which are based on sound principles I suppose you may be willing to look at Objective Caml (also see INRIA page). IMNSHO, it is currently the best programming language/implementation combination in the world. I'm not a fan of its object system, but you may find it a good way to get your foot in the door. Ocaml is an academic language with a very practical bent, a relatively powerful typing system, bytecode and native compiler implementations (both very fast), supporting both UNIX and Windows. The user community is active and growing, and the implementors are quick with support issues. I think you will not a more practical academic language.

If you're really serious about adopting well-designed languages, then you can also look at Haskell, though you will find the learning curve steeper than for Ocaml, and the implementations are not quite as good.

Dan: D could almost have been written to my specification. I agree with almost all of the additions-to and subtractions-from C++ that the language incorporates. I almost exactly fit the profile of the intended `who is it for ?' programmer. HOWEVER, if I'm going to change, it might as well be to a completely fresh language, as I don't really need to maintain any appearance of C. I was also rather disturbed to note that D maintains the fall-through in the switch branches (NOOOO!) and the cranky numerics of C. Oh well.

Frank: I hardly feel qualified to comment on OCAML, because I have little practical knowledge of functional languages and I only spent about an hour looking over the web site. It will be absolutely necessary to port the existing Pascal code base over; there's no way I'm rewriting that by hand, and a functional language is just so utterly different from an imperative language like Pascal that I don't know how the port could be automated. You might as well translate English to Japanese: the thought patterns are too different. So, interesting as OCAML is, I don't see how it could be used. That requirement to port the existing code base does unfortunately exclude some of the more interesting languages, including the entire functional family.

You might be interested in http://www.cs.berkeley.edu/~necula/cil. "[A] complete C parser (both for ANSI C and also the GCC and Microsoft Visual C extensions), pretty-printer and an intermediate language that hides most of the dark corners of the C language."

I don't see why it would be significantly more difficult to translate Pascal to Ocaml than, say, Pascal to C. It's true that a functional programmer structures his code differently than a procedural one, but if you're translating you don't need to restructure the code, just map to suitable concepts. And Ocaml has assignment, loops and mutable records, all the trappings of a procedural language. The result won't look like a functional programmer wrote it, of course, but that's not so relevant if you have a decent interface, and if not, you can wrap it in a more functional one.

When I made my posting of the 23rd, I didn't realise that Ocaml had all the imperative programming facilities as well. Obviously it's not Bertrand Meyer's baby.

I'm having another look at Ocaml. I've downloaded the prototype English translation of the O'Reilly book, as well as the 3.06 distribution (many thanks to whomever produced the Mandrake RPM) and will have a play over the weekend.

If I can automate the port from Pascal to Ocaml, this may very well be the way to go. I also see that it is possible to interface between Ocaml and C, but my experience of inter-language interfacing is that it is always a compromise and I would rather have a `pure' Ocaml system, even if most of it is ugly imperative code.

I also suspect that it would be much better to port directly from Pascal to Ocaml, as the semantics of C are slighty different; for example, one cannot have pointers to stack-based objects in Pascal. This minimises the possibility of subtle errors. I already have a Pascal to C translator so half the job is done.

Question for Frank: does Ocaml support a basic Win32 API ? I can't find any reference thereto in the documentation, or is this handled by the interface to C ? It's not an obvious question because the Win-32 functions are calls to DLLs not normal library calls, as I understand it. If I can't have CreateWindow, GetWindowWord etc. (or a Ocaml wrapper thereof), them Ocaml is unfortunately a non-starter.

Looking at O'Caml, it is quite elegant in a sense, in that it dispenses almost entirely with type declarations, however it is over-cleverness (doing things for you) of just that sort that has driven me away from C++. I quite like the way that Ada makes you write it all down; the source then becomes more self-documenting, and you know that what is written is what you get, no more and no less, with no promotions, implicit constructors, conversions, preprocessing and other cleverness getting in the way. I liked that in Sather, as well. However, again, I wonder if this attendance to great detail is actually a help or a hindrance in a large project, and I suspect that my attraction to Ada may be more down to personal prejudice than rational instinct.

Are there any opinions, experiences or references to papers on the subject ?

Which is better in the long term, the compact form of O'Caml or the spelling-it-out-in-detail of Ada ?

I guess the question you should ask yourself is for how long and by how many people the software is going to be maintained. This basically tells you how much to value readability. Ada is very much into readability (at times, perhaps, too much). The price you pay comes from having to write more verbose code.

Notice, however, that Ada makes you write code at the semantic level appropriate to the problem domain. The language is, overall, expressive enough that you don't waste too much time worrying on low level details (as compared to other mainstream imperative languages).

Personally, I tend to flunctuate between really liking the Ada approach and running to Python and Scheme for some sanity... On a large project I would go for readability.

Obviously, langugae choice is about tradeoffs. I think there several important features Ada lacks (e.g., DbC, and a decent standard collection libray). But when you balance all factors, in most cases, I prefer it to many of the other possible choices.

The hard part, of course, is balancing all these factors for yourself...

does Ocaml support a basic Win32 API ?

You mean, is there a library for interfacing with Win32 graphics? I don't know; that would be a good question for the Caml mailing list. However, checking the "Caml Humps," I see at least one such library available.

Looking at O'Caml, it is quite elegant in a sense, in that it dispenses almost entirely with type declarations, however it is over-cleverness (doing things for you) of just that sort that has driven me away from C++. I quite like the way that Ada makes you write it all down; the source then becomes more self-documenting, and you know that what is written is what you get, no more and no less,

First, nothing in Ocaml prevents you from writing down type signatures.

Second, if Ada's type system were as powerful as Ocaml's, the number and size of type declarations would become unwieldy and you would quickly get tired of informing the compiler of what you think is an obvious type for some variable. (And if you don't believe me, try programming in Coq!)

Third, I think there is a difference between "over-cleverness" and ingenuity. Things like implicit conversions are not exactly what I would call over-clever, but IMO they do not work well because they aren't a general solution to the problem.

For example, there are many ways to embed the integers in the reals, or to convert a number to a string (as in Java); why institutionalize only one? (Indeed, induction on the naturals depends on an isomorphism between the naturals and a subset. Why isn't this at least given given special status?) Furthermore, conversions should at least be injections. But C++ doesn't guarantee this. Even better, they should be retracts; that would ensure that if you convert to and back from a type, that you get back the original result. But C++ doesn't guarantee this, and that is precisely why you lose precision depending on how you write out arithmetic expressions in C++. In short, mathematical properties of such things can be very important in practice.

But type inference interacts very naturally with the pure functional sublanguage of ML, and a bit less naturally (but still acceptably and always safely) with the imperative subset. It always calculuates the most general type, so you need never worry that you are more clever than the compiler, and that the compiler just couldn't figure out what type you were thinking of. This is the important property which makes type inference a boon rather than a bane. You will find, I think, that the ML designers take much greater care in ensuring that powerful features are well-behaved. Indeed, this is the hallmark of the ML family of languages.

Fourth, type inference makes it easier to change programs. If you change a value's definition so that the type changes, you need not write down it's new type, either where it is defined or where it is used (which can be very many places).

Fifth, thousands of people use type inference in Ocaml, Standard ML, Haskell and Clean every day. Lots of people write complicated things in these languages, like compilers and theorem provers. (Indeed, the compilers for these languages are mostly written in those respective languages.) The reaction to type inference is, I think, overwhelmingly positive. People acclimate to type inference quite quickly as well.

There are two downsides to type inference that I know of, but only one will affect you.

First, without the requirement that type inference be decidable, ML's type system could be more powerful. But that doesn't affect you because it will already be more powerful than the type systems you are used to.

Second, some compilers for languages that support type inference produce misleading type error messages, because if a type error occurs, then the order in which the program was traversed to do the inference matters. (If the program is well-typed, then the order never matters.) The effect of this is that the compiler may complain that, for example, a function accepting ints was applied to a string, but in fact the problem is that the function's intended argument type was string, not int, and this would have been a non-issue if you had declared the type of the function. But this is really a problem with the compiler error messages, and not the type inference algorithm. In any case, you get used to it soon enough.

It is simply not the only factor.

Thanks all, and maybe I'll be reporting back in a year or so on ``One user's experience of moving away from C++ to Ada|O'Caml or whatever''.

At the moment I am quite strongly leaning towards Ada for the rewrite, for many reasons: long-term maintainability (through readability), diversity of compiler sources, ease of transition and, well, aesthetics.

Criticising those last two myself: yes, Ada is a conventional block-structured imperative language like Pascal, so it will be easier to make that move, rather than to O'Caml (that language has the imperative features, yes, but bear with me ...). That objection ties-in with the aesthetics. O'Caml is a functional language: it is a member of the ML family, and ML is functional. It seems to me that the functional `paradigm' is incompatible with the object-oriented paradigm, which again sits uncomfortably with the imperative paradigm of the Pascal. So if I go with O'Caml, I'll have imperative code (the output of the Pascal to O'Caml translator) mixed with object-oriented code (the Win-32 stuff and file handling, which is written in C++ at the moment) for IO (which is therefore not very `functional'), all running on a language which only shows its real strength when used functionally.

The reason that functional programming is incompatible with OOP (ISTM) is that in FP one is manipulating *values* and particular instances of those values are unimportant: all that matters is the end result, so that when one says for example let concat = left ^ right, left and right are not altered and they may or may not be temporaries (in the C++ sense) that are GCed afterwards. OOP on the other hand manipulates objects, so that it is more natural to write concat ^= right, which changes the object, as does any method: that's what methods do. Theoretically in FP one nevers assigns-to or changes an entity once it is set up. Maybe I should use {< >} for all the method declarations.

So that's why I'm unhappy with going with O'Caml, although the objection would apply to any language that started from a functional base and added OO later. It seems to me that I would not be using the language properly, whereas Ada seems like a perfect fit. At the same time, I feel cowardly. Ada is a rather conventional 3GL, a well-designed one certainly, but nevertheless a straightforward von Neumann imperative carthorse. I do just wonder if the productivity reduction that gave rise to this discussion will be repeated with the less adventurous choice. Comments ?

With regards to some of the comments that you've made in this thread:

1. The fall-through on the switch statement appears to be a very big issue for you. While I certainly accept the validity of your objection to this feature, I have to say that for me it's not a biggy. In over 15 years of C/C++/Java programming I don't recall ever getting bit by this one. I'm not saying it never happened only that if it did, it was probably very rare. OTOH I do not make constant switches from C++ to Pascal.

2. Probably my biggest issue with Java right now (and one I actually forgot in the list of features Java lacks) is missing enums.

Also looking at your last comment I sort of get the you want to eat your cake an keep it too. On one hand you want to retain the old paradigms because of your existing code. OTOH you want to use a new paradigm because the old ones don't appear to be good enough. Wait a minute, I have the same problem ;-)

long-term maintainability (through readability)

Since you are still maintaining Pascal code, I guess you have a very long term in mind, and on that score I cannot contest that Ada, like Fortran, LISP and C, is more likely to limp its way into the next decade than Ocaml. :) On the other hand, when it comes to readability, I cannot agree. Ocaml has more powerful facilities for abstraction than any other typed procedural or object-oriented language, which means that it is easier to separate concerns, which means, to me, that it is easier to read programs, because I don't have to keep the whole thing in my head at once.

diversity of compiler sources

Conceded.

ease of transition

Conceded. But then you have to ask yourself about the path less traveled.

aesthetics

Ah, aesthetics! Of course no one can make a rational argument against aesthetics, since the very word nullifies rational concerns. Nevertheless:

It seems to me that the functional `paradigm' is incompatible with the object-oriented paradigm, which again sits uncomfortably with the imperative paradigm of the Pascal.

What do you mean by "incompatible"? (I know you elucidate this later, but I still find it unclear.) Surely you don't mean that it is imposiible to mix functional and OO code. I guess you mean that there is an overlap in functionality, that they are not "orthogonal." I agree this is distasteful.

But surely there are already plenty of non-orthogonal features and infelicities in OO languages. For example, one has procedural loops and iterators; update via assignment and update via method call; "out" and "in-out" parameters versus return values; the fact that you can initialize an object via its constructors, or by creating it and then setting its fields via method calls; templates vs. inheritance; inheritance vs. subtyping; private, protected and public methods; classes vs. packages. The list goes on and on.

So, if you are willing to put up with all of that already, why are you unwilling to put up with the additional overlap you get when you mix functional and OO code? (Anyway, there are plenty of OO languages which have closures already... Smalltalk, Ruby and Python come to mind.)

So if I go with O'Caml, I'll have imperative code (the output of the Pascal to O'Caml translator) mixed with object-oriented code (the Win-32 stuff and file handling, which is written in C++ at the moment) for IO (which is therefore not very `functional'), all running on a language which only shows its real strength when used functionally.

I think it is true that languages like Ocaml are at their best when you use them to write "purely functional" code, because that sort of code is the most modular. However, that does not imply that FP languages are worse than extant languages when you use them to write "OO" or "procedural" code. Indeed, I think Ocaml would still be a better language than Ada or C or C++ even if it didn't have higher-order functions.

The reason that functional programming is incompatible with OOP (ISTM) is that in FP one is manipulating *values* and particular instances of those values are unimportant: all that matters is the end result, so that when one says for example let concat = left ^ right, left and right are not altered and they may or may not be temporaries (in the C++ sense) that are GCed afterwards. OOP on the other hand manipulates objects, so that it is more natural to write concat ^= right, which changes the object, as does any method: that's what methods do. Theoretically in FP one nevers assigns-to or changes an entity once it is set up.

So? You have described pieces of code, involving concatentation of strings, as if they were equivalent, regarding them as differences in "style". But they aren't equivalent, and the difference is more than style.

The first piece of code, which you dub "FP-like", and does not alter the state of concat, is more reusable. It will give the same result in any context, since there is no side effect.

The second piece of code, which you dub "OO-like" (though in fact it only uses imperative features), and does alter the state, is less reusable. It depends on the context.

In order to regard them as equivalent in all contexts, one needs to restrict the contexts in which the OO-like code can be used. This burden is a proof obligation on the programmer. Conversely, if you let me transform the FP-code, then I can make it equivalent to the OO-code simply by adding an assignment after it. These are not matters of opinion, style or aesthetics; they are facts which point to genuine differences in the intrinsic reusability of FP-like and OO-like code.

Lots of people argue that programmers should write FP-like programs all the time, because they are more reusable. (I prefer to do so myself.) But that is really beside the point. It is possible to write correct programs in either style, provided you are careful to respect the difference in semantics. The point is whether you trust yourself and the programmers who will be maintaining your code to be aware of the extra conditions which need to be fulfilled in order to reuse OO-like code correctly.

Programming languages offer mechanisms that help with program design (abstraction facilities, mainly), but realizing how to use them best is not trivial.

I agree that modern functional languages (with rich type systems, GC, syntactic extension etc.) provide many very useful tools. Alas, learning how to use these takes time and practice. (Time well worth spending, for sure).

Let me just say that Ada is a bit more than a conventional 3GL. It provides several mechanisms that can help you achieve productivity gains, via better software design.

Top among these is Ada's generic units, which are quite expressive and useful (and are still compile time checked). ML functors are richer, but Ada's generic facilities are not to be sneered at.

Ada's packages allow richer abstractions than class based languages (a package can contain various related types, with shared visibility).

Ada's tasking facilities alow you describe concurrent activities naturally.

Underneath all these you find a fairly expressive type system. It is a far cry from what you'd find in ML or Haskell, but compare to most imperative languages it adds great value (indeed, you should spend some time learning how to use it to the max).

Ehud: are Ada generics any more powerful than C++ templates ?

Another pseudo-troll: no one has leapt to the defence of Eiffel, which I rather sneered-at in my original posting.

Though similar in function, C++ templates and Ada generic units are quite different, making it hard to answer your question.

C++ has several useful features that Ada generics lack. These include template specialization and automatic instnatiation, both missing in Ada.

Ada generics are closely related to the type system, allowing better specification of the requirements from the generic parameters. This also leads to the possibility of separate compilation, and various kinds of optimization.

Since instnation is explicit, and is not done by matching names, you can easily do things like procedure Rev_Sort is new Sort(..., "<" => ">") (in effect reversing the order relation). In general, this gives you more flexibility when combining various pieces of code with your generic units. [Other Ada features like renaming can help to].