Home » forums » LtU Forum

Language comparison experiment

Another quiet time post...

I am considering a language comparison experiment that will take a bit of time, and would be interested in any comments or suggestions that might make the results more interesting/significant. This is primarily for my own interest.

There don't seem to be any studies where the same piece of software has been implemented twice, in two different languages and paradigms by the same person/people. I plan to do this, writing a smallish, but non-trivial program twice from scratch.

Such a comparison will of course quite subjective. My intention is to keep a journal tracking how easily various features are added. In order to be more even handed, I will build the software according to a staged plan, and will do each stage in both implementations, alternating which I do first. Other than trivial metrics (line counts, time taken etc), the main results will be my opinions and judgements: on such things as the relative effort required for each stage, and the my perceived quality of the results. Nevertheless, I'm quite interested in what conclusions, if any, I reach.

I'm primarily interested in a functional versus objected-oriented comparison. Hence, I probably should choose two implementations where this is the only difference. (two implementations in ocaml would be a possibility). However, another driver is that the experiment remain interesting long enough to actually achieve something! Hence, I plan to do an implementation in my two preferred languages, python and haskell. These two languages differ in many other aspects: dynamic versus static typing; interpreted versus compiled; etc.

Finally, specific languages are better suited to certain types of tasks. I've got a couple of ideas here, and don't want to select a task whose implementation is obvious from the the description/plan. I'm tempted by a GUI application using the wxWidgets framework, since it is supported in both languages.

By Tim Docker at 2004-11-09 08:15 | LtU Forum | previous forum topic | next forum topic | other blogs | 13557 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Language comparison

Here at Chalmers Univ. we've had our students do such comparisons for quite some years now. In their first programming course they learn Haskell and in the second course we used to teach them Ada but we've now switched to Java. In their third programming course they get to do a little bit of programming in both languages they've learnt. Among other things they get to implement one particular program in both languages and then write up their experiences. The particular program they write is AFAIR a program which reads a text file and can then anwer queries about the occurrences of certain words.

The general conclusion from the students the year I supervised this course was that the Haskell program was much easier and faster to write even though they started writing in Haskell. I think this is remarkable since when they start with the Ada/Java program they have already thought alot about the problem and it should be fairly easy to write the solution. Still they thought the Haskell version was nicer to write.

Just my two cents.

By Josef Svenningsson at Tue, 2004-11-09 11:38 | login or register to post comments

Haskell v. Java unfair?

I was actually thinking of a more significant project, but your comments on the students' assignments are interesting.

Given my personal preferences, I'm not surprised that a statically typed language with type inference was preferred over one with explicit type annotations. However, I'm unsure what I'll end up thinking when I compare python (dynamically typing) with haskell (static type inference). By choosing a task that is traditionally considered "imperative" in nature, I'm hoping for a interesting comparison.

By Tim Docker at Tue, 2004-11-09 21:05 | login or register to post comments

wxFruit

If you really want to live on the bleeding edge, you might try some wxFruit to make your gui more functional and less imperative.

By Greg Buchholz at Tue, 2004-11-09 21:19 | login or register to post comments

too bleeding edge perhaps...

I got enthusiastic reading about wxFruit, at least until I came to this bit: "Currently wxFruit only supports four widget types: simple buttons, static text labels, slider controls, and the amorphous wxpicture widget".

Praise the author for the experiment, but it looks a long way from prime time.

By Tim Docker at Tue, 2004-11-09 23:57 | login or register to post comments

As long as you mentioned it...

it looks a long way from prime time

So you're saying that finishing off the wxFruit project could be a good non-trivial candidate for a language comparison study ;-)

By Greg Buchholz at Wed, 2004-11-10 04:35 | login or register to post comments

I switched from Python to Haskell.

For me, Python was my first love, but Haskell is deeper, cleaner, and more powerful. I want expressiveness first and foremost. Python is expressive, more than the sum of its parts even, but Haskell does so much more.

I'm a firm believer in using whatever fits the way you think, so this comparison may only be true for me. I look forward to more comparison data from you. Do you have any small projects ideas where the comparison would be interesting to you? I may be able to contribute some code.

----

Shae Erisson - www.ScannedInAvian.org

By shapr at Tue, 2004-11-09 22:03 | login or register to post comments

Reimplementation Study

There's at least one reimplementation study, by Howard Trickey of Bell Labs. The reference I have is

H. Trickey, C vs Lisp: A Case Study, SIGPLAN Notices, 1988.

By Tom Duff at Tue, 2004-11-09 16:31 | login or register to post comments

Haskell vs. ML study

There is:
Leucker, Noll, Stevens, Weber: Functional programming languages for verification tools: Experiences with ML and Haskell. A revised version will appear in STTT.

We were not talking about a reimplementation, but we believe the systems we compared (Concurrency Workbench and Truth) are sufficiently similar in terms of features.

We were also not only considering the languages themselves, but also their environments, because we believe that to be equally important, at least.

BTW: Our conclusion: we'd like to try out Java next time... And the reason for this was mostly NOT the languages itself.

Regarding reimplementation: we found it rather hard to imagine a fair setting for this:

One team implementing the same thing twice is unfair towards the second language, because they have a much better idea of the problem domain after the first try.

So we assumed that you'd need two teams of equal programming skills which independently implement the same thing. Also, the problem domain would have to be sufficiently generic. Now go and find such a setting... especially in scientific context, but well, have a look at paper... :)

By michaelw at Sat, 2004-11-13 18:19 | login or register to post comments

surprising conclusion

Thanks for the reference - it made interesting reading

BTW: Our conclusion: we'd like to try out Java next time... And the reason for this was mostly NOT the languages itself

This is surprising. In my experience it seems that most who use functional languages for significant projects tend to claim that the project would have been much harder (or impossible) in more mainstream languages.
Which features of the "impoverished environments" would java/C++ provide that are so sorely missing in FPLs?

By Tim Docker at Sun, 2004-11-14 21:44 | login or register to post comments

re: conclusion

Well, to answer your question directly:
more readily accessible libraries, debuggers, more expertise, ...

(Please refer to the paper for details, and keep in mind that we are talking about a specific setting: Haskell and Standard ML for verification tools. When applying our experiences to other settings (languages, problem domains), some care has to be taken not to draw wrong conclusions.)

Longer answer:
We identified certain very helpful features in functional languages for our field, but we also talked about some shortcomings. As I said before, some are not directly connected to the languages itself, but to their environments.

It all depends on what is needed by the project, and what you are willing to sacrifice for it, and I think we came to the conclusion that (a) it's not impossible to do such a project in an imperative language, and (b) it would be interesting to see how it works out, and to compare experiences.

Mind you that I, for example, do use functional languages for prototyping work a lot. Their conciseness and expressivity are extremely well suited for such a task, in my opinion.

But in the paper, we also talked about software engineering issues, because they are equally important for a successful project. There are some areas where we would like to see some improvements in the two languages which we investigated.

By michaelw at Tue, 2004-11-16 17:38 | login or register to post comments

Haskell vs. Ada vs. C++ vs. Awk vs. ...

... An Experiment in Software Prototyping Productivity

Just remembered this... Might be relevant as well.

By michaelw at Sat, 2004-11-13 18:22 | login or register to post comments

Another comparison

Java vs. Lisp, a follow-up study on yet another case study comparing Java vs. C++.

Well, these are not done by the same group of people, but might still be relevant.

By michaelw at Sat, 2004-11-13 18:37 | login or register to post comments

A useful tool for comparisons of languages

I have made a few experiments along this line using the dot net framework. I have managed to rustle up 27 different dot net languages and when they are all installed you can do some pretty interesting things: run parallel implementations of the same things and examine the IL code produced. Anyway, the .NET provides a nifty way of doing this - you can compare functional, logical, imperative and the rest.

By Diarmuid Pigott at Sat, 2004-11-13 19:03 | login or register to post comments

dot netting and language comparisons

Zipping between C++ and Fortran 90 is really nifty when it comes to doing matrix transforms of subregions. An interesting thing is how F90 makes everything to do with arrays, vectors, matrices etc etc easy. - but no OO...

By Diarmuid Pigott at Sun, 2004-11-14 17:55 | login or register to post comments