Lambda the Ultimate

inactiveTopic Erlang: A Case Study of Technology Introduction
started 2/18/2003; 2:41:52 PM - last post 2/24/2003; 10:27:44 AM
Ehud Lamm - Erlang: A Case Study of Technology Introduction  blueArrow
2/18/2003; 2:41:52 PM (reads: 2752, responses: 22)
Erlang: A Case Study of Technology Introduction
Bjarne Dacker. Concurrent Functional Programming for Telecommunications: A Case Study of Technology Introduction

It seems several of the LtU editors discussed this paper over on the LL mailing list, but not one of them thought of mentioning it here.

Still, I think this thesis provides valuable information about the evolution of erlang, and its use inside and outside Ericsson.

Getting technology from research labs into industry is a well known problem, and programming languages are no exception. However, it may be useful to recall that quite a few succesful programming languages followed this route (think C, C++, and even Smalltalk).

So that's the historical angle.

The other reason for posting this here on LtU is to encourage anyone who has a cool (functional) inhouse language to go ahead and release it to the public domain (be sure to post a message on the LtU discussion board )


Posted to history by Ehud Lamm on 2/18/03; 2:44:10 PM

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 1:02:08 AM (reads: 1607, responses: 0)
Perhaps the most interesting event described in this thesis is the so-called Backlash (chp. 7). In February 1988 Erlang was banned within Ericsson Radio. The thesis quotes an internal memo giving the reasons for this decision. Here it is:

"The selection of an implementation language implies a more long-term commitment than selection of processors and OS, due to the longer life cycle of implemented products. Use of a proprietary language, implies a continued effort to maintain and further develop the support and the development environment. It further implies that we cannot easily benefit from, and find synergy with, the evolution following the large scale deployment of globally used languages."

I don't find these arguments persuasive, but they are worth analyzing in detail.

A more detailed cost analysis might have led to a different decision, since the cost of supporting an (internal) language is often less than the cost of using general purpose languages that might not fit your exact needs.

This issue is one that comes up in almost all situations involving in hose languages and programming tools, making it quite important.

I am unaware of good discussions of this issue, either based either on economic model or on empirical data.

Brent Fulgham - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 9:13:03 AM (reads: 1543, responses: 1)
It's an interesting problem. At one company I am familiar with, a decision was made to use a proprietary, non-standard programming language (available from a single vendor) rather than a "globally used" language like Java, C++, Smalltalk, etc. The goal was to use this language as the basis for all future products.

The ultimate decision was made at a high level, based on the belief that this proprietary language provided "good example programs" and "nice web tutorials", and had a nicely integrated development environment. Amazingly, no other criteria was used in the selection, and no engineering analysis of alternatives was ever conducted.

I know of no other industry in which difficult technical decisions are made based on glossy sales brochures and advertising in trendy magazines by effectively untrained managers. If this was done in any of the scientific disciplines governed by licensing acts, licenses would be revoked left and right, and many companies would be held liable for damages when projects were delivered past due and over budget.

Brent Fulgham - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 9:33:31 AM (reads: 1547, responses: 1)
Of course, I didn't really respond to your thesis. I have to say that my gut reaction is that using a "globally used" language is often a good choice for the reasons they mention. After all, as we have discussed on this page in the past, "It's the Libraries, Silly!" If suitable special purpose libraries are available (or can be built), that is often a more economical and schedule-friendly solution that trying to build and maintain an entire programming language.

On the other hand, we also know that DSLs are extremely helpful in many tasks, and can produce code that is far easier to extend and maintain.

Perhaps the only good choice is to use a "globally used" language that incorporates facilities that enable the easy and efficient implementation of "Little Languages", and rely on libraries to provide any special-purpose constructs needed for the task at hand.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 9:42:42 AM (reads: 1583, responses: 0)
So the lunatics are running the asylum, ah?

I was thinking more about using languages the company itself develops and supports, so at least there's no vendor lock-in.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 9:50:40 AM (reads: 1570, responses: 0)
I am not convinced any of these options is the single right choice. I think the context does matter (i.e, if you need non-programmers to be able to produce code, than using Java isn't the right way to go, and no library is going to solve that).

What is really needed, I think, is a sophisticated pattern for analyzing such choices and making informed decisions.

That's why I highlighted the faulty reasnoing quoted from the Ericsson memo. The decision may have been the right one to make, but the reasons given aren't enough to make the case for it.

You must at least measure true sotware costs (of maintaining the language, the tools, the libraries, not only of producing the first production versions of applications), other costs (documentation, training) etc. Only then can you comapre the different solutions.

Oh, one other point. I hi-tech time to market is esp. important. If choosing the right language gives you an important time to market advantage, it may make the difference between success and chapter 11.

Luke Gorrie - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 10:00:18 AM (reads: 1530, responses: 0)
You can also find some interesting thoughts on the Erlang experience in some slides of Joe Armstrong's: http://www.research.avayalabs.com/user/wadler/armstrong.ps

Isaac Gouy - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 10:25:22 AM (reads: 1534, responses: 1)
languages the company itself develops and supports
Smallworld, a UK GIS company, have been successful with their in-house hybrid-OO programming language Magik.

why would a small company like Smallworld devote considerable resources to the creation of such a development environment?

An Object-Oriented GIS Issues and Solutions

Smallworld 3.0 can write in Java

Todd Coram - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 1:38:27 PM (reads: 1480, responses: 1)
The problem with globally used languages is that those who say this tend to mean languages by which I can hire from a large pool of programmers rather than well known languages that fit the domain. You could argue that there are 4 "acceptable" general purpose (globally used) languages: C/C++, Java, Visual Basic and Perl. Perl is fading. Python may get added. Cobol and Tcl are too old. Lisp had no chance and FPs look too weird.

When I approach a problem domain, I consider what language is the most expressive in describing a solution. If your problem domain suggests a great deal of concurrent processing, is concurrency better expressed in Posix threads or a more message passing approach like Erlang (or Mozart or Tcl-Threads or whatever)? (Unfortunately, I am usually coerced into using the popular language of the day...)

If Erlang makes it easier (and safer) to describe telecommunication systems, then it would make sense to teach (newly hired) programmers what they need to know about Erlang.

Perhaps the best way to market a non-popular language is to make it's implementation a library available from a popular language. If Erlang (or Lisp or Haskell or XXXX) was a library call from Java/C++: think "eval_expression(....)", then maybe it could be snuck in... nah.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 1:58:03 PM (reads: 1519, responses: 0)
But the point is (remember this is LtU) that not everything can be provided by a library.

Syntax is usually problematic (e.g., standard Java doesn't have syntax extension), but other semantic issues are even more fundamental. You cannot ensure referential transparency via a libray, in an impure language. You cannot make the language properly tail recursive. You can't make the type system more expressive. And so on, and so on.

And let's not forget that some of these issues are related to producing efficient code (e.g., can the compiler assume no aliasing).

So it's not that simple to build a library instead of a language. Sure, if it's possible than this solution is much cheaper, but then again library is simply not a viable solution for more interesting DSLs.

(Unless, of course, the host language is Haskell )

Patrick Logan - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/19/2003; 3:41:20 PM (reads: 1451, responses: 2)
Lisp had no chance and FPs look too weird.

I'd like to see a technologist, a linguist, a psychologist, and a sociologist come up with some integrated reasons for this.

Perhaps the best way to market a non-popular language is to make it's implementation a library available from a popular language.

In fact this is the only way Lisp ever achieved any kind of mass use, i.e. via Autocad, Emacs, various CAD tools.

This is absolutely the way the software world will evolve into a more massively concurrent system (e.g. Erlang-like but multi-lingual). People will continue to use language X, only to write little imperative snippets of concurrent message-based systems. Over time the more applicable(!) languages will survive and the C-like imperative junk heaps will just rust away bit by bit.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 12:18:19 AM (reads: 1453, responses: 0)
see a technologist, a linguist, a psychologist, and a sociologist

I was so sure this is the beginning of a joke...

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 1:03:41 AM (reads: 1441, responses: 0)
Over time the more applicable(!) languages will survive and the C-like imperative junk heaps will just rust away bit by bit.

Natrually, some languages have the power to host embedded languages better than others.

I guess that if you are into dynamically checked languages, you should choose Scheme, and if you prefer strong statci checking, choose Haskell.

But that's me. I guess as strong a case could be made of ML (Ocaml), and Lisp.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 1:15:22 AM (reads: 1451, responses: 0)
I visited the Smallworld site. The whitepaper Ten Difficult Problems in Building a GIS discusses the decision to build a new programming languages.

The analysis shows the same fautls as the Erlang memo: no cost analysis, or at most some hand waving arguments about costs.

Of course, these are public documents, so one can still hope that the proper analysis was in fact done.

Isaac Gouy - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 8:37:27 AM (reads: 1363, responses: 1)
Are you bemoaning the immaturity of the software industry (compared to manufacturing)?

Manufacturing organizations adopted ABC (Activity Based Cost accounting, process cost accounting) from the 80s.
Has the software industry adopted these manufacturing management techniques?
If not, why not? (No pricing pressure??)

"The Inmates are Running the Asylum" makes a nice analogy between making movies and developing software. What's cost accounting like in the movie business?

Patrick Logan - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 9:36:39 AM (reads: 1343, responses: 0)
Ehud: I was so sure this is the beginning of a joke...

I am sure there is a joke in there somewhere. I just haven't found a good punch line yet.

Isaac: Are you bemoaning the immaturity of the software industry (compared to manufacturing)?

Remember, we need to emphasize that software development is like designing a product, not manufacturing a product.

Isaac Gouy - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 10:36:22 AM (reads: 1341, responses: 0)
like designing, not manufacturing
Manufacturing organisations often integrate design groups and engineering groups and production operations. Manufacturing is not just production.

Does like designing, not manufacturing change how Activity Based Costing could be applied in the software industry?

Emphasizing design makes the movie analogy appropriate (The gist is summarized here Cooper, A. The Inmates are Running the Asylum)

Patrick Logan - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 2:27:38 PM (reads: 1312, responses: 0)
Manufacturing organisations often integrate design groups and engineering groups and production operations. Manufacturing is not just production.

OK. So I mean software development is like the design part of manufacturing, not the production part of manufacturing.

;^)

Does like designing, not manufacturing change how Activity Based Costing could be applied in the software industry?

Absolutely. ABC assumes the value is constant, and the goal is to cost each step toward that constant value.

In the design phase of anything, you need to determine which steps are value add, i.e. where the real creativity comes from. The other steps are just enablers to get to the creative steps and from there into production.

Looking at the software development process you use, categorize each activity into three buckets.

Standardized
Standardized activities just need to be done a certain way, maybe "Use CVS for version control".
Continuous Improvement
Continuous improvement activities just need to monitored for better practices, maybe "Use CVS branches for experiments and the main branch for the the latest complete product"
Breakthrough
Breakthrough activities are where the creativity happens, maybe "Develop spike solutions to any task that calls for a design or a tool the team is unfamiliar with"

The first two can use Activity Based Costing. You want the lowest cost that works, but there are probably no breakthroughs that will drastically alter the value.

The breakthrough tasks are those that have to be done, but since they will dramatically alter the value of the product, the goal is not to do them for the lowest cost, but to do them for the most value.

Emphasizing design makes the movie analogy appropriate...

Yes, thanks for the analogy and reference.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 3:35:07 PM (reads: 1340, responses: 0)
Introduction to Activity Based Costing

Seems to me more appropriate for support organisation than development (please, no flames).

But maybe this make this even more relevant, seeing as language support is costlier than language development.

Isaac Gouy - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/20/2003; 3:54:33 PM (reads: 1286, responses: 0)
For you Ehud, ABC applied to a university department ;-)
Using Activity-Based Costing to Manage More Effectively

more appropriate for support organisation than development
Although there are plenty of routine mundane activities within software development.

Patrick, I found that helpful.
Accounting for 2 out of 3 kinds of activity, would be a start.

ABC in service industries.

Ehud Lamm - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/24/2003; 12:55:58 AM (reads: 1218, responses: 0)
I received the following interesting email from Arthur Chance:

A friend of mine pointed me at the Erlang discussion on Lambda the
Ultimate as it mentions Smallworld's language Magik. I was the
designer and implementor of Magik.

You remarked:

> I visited the Smallworld site. The whitepaper Ten Difficult Problems > in Building a GIS discusses the decision to build a new programming > languages.

Historical update: Smallworld does not exist as an independent company any more, it has been part of GE Network Systems since Q4 2000. I left at the end of 2000 and have no connection with them anymore other than friends still working there.

> The analysis shows the same fautls as the Erlang memo: no cost > analysis, or at most some hand waving arguments about costs.

Remember it was a white paper - i.e. a technical document which is also part of the marketing effort. You don't show internal debates and costings in white papers.

> Of course, these are public documents, so one can still hope that > the proper analysis was in fact done.

It was 14+ years ago that all the debates were held and the decisions made so my memories are somewhat hazy. There's also a lot of back story to the founding of Smallworld that would take too long to cover but our prior experience influenced various decisions. However, at an executive summary level, the key points behind Magik were:

1) As mention in the white paper on Magik (TP5), we wanted the customisation (aka scripting) language to be the same as the development language.

2) We wanted that language to be OO and capable of incremental modification rather than a compile and link system.

3) For strong commercial reasons (i.e. people offering us money immediately for it :-) we wanted to run on a variety of platforms. (Initially VMS on Vaxen, SunOS 4 aka Solaris 1, on Sun 3s and 4s, and SGI Irix. Later we sold on SunOS 4, Solaris 2, AIX, HP UX, OSF/1 aka Digital Unix aka True64, Linux, Windows NT and Win 2000.)

The only commercially available languages around at the time (1988-9) that satisfied point 2 were Lisp + some form of object system (CLOS 1 was just about arriving IIRC) or Smalltalk-80, both of which I (but nobody else in the team) had worked with.

ParcPlace's ST-80 only ran on Suns at that time, had a fixed per seat cost (with no discount for multiple seats) that was 30% of our target product price per seat, and had no way to deliver products without full source code so would have made all our IP immediately visible.

There were various commercial Lisp implementations for all our target platforms but no two implementations were exactly the same language or had the same OO facilities, most were remarkably buggy, some forced delivery in source form and at least one implementation cost more per seat than we intended to sell the product for.

Given that I have a background in computer language implementation, implementing our own was pretty much a no-brainer, especially as I'd been doing the design work towards a language like Magik for some time before Smallworld was set up. It also meant that we could consider point 1 and make the syntax a little more end user friendly. Two of us implemented the first version of the core language (roughly equivalent to ST-80 in capabilities) in 16-18 man-months.

Smallworld went on to become market sector leader and a multinational with a nine digit US$ turnover so we must have done something right. Just don't start me on what we did wrong. :-)

I'm afraid I don't have the (personal) bandwidth available to join in on LtU, but if you wish to quote this email there, feel free. Just obscure my email address, I get enough spam as it is.

Patrick Logan - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/24/2003; 6:57:40 AM (reads: 1166, responses: 0)
ParcPlace's ST-80 only ran on Suns at that time, had a fixed per seat cost (with no discount for multiple seats) that was 30% of our target product price per seat, and had no way to deliver products without full source code so would have made all our IP immediately visible.

PPST ran on more than Suns at the time, and it had a cross platform GUI even at that time better than Swing.

The pricing was ridiculous, and especially ironic these days, where Cincom has extremely flexible pricing and cannot give PPST away to teams like this for new development.

Back then ParcPlace had visions of a system so good that enterprises would fall all over each other with large amounts of money for enterprise-wide development licenses. Well, it was the best IT development system around. But hardly anyone was aware of it.

IT developers were busy with crippled tools like VisualBasic and PowerBuilder that could do 1/10th of what PPST could. Better pricing, and seeding some key entreprenuers could have had more effect on the industry at the time.

As it turned out, Smalltalk gained popularity over the next seven years or so, due to its adoption by IBM. But just as Smalltalk was getting some real press and word of mouth, everything changed with the arrival of Java. This is also ironic, since Sun tried to license Smalltalk from ParcPlace at lower prices in order to use it in small devices. The answer was no, and so Gosling invented Java (nee Oak).

Isaac Gouy - Re: Erlang: A Case Study of Technology Introduction  blueArrow
2/24/2003; 10:27:44 AM (reads: 1167, responses: 0)
PPST ran on more than Suns at the time
More specifically their product needed to be available for "VMS on Vax... SGI Irix" - which don't seem to have been supported PPST platforms.