Lambda the Ultimate

"The selection of an implementation language implies a more long-term commitment than selection of processors and OS, due to the longer life cycle of implemented products. Use of a proprietary language, implies a continued effort to maintain and further develop the support and the development environment. It further implies that we cannot easily benefit from, and find synergy with, the evolution following the large scale deployment of globally used languages."

I don't find these arguments persuasive, but they are worth analyzing in detail.

A more detailed cost analysis might have led to a different decision, since the cost of supporting an (internal) language is often less than the cost of using general purpose languages that might not fit your exact needs.

This issue is one that comes up in almost all situations involving in hose languages and programming tools, making it quite important.

I am unaware of good discussions of this issue, either based either on economic model or on empirical data.

The ultimate decision was made at a high level, based on the belief that this proprietary language provided "good example programs" and "nice web tutorials", and had a nicely integrated development environment. Amazingly, no other criteria was used in the selection, and no engineering analysis of alternatives was ever conducted.

I know of no other industry in which difficult technical decisions are made based on glossy sales brochures and advertising in trendy magazines by effectively untrained managers. If this was done in any of the scientific disciplines governed by licensing acts, licenses would be revoked left and right, and many companies would be held liable for damages when projects were delivered past due and over budget.

On the other hand, we also know that DSLs are extremely helpful in many tasks, and can produce code that is far easier to extend and maintain.

Perhaps the only good choice is to use a "globally used" language that incorporates facilities that enable the easy and efficient implementation of "Little Languages", and rely on libraries to provide any special-purpose constructs needed for the task at hand.

I was thinking more about using languages the company itself develops and supports, so at least there's no vendor lock-in.

What is really needed, I think, is a sophisticated pattern for analyzing such choices and making informed decisions.

That's why I highlighted the faulty reasnoing quoted from the Ericsson memo. The decision may have been the right one to make, but the reasons given aren't enough to make the case for it.

You must at least measure true sotware costs (of maintaining the language, the tools, the libraries, not only of producing the first production versions of applications), other costs (documentation, training) etc. Only then can you comapre the different solutions.

Oh, one other point. I hi-tech time to market is esp. important. If choosing the right language gives you an important time to market advantage, it may make the difference between success and chapter 11.

why would a small company like Smallworld devote considerable resources to the creation of such a development environment?

An Object-Oriented GIS – Issues and Solutions

Smallworld 3.0 can write in Java

When I approach a problem domain, I consider what language is the most expressive in describing a solution. If your problem domain suggests a great deal of concurrent processing, is concurrency better expressed in Posix threads or a more message passing approach like Erlang (or Mozart or Tcl-Threads or whatever)? (Unfortunately, I am usually coerced into using the popular language of the day...)

If Erlang makes it easier (and safer) to describe telecommunication systems, then it would make sense to teach (newly hired) programmers what they need to know about Erlang.

Perhaps the best way to market a non-popular language is to make it's implementation a library available from a popular language. If Erlang (or Lisp or Haskell or XXXX) was a library call from Java/C++: think "eval_expression(....)", then maybe it could be snuck in... nah.

Syntax is usually problematic (e.g., standard Java doesn't have syntax extension), but other semantic issues are even more fundamental. You cannot ensure referential transparency via a libray, in an impure language. You cannot make the language properly tail recursive. You can't make the type system more expressive. And so on, and so on.

And let's not forget that some of these issues are related to producing efficient code (e.g., can the compiler assume no aliasing).

So it's not that simple to build a library instead of a language. Sure, if it's possible than this solution is much cheaper, but then again library is simply not a viable solution for more interesting DSLs.

(Unless, of course, the host language is Haskell )

I'd like to see a technologist, a linguist, a psychologist, and a sociologist come up with some integrated reasons for this.

Perhaps the best way to market a non-popular language is to make it's implementation a library available from a popular language.

In fact this is the only way Lisp ever achieved any kind of mass use, i.e. via Autocad, Emacs, various CAD tools.

This is absolutely the way the software world will evolve into a more massively concurrent system (e.g. Erlang-like but multi-lingual). People will continue to use language X, only to write little imperative snippets of concurrent message-based systems. Over time the more applicable(!) languages will survive and the C-like imperative junk heaps will just rust away bit by bit.

I was so sure this is the beginning of a joke...

Natrually, some languages have the power to host embedded languages better than others.

I guess that if you are into dynamically checked languages, you should choose Scheme, and if you prefer strong statci checking, choose Haskell.

But that's me. I guess as strong a case could be made of ML (Ocaml), and Lisp.

The analysis shows the same fautls as the Erlang memo: no cost analysis, or at most some hand waving arguments about costs.

Of course, these are public documents, so one can still hope that the proper analysis was in fact done.

Manufacturing organizations adopted ABC (Activity Based Cost accounting, process cost accounting) from the 80s.
Has the software industry adopted these manufacturing management techniques?
If not, why not? (No pricing pressure??)

"The Inmates are Running the Asylum" makes a nice analogy between making movies and developing software. What's cost accounting like in the movie business?

I am sure there is a joke in there somewhere. I just haven't found a good punch line yet.

Isaac: Are you bemoaning the immaturity of the software industry (compared to manufacturing)?

Remember, we need to emphasize that software development is like designing a product, not manufacturing a product.

Does like designing, not manufacturing change how Activity Based Costing could be applied in the software industry?

Emphasizing design makes the movie analogy appropriate (The gist is summarized here Cooper, A. The Inmates are Running the Asylum)

OK. So I mean software development is like the design part of manufacturing, not the production part of manufacturing.

;^)

Does like designing, not manufacturing change how Activity Based Costing could be applied in the software industry?

Absolutely. ABC assumes the value is constant, and the goal is to cost each step toward that constant value.

In the design phase of anything, you need to determine which steps are value add, i.e. where the real creativity comes from. The other steps are just enablers to get to the creative steps and from there into production.

Looking at the software development process you use, categorize each activity into three buckets.

Standardized

Standardized activities just need to be done a certain way, maybe "Use CVS for version control".

Continuous Improvement

Continuous improvement activities just need to monitored for better practices, maybe "Use CVS branches for experiments and the main branch for the the latest complete product"

Breakthrough

Breakthrough activities are where the creativity happens, maybe "Develop spike solutions to any task that calls for a design or a tool the team is unfamiliar with"

The first two can use Activity Based Costing. You want the lowest cost that works, but there are probably no breakthroughs that will drastically alter the value.

The breakthrough tasks are those that have to be done, but since they will dramatically alter the value of the product, the goal is not to do them for the lowest cost, but to do them for the most value.

Emphasizing design makes the movie analogy appropriate...

Yes, thanks for the analogy and reference.

Seems to me more appropriate for support organisation than development (please, no flames).

But maybe this make this even more relevant, seeing as language support is costlier than language development.

more appropriate for support organisation than development
Although there are plenty of routine mundane activities within software development.

Patrick, I found that helpful.
Accounting for 2 out of 3 kinds of activity, would be a start.

ABC in service industries.

A friend of mine pointed me at the Erlang discussion on Lambda the
Ultimate as it mentions Smallworld's language Magik. I was the
designer and implementor of Magik.


You remarked:


> I visited the Smallworld site. The whitepaper Ten Difficult Problems
> in Building a GIS discusses the decision to build a new programming
> languages.


Historical update: Smallworld does not exist as an independent company
any more, it has been part of GE Network Systems since Q4 2000. I left
at the end of 2000 and have no connection with them anymore other than
friends still working there.


> The analysis shows the same fautls as the Erlang memo: no cost
> analysis, or at most some hand waving arguments about costs.


Remember it was a white paper - i.e. a technical document which is
also part of the marketing effort. You don't show internal debates and
costings in white papers.


> Of course, these are public documents, so one can still hope that
> the proper analysis was in fact done.


It was 14+ years ago that all the debates were held and the decisions
made so my memories are somewhat hazy. There's also a lot of back
story to the founding of Smallworld that would take too long to cover
but our prior experience influenced various decisions. However, at an
executive summary level, the key points behind Magik were:


1) As mention in the white paper on Magik (TP5), we wanted the
customisation (aka scripting) language to be the same as the
development language.


2) We wanted that language to be OO and capable of incremental
modification rather than a compile and link system.


3) For strong commercial reasons (i.e. people offering us money
immediately for it :-) we wanted to run on a variety of platforms.
(Initially VMS on Vaxen, SunOS 4 aka Solaris 1, on Sun 3s and 4s, and
SGI Irix. Later we sold on SunOS 4, Solaris 2, AIX, HP UX, OSF/1 aka
Digital Unix aka True64, Linux, Windows NT and Win 2000.)


The only commercially available languages around at the time (1988-9)
that satisfied point 2 were Lisp + some form of object system (CLOS 1
was just about arriving IIRC) or Smalltalk-80, both of which I (but
nobody else in the team) had worked with.


ParcPlace's ST-80 only ran on Suns at that time, had a fixed per seat
cost (with no discount for multiple seats) that was 30% of our target
product price per seat, and had no way to deliver products without
full source code so would have made all our IP immediately visible.


There were various commercial Lisp implementations for all our target
platforms but no two implementations were exactly the same language or
had the same OO facilities, most were remarkably buggy, some forced
delivery in source form and at least one implementation cost more per
seat than we intended to sell the product for.


Given that I have a background in computer language implementation,
implementing our own was pretty much a no-brainer, especially as I'd
been doing the design work towards a language like Magik for some time
before Smallworld was set up. It also meant that we could consider
point 1 and make the syntax a little more end user friendly. Two of us
implemented the first version of the core language (roughly equivalent
to ST-80 in capabilities) in 16-18 man-months.


Smallworld went on to become market sector leader and a multinational
with a nine digit US$ turnover so we must have done something right.
Just don't start me on what we did wrong. :-)


I'm afraid I don't have the (personal) bandwidth available to join 
in on LtU, but if you wish to quote this email there, feel free. 
Just obscure my email address, I get enough spam as it is.

PPST ran on more than Suns at the time, and it had a cross platform GUI even at that time better than Swing.

The pricing was ridiculous, and especially ironic these days, where Cincom has extremely flexible pricing and cannot give PPST away to teams like this for new development.

Back then ParcPlace had visions of a system so good that enterprises would fall all over each other with large amounts of money for enterprise-wide development licenses. Well, it was the best IT development system around. But hardly anyone was aware of it.

IT developers were busy with crippled tools like VisualBasic and PowerBuilder that could do 1/10th of what PPST could. Better pricing, and seeding some key entreprenuers could have had more effect on the industry at the time.

As it turned out, Smalltalk gained popularity over the next seven years or so, due to its adoption by IBM. But just as Smalltalk was getting some real press and word of mouth, everything changed with the arrival of Java. This is also ironic, since Sun tried to license Smalltalk from ParcPlace at lower prices in order to use it in small devices. The answer was no, and so Gosling invented Java (nee Oak).