Lambda the Ultimate

"Other endangered languages express unusual grammar, challenging Noam Chomsky's famous assertion 50 years ago that a universal grammatical structure is hard-wired into the human brain. For instance, the 300 Hixkaryana people in Brazil use a sentence structure in which the object comes first, verb second and subject last...

A few languages appear to make no verbal distinction between the familiar parts of speech. Words can serve interchangeably as nouns, adjectives and verbs but the meaning is clear from the context...

an endangered Australian language in which people use the spacial concepts of north, east, south and west - rather than left, right, front and back - to describe their immediate surroundings and body parts."

Are programming language designers being imaginative enough?

No.

A few languages appear to make no verbal distinction between the familiar parts of speech.

OK, I'll bite.

These kinds of claims are typical of people who haven't got a linguistic clue. The first does nothing to threaten UG or Chomsky, the latter is a misunderstanding arising from a limited exposure to non-European lanaguages.

As for what it tells us about PLs: can anyone can tell me what advantage there would be to write the operands before the operator in a function call, or to use a number as operator?

Are programming language designers being imaginative enough?

Why should PL designers be more imaginative?

What benefit is there in imaginativeness for users of PLs (as opposed to geeks like us who would enjoy studying freaky PLs. ;-) )?

Expressiveness (in the PL sense). Especially domain specific.

This includes things likes static checks (think about designing type systems. HM wasn't handed down from Mount Sinai...); code that's easier to read and write (e.g., structured contol constructs. Yep, these too were invented somewhere along the line..) as well as domain specific language constructs (e.g., think about PIC).

To change evaluation order, or to improve readability when the function is an anonymous literal function, or just to give the language a more OO feeling.

...or to use a number as operator?

Obviously that would mean repetition.

Why should PL designers be more imaginative?

Because language design is an evolutionary process, and evolution speeds up when there is more randomness.

surely there's an optimal amount. too random and you are dominated by noise; not random enough and you get stuck in local minima. you need to argue that we aren't already at the optimal point and i doubt you can provide objective support for that.

(sorry, feeling pedantic)

So do the same old things with a twist? Fun yes, but useful...?

code that's easier to read and write

I would love nothing better. However, we still await a good definition of what measurable characteristics lead to "easy to read and write", or even if there is a universal set of characteristics that fill this bill.

Without that, the only principle seems to be "variety is the spice of life", which hasn't necessarily led to better programming languages, but plenty of flamewars.

How is that a measurable advantage? I continue my quest to make "human factors" more objective...

Obviously that would mean repetition.

You must be a vi user. ;-) I don't see that this is obvious unless we unimaginatively follow the design decisions of previous generations of Unix geeks.

Because language design is an evolutionary process, and evolution speeds up when there is more randomness.

That sounds like a fallaciously teleological version of evolution to me. It takes for granted that the "fitness function" for PLs would reward the kind of language LtU readers would favour. I think the history of popular PLs would refute this. ;-)

On the contrary, I'm hoping to find out what objective improvements you believe can be had by PL designers being more imaginative.

Or put differently, what problems exist in current PL designs that you think are the result of lack of imagination. (As opposed to a lack of a solution.)

I am not sure I understand or agree with this distinction. The only case were these two things are different is when we can prove that no solution exists. As you are wont to remind us, proving anything as regards "human factors" is essentially impossible.

(Quickly running for cover). :-)

So I guess what you mean when you say PL designers need more imagination is really that we need to have the next great PL discovery.

I'm sure we can all agree with that. ;-)

It would probably be just the thing if you wanted to find the perfect algorithm for shopping. ;-)

I agree with andrew: too much mutation and the next generation will never inherit the beneficial adapations of its ancestors.

Also, language design is not entirely an evolutionary process; the key word `design' is not entirely compatible with the idea of natural selection, which is a spontaneous, undirected process. (Unless, I suppose, you are some sort of secular creationist.)

Marc: That sounds like a fallaciously teleological version of evolution to me. It takes for granted that the "fitness function" for PLs would reward the kind of language LtU readers would favour. I think the history of popular PLs would refute this. ;-)

Marc conflates fitness with popularity. Nothing ever changes on LtU...

Or put differently, what problems exist in current PL designs that you think are the result of lack of imagination.

All of them? Every time someone generalizes some language feature they are employing their imagination, thinking outside the box. I agree it's trivial and unhelpful to say, "We need to be more imaginative," but it's not untrue.

What I don't like is when people mistake novelty for innovation. I can think of, oh, ten or twelve scripting languages that evoke this feeling in me. How many different syntaxes for Scheme do we really need? This situation is symptomatic, I feel, of too much random mutation and the view that language design is purely evolutionary.

[A women-only programming language] would probably be just the thing if you wanted to find the perfect algorithm for shopping. ;-)

Man, you clearly want to spend the night in the dog house, don't you...! :)

Quite right. When I answered "yes" to that question, I wasn't offering this answer as a research agenda...

Agreed. But do you (and Andrew) think we are currently anywhere close to too much mutation??

How many really new species of languages have you seen lately? Most are simply a different race of the same species. I wish there were more wildly mutated languages that are doomed to die, but that at least have some really original traits.

And Frank grumpily rejects popularity as a valid metric. You're right, nothing does ever change... ;-)

But seriously, the key metric that we would use to evaluate an organisms evolutionary fitness is precisely its "popularity", i.e. how robust its population is. You could qualify this with the duration of its existence, with its size relative to other populations, with the diversity of environments it inhabits, but all of these factors are present in our perception of popularity too.

So I do believe that the popularity of a PL is one valid metric of its "success". (Obviously not the only one, or even the most interesting one, but valid nonetheless.)

I'm perfectly willing to entertain other "fitness functions" for PLs, but I want to have them explicitly articulated and subject to critique.

I would go so far as to say that finding and evaluating fitness functions for PLs is a fundamental part of what LtU is about.

Man, you clearly want to spend the night in the dog house, don't you...! :)

Luckily my wife has a good sense of humour ( and doesn't read LtU ;-) )

Probably not, but I think vast majority of the `mutations' we do see are trivial permutations of syntax, which have hardly anything to do with `fitness' (actually utility, not fitness---see below).

Marc: But seriously, the key metric that we would use to evaluate an organisms evolutionary fitness is precisely its "popularity", i.e. how robust its population is.

Fundamentalism, anti-semitism and Nazism are all extremely successful memes, and so perhaps very fit. Yet, I don't think their popularity reflects anything about their `quality' or the desirability of their adoption, e.g., I don't think being a fundamentalist or whatever will improve your quality of life or productivity, though it probably will make it more likely that you children and friends will become fundamentalist. Similarly, by these standards, C and Perl are extremely fit, yet I don't think adopting them will improve the quality of your programs, though you may find it easy to convince your fellow programmers to adopt them.

I suppose you are right that language fitness and popularity go together, but I think I was wrong to regard fitness in this sense as correlating with quality or usefulness in the first place. Fitness in the evolutionary sense only has to do with self-propagation. Indeed, I am about ready to abandon this evolution metaphor altogether.

i was wondering about this after i posted. i really don't know. i thought of two arguments against more mutation:

- we already have complaints that many languages are too academic, with too few commercial users.

- there are more languages out there than you may expect - i'm thinking of the guy who seems to turn out a language every year or so (the one with several visual languages that have been discussed here in the past, but whose name escapes me at the moment).

I wish there were more wildly mutated languages

i don't think mutation is the word here. mutation only gets you variation within the space over which the genes have effect. for biological systems the results may var wildly, even though the genes change in a relatively simple way, because of the complexity of the development environment, which supports horribly non-linear feedback.

however, if you consider how you would model programming language evolution genetcially, the "distance" between your genes and the final product is much smaller (you could almost have an "oo" gene, for example). so no matter how wild the mutation, you end up with something similar.

this ties in with the parallel discussion here about imagination being important.

in other words, to get something wildly different we either need to dream up the different "out of nothing" (not by evolving what we have), or we need a much deeper theory of programming languages, which separates the "genetic variables" from the resultant language by a much richer process than i imagine existing today.

You aren't trying to trigger Godwin's Law, are you? ;-)

I don't think being a fundamentalist or whatever will improve your quality of life or productivity

On the contrary, I think most fundamentalists would perceive that their life IS improved by their beliefs. (That we may not agree probably wouldn't faze them ;-) )

I am about ready to abandon this evolution metaphor altogether.

I think it has its limits too. This still leaves us with the question of what relatively objective criteria we can use to evaluate the quality of a PL.

That's part of the problem: popularity maybe a lame criterion, but it is one of the few that is fairly objectively determinable.

Frank: Probably not, but I think vast majority of the `mutations' we do see are trivial permutations of syntax, which have hardly anything to do with `fitness' (actually utility, not fitness).

Why do you say "but"? These trivial permutations are caused by lack of imagination/randomness.

Andrew: - we already have complaints that many languages are too academic, with too few commercial users. - there are more languages out there than you may expect - i'm thinking of the guy who seems to turn out a language every year or so (the one with several visual languages that have been discussed here in the past, but whose name escapes me at the moment)

That would be Wouter van Oortmerssen, he crossed my mind too when thinking about this topic. I'm an absolute fan of his work. Creating a new language every year, based on the experience of previous work is the perfect way to advance programming language design. It also prevents the languages to become too academic.

in other words, to get something wildly different we either need to dream up the different "out of nothing" (not by evolving what we have), or we need a much deeper theory of programming languages, which separates the "genetic variables" from the resultant language by a much richer process than i imagine existing today.

I don't agree. This sounds like the opponents of Darwin. They argued that an eye could never evolve from pure random mutations.

A small wacky idea is nothing on paper. When you implement it in a language, and play a bit with it, you get more feeling for the essence of the idea. This can then lead to a more general idea. Implement that again, and maybe you'll find it changes your whole programming style. Then you try to make a new language with the idea at the core and you end up with a completely new branch of languages.

no, it's from experience writing code that uses genetic algorithms. you need to devise some encoding - choose what your genes represent. at the moment i don't see any candidate for a sufficiently complex theory that gives you a basis for representing languages in a way that lets you find anything new.

for example, consider a very simple genetic representation, that uses two bits. one indicates whether we have compile time type checking and the other whether evaluation is lazy or strict. no matter how often you flip those bits you're not going to get anywhere. obviously you need something more complex. not just with more details, but with more layers of abstraction - the toy model i describe is equivalent to having a gene for eyes (seeing or blind).

i'm saying we simply don't have such a theory (presumably category theory is as good as it gets). in the case of nature it's not necessary because the biochemical environment that is its equivalent co-developed with the genes. that is not the case for what is being suggested here.

No, but I realized after posting that those examples are so inflammatory that they obscure my point, which is really pretty straightforward: we both walked right into one of the classic traps of evolutionary debate, namely interpreting `fitness' as `fitness to a purpose' rather than `likelihood of propagating'. For example, in the case of species/genes, it's wrong to say that more intelligent species or more predatory species are more fit than another unless you cans show that intelligence or being at the top of the food chain actually increases the likelihood of transmitting those genes.

Take humans, for example; we're the most intelligent, and at the top of the food chain. Yet are we the most fit? Not by a longshot, it seems; the most successful species are insects. They've been around millions of years longer, have hardly changed at all and far outnumber us; even if you measure biomass, they outweigh us substantially. (Maybe some bacteria are even more successful; I don't know, but the example of insects was pointed out to me once.) From an evolutionary viewpoint, they're much fitter than us, and the fact that humans are beautiful machines of much greater complexity, the fact that we have completely transformed our environment, the facts that we have language, culture and science are all irrelevant. (However, they may prove relevant in the end: if there is a human diaspora to space, and we start colonizing the rest of the universe, we will outnumber the insects... assuming, of course, we don't take them with us. Which is pretty unlikely, don't you think?)

But I think those features, which are irrelevant to evolution, are the important things about us. Similarly, thinking of programming language features as genes, the ones that are successful are the ones which propagate, which says nothing about their utility or value. To make the connection, you must assume that programmers adopt languages on a rational basis, and I think we all agree here on LtU (to some extent at least) that programmers are notoriously irrational about such things. A successful feature, by evolutionary standards, is simply a popular one, as you suggested. And so we have many languages with C-like syntax, imperative update, dynamic typing and OO features. Do these really contribute to productivity and utility? Possibly; but without additional evidence, all we can conclude from their success is that they are popular, and not that they contribute to `fitness' in a non-evolutionary sense.

You can, of course, argue that popularity affects productivity because of the network effect: more users means a larger codebase and library. Still, if you define `language' as the language itself, that is, without its library, implementation, documentation, etc., then popularity says nothing about quality or productivity. (I know some people prefer to define `language' more holistically; in that case just pick a different word, like `core language' or `syntax and semantics' for the concept I mean.) In fact, the utility of a large codebase is directly impacted by the language's ability to support reuse, and that is undeniably an intrinsic property, which has nothing to do with its popularity.

i think your problem is more that you're somehow expecting a single species to be "the best". it's not like that - evolution works within a complex environment and you're much more likely to end up with a population of many different species, all of which are best in some particular niche, and which together - hopefully - find a stable equilibrium. if you want to define "best" in some restricted way like "largest biomass" then you can do so, but it's rather meaningless - for example, since insects don't photosynthesise they're not going to be stable as a monoculture (if only insects existed, they'd die out, biggest first) (and also, the gene diversity is huge, so "insects" isn't one thing at all).

that's true for nature - if the environment for programming languages is "sufficiently complex" then it will be true for those too.

This is where we can get tricky. If we refine our notion of fitness to include environment (including ecological niche), we can argue that we are more fit within our environment or niche.

For example, there aren't any insects in the Arctic (that I know of) but there are people. Also there aren't any insects our size.

So even the supposedly clear-cut notion of fitness with respect to natural selection can be skew by relativity.

To make the connection, you must assume that programmers adopt languages on a rational basis, and I think we all agree here on LtU (to some extent at least) that programmers are notoriously irrational about such things.

While my heart agrees with you (I'm not too impressed with most things popular), I must concede that most programmers select PLs for reasons that are rational to them. (This is in line with my argument about fundamentalists).

When we say that they are being irrational, what we really seem to mean is that we think their criteria for evaluation are flawed because they lack important distinctions or because they are fixated on features that PLT geeks would find superficial.

(Incidently, this is precisely the feeling I get when I read articles like the one that started this thread where the author oohs and aahs over trivial phenomena such as OVS sentences or unmarked parts of speech that have nothing to do with the "hard stuff" in HL.)

Still, if you define `language' as the language itself, that is, without its library, implementation, documentation, etc., then popularity says nothing about quality or productivity.

If a lot of people are using the language productively, isn't that at least a modest and limited test of its effectiveness and quality?

Another thing to consider, drawing again on our evolutionary scenario, sometimes the cost of improving quality beyond "good enough" is too heavy for the quality-first organism or solution to be viable. Perhaps the very lack of the qualities that we LtUers would consider better are perceived as a feature to those who want to spend less time or mental capacity and still accomplish their practical programming goals.

there aren't any insects in the Arctic
Depends what you mean by the Arctic ;-)

most programmers select PLs for reasons that are rational to them
Not only that, we might also agree that their reasons are rational in their situation.

the author oohs and aahs over trivial phenomena
Journalists humouring linguists ;-)

drawing again on our evolutionary scenario... still accomplish their practical programming goals
Let's bring those threads back-together: for practical programming goals, the web of supporting systems and tools and libraries matters more than the language, fit within our environment or niche.

(As in, you can use any language as-long-as it runs on a jvm in the same process as Oracle...)

I hear they have a name for it.

Even an acronym.

LtU, I'm told

No, I don't expect that at all. I only brought up the insects example because insects are `simple' compared to humans, and mostly beneath our notice, to show that evolutionary fitness has nothing to do with more obvious metrics by which we might `evaluate' an organism. (I keep thinking of Shakespeare's Hamlet: "What a piece of work is man!")

OTOH, population or biomass or something along that line seems to me like a reasonably good way of measuring the ability of a gene to propagate itself, its fitness. As both those figures are linear orders, you do get a notion of `best' . How artificial or not it is doesn't really interest me, because evolutionary fitness doesn't really interest me.

Marc: If we refine our notion of fitness to include environment (including ecological niche), we can argue that we are more fit within our environment or niche.

Fine, but that notion of fitness still says nothing about quality or utility, only ability to propagate.

While my heart agrees with you (I'm not too impressed with most things popular), I must concede that most programmers select PLs for reasons that are rational to them.

By this definition, everyone is rational, even a lunatic. Besides, your claim is belied by the oft-cited eruptions of `religious wars' over programming languages.

But, very well, I will indulge you; let's say rather that most programmers are ignorant. My original point, you recall, was that programmers do not select languages in a rational way, so language popularity is not a good metric of language utility. Now I mean to say that programmers do know how to evaluate a language, so cannot select a language in an informed manner, so still language popularity is not a good metric of language utility. (To me, it's not much of a difference: to make a rational choice, you must be informed about your options.)

When we say that they are being irrational, what we really seem to mean is that we think their criteria for evaluation are flawed because they lack important distinctions or because they are fixated on features that PLT geeks would find superficial.

Yes. (And, FYI, I am no `geek'. :)

If a lot of people are using the language productively, isn't that at least a modest and limited test of its effectiveness and quality?

Productively compared to what? A stone axe is a productive tool for a caveman, but not by modern standards.

Another thing to consider, drawing again on our evolutionary scenario, sometimes the cost of improving quality beyond "good enough" is too heavy for the quality-first organism or solution to be viable.

I don't want to get into the 90% solutions thing. Or the cathedral versus bazaar thing.

Perhaps the very lack of the qualities that we LtUers would consider better [is] perceived as a feature to those who want to spend less time or mental capacity and still accomplish their practical programming goals.

Certainly. Lack of static typing comes to mind. But just because people perceive it as a non-feature, doesn't mean it is a non-feature. IMO, static typing solves a problem which arises in almost every program; if you don't exploit static typing, you still have to solve the problem, only you are doing it blindfolded and reinventing the wheel.

Allow me to reiterate Dijkstra's Next Fifty Years (the italics are mine):

It is time to unmask the computing community as a Secret Society of the Creation and Preservation of Artificial Complexity. And then we have the software engineers, who only mention formal methods in order to throw suspicion on them. In short, we should not expect too much support for [sic] the computing community at large. And from the mathematical community I have learned not to expect too much support either, as informality is the hallmark of the Mathematical Guild, whose members --like poor programmers-- derive their intellectual excitement from not quite knowing what they are doing and prefer to be thrilled by the marvel of the human mind (in particular their own ones). For them, the Dream of Leibniz [of "presenting calculation, i.e. the manipulation of uninterpreted formulae, as an alternative to human reasoning"] is a Nightmare.

He is talking about mathematicians, but this applies also to programmers who labor under the illusion that the lack of a feature such as static typing can somehow be beneficial.

Oh, and lest someone misunderstand me, this assumes a situation with two languages X and Y and a feature F, where X lacks F and Y has it, and there is a translation of X into Y. For example, if F is static typing, the translation may be the one using a universal type, which I showed once. If F is imperative update or more generally side effects, the embedding may involve a monad. (So, yes, I am saying that a pure functional language without any way of performing side effects is worse than one which has monadic side effects. This is the sort of point Hughes makes in Why Functional Programming Matters: it does not make any kind of sense to say X is better than Y because it lacks F.)

You seem to be presupposing some Platonic Ideal of quality or utility. I'm contending that the ability to propagate IS a particular measure of a particular type of utility and quality, and that furthermore it has the nice property that you can measure it objectively.

Besides, your claim is belied by the oft-cited eruptions of `religious wars' over programming languages.

I take it for granted that the rational choice for one group may be at odds with the rational choice of another group.

so still language popularity is not a good metric of language utility

OK. So what are you proposing as a better metric? I agree it is desirable that there be such a metric, but I have never found one that did not rest on fuzzy human factors, and my quest to make said factors less fuzzy has so far drawn a blank. ;-)

Productively compared to what?

Compared to doing no programming because you lack a language or because it is too complex, or too weak, or too badly implemented to accomplish your goals.

it does not make any kind of sense to say X is better than Y because it lacks F

I don't think I buy this, since you can always rephrase your claim to make the negative a positive.

So for example, you can say that a language is good because it has dynamic typing, whereas the other is bad because it lacks dynamic typing.

Likewise, somewhat more subtly, you can say that a language is good because it has type inference, but really why that is good is that it lacks the requirement to explicitly type all expressions.

(For the record, in case anyone has forgotten, I favour static typing (and even explicit typing) because it increases the programmer's understanding of the program.)

No, I'm not; but I can think of a number of meaningful ways in which to compare programming languages which most people would agree impact on quality and utility; ability to propagate is not among these ways. If you believe that popularity says anything about quality or utility, then you have to make a case for it, and I don't think you have.

I'm contending that the ability to propagate IS a particular measure of a particular type of utility and quality

So far, all the evidence I've heard from you on this score is circular: "it's popular, therefore it must be useful." Well, yeah, if you already accept that as an axiom.

furthermore it has the nice property that you can measure it objectively.

Other standards of utility, for example relative expressiveness, can also be rendered objectively, and have a much clearer relationship to utility.

I take it for granted that the rational choice for one group may be at odds with the rational choice of another group.

You also seem to take for granted that everyone is rational. By your definition of `rational', can you name an instance in which any person has ever behaved irrationally?

So what are you proposing as a better metric?

Gee, I dunno, exotic things like the ability to support code reuse, express abstractions, express and enforce invariants, definability, soundness, completeness and so on. You know, the sorts of things one might find discussed in a programming language textbook?

OTOH, maybe you're right; I never saw much use in scientific literature myself...

Marc: If a lot of people are using the language productively, isn't that at least a modest and limited test of its effectiveness and quality?

Frank: Productively compared to what?

Marc: Compared to doing no programming because you lack a language or because it is too complex, or too weak, or too badly implemented to accomplish your goals.

First, implementation quality is not a property of a language.

Second, there are plenty of unpopular languages that are `better' than popular languages and not too complex or too weak to do programming. Modula-3, Haskell, SML, Ocaml, Clean, Scheme. So, no, I still don't buy the popularity argument.

Frank: it does not make any kind of sense to say X is better than Y because it lacks F

Marc: I don't think I buy this, since you can always rephrase your claim to make the negative a positive.

Come now, you know what I mean. Something like: `If there is a `sound' but `incomplete' translation of X into Y, then the set of Y-programs which are not definable in X describe a feature F which Y possesses but X lacks.'

So for example, you can say that a language is good because it has dynamic typing, whereas the other is bad because it lacks dynamic typing.

By the definition above, dynamic typing is not a feature (or rather it is a trivial one), since:

1. if a programming language supports static typing it necessarily supports dynamic typing,

2. untyped lambda-calculus is dynamically typed,

3. by definition, every programming language is Turing-complete,

4. if a language is Turing-complete, then there is a translation from untyped LC into it.

So every programming language supports dynamic typing; it's trivial.

Likewise, somewhat more subtly, you can say that a language is good because it has type inference, but really why that is good is that it lacks the requirement to explicitly type all expressions.

All bound variables, you mean.

But I think type inference is genuinely a non-trivial feature, since we know for a fact that there are languages for which it is undecidable, like polymorphic lambda-calculus, and supporting type inference for a fragment does not preclude allowing explicit annotation in other places (as recent versions of GHC support). I think I can exhibit it as a feature using the framework I outlined above, but I'm not sure how off the top of my head.

Update: I edited out a failed attempt at exhbiting type inference as a feature.

I hate to break it to you, but "most people would agree" is an appeal to popularity as an argument. I'm OK with that, but would consider it that you had proved my point if it is admissible on your side. ;-)

So far, all the evidence I've heard from you on this score is circular: "it's popular, therefore it must be useful." Well, yeah, if you already accept that as an axiom.

You seem to be treating "popularity" as some atomic and primitive state that randomly falls on some things and not others.

In the case of a PL, "popularity" precisely means that a large number of programmers find it useful to use that language. This is pretty much the very definition of utility. I take as a given that people wouldn't use a PL if they couldn't do the things they wanted with it, i.e. it didn't have utility for them.

Are there languages with COULD provide more utility to them? Maybe, but it they don't actually need that utililty, why should they rationally care?

Other standards of utility, for example relative expressiveness, can also be rendered objectively, and have a much clearer relationship to utility.

Let me provide a counter-example. Imagine a dishwasher (shades of past appliance threads ;-) ) that has a button on it to ring your doorbell. It costs $100 more than the the model without the doorbell feature. It has as a subset all the functionality of the plain model, plus one piece of functionality that is provably not on the plain model. Does that make it a better (higher quality, higher utility) model? Would it be more rational to spend the extra $100?

By your definition of `rational', can you name an instance in which any person has ever behaved irrationally?

I would consider it irrational for someone to insist that most other people are irrational. ;-)

Gee, I dunno, exotic things like the ability to support code reuse, express abstractions, express and enforce invariants, definability, soundness, completeness and so on. You know, the sorts of things one might find discussed in a programming language textbook?

I raise my sarcasm shield to full. (Phew, that was close... ;-) ) While I agree whole-heartedly that all of these features are of interest and non-trivial for PLs, there is a difference between asserting "has X" and "having X is necessarily better". (See dishwasher example)

So every programming language supports dynamic typing; it's trivial.

Which comes back to my point that what people really mean when they say "dynamic typing is a feature" is that they like not having to bother with type. We can both disagree with them, because we value other things that make the "bother" worthwhile, but that doesn't invalidate their preference based on their needs or demonstrate that our preference is necessarily more rational.

There is a clear difference between peer review in scientific circles and popularity contests.

You seem to be treating "popularity" as some atomic and primitive state that randomly falls on some things and not others.

Not randomly. Popularity is affected by marketing and evangelism, eye candy, familiarity, undoubtedly a slew of other factors and, yes, even quality and utility. But using popularity as a metric for the last two is clearly not a good tactic when there are so many other variables.

In the case of a PL, "popularity" precisely means that a large number of programmers find it useful to use that language.

No, it doesn't! I don't understand how you can make such an unwarranted claim, much less so abuse the word `precisely'!

Are there languages with COULD provide more utility to them? Maybe, but it they don't actually need that utililty, why should they rationally care?

How do you, or they, know that they don't need it? Most programmers are barely even aware of the existence of languages besides C++, Java and Perl.

Let me provide a counter-example. Imagine a dishwasher (shades of past appliance threads ;-) ) that has a button on it to ring your doorbell. It costs $100 more than the the model without the doorbell feature. It has as a subset all the functionality of the plain model, plus one piece of functionality that is provably not on the plain model. Does that make it a better (higher quality, higher utility) model? Would it be more rational to spend the extra $100?

Granted, the dishwasher's utility depends on whether the buyer needs the missing functionality (you wrote `subset') and whether you lack a doorbell button; its quality doesn't. The quality of an object does not depend on its price, your circumstances, the phase of the moon, how you feel about gay marriage or Dolly Parton's hairstyle; it is an intrinsic characteristic.

But your example is bogus because the additional expressiveness of good languages (by my definition of `good') is almost always useful, because it addresses general problems in computation that arise in all non-trivial programs, and not accidents which depend on the particular application domain or input or whatever. (And this is why I don't advocate putting domain-specific features in programming languages.) In my suggestion to gauge language utility via translations between them, non-computational language features will not be expressible anyway (assuming a reasonable categorical setting).

I would consider it irrational for someone to insist that most other people are irrational. ;-)

Now you're putting words in my mouth. I only ever said that some people tend to behave irrationally in some circumstances.

I raise my sarcasm shield to full.

Well, you are always a good sport about my sarcasm. But you richly deserved that salvo. :)

While I agree whole-heartedly that all of these features are of interest and non-trivial for PLs, there is a difference between asserting "has X" and "having X is necessarily better". (See dishwasher example)

Certainly there is a difference. But at least I can suggest a causal relationship between instrinsic and well-defined features of a language and its utility as a tool. Your insistence that popular languages are good ipso facto is not only unsupported, subjective and ill-defined, but it doesn't even hint at any causal relationship. Hell, you might as well claim that language X is good because its name has pleasant numerological qualities; that would be about as easy to defend.

Which comes back to my point that what people really mean when they say "dynamic typing is a feature" is that they like not having to bother with type[s].

Sure.

We can both disagree with them, because we value other things that make the "bother" worthwhile, but that doesn't invalidate their preference based on their needs or demonstrate that our preference is necessarily more rational.

This is not about `invalidating their preference', or taking away their First Amendment rights or any hippy nonsense like that. Really, you are starting to amaze me. Did you just take a course in postmodernism or situationist discourse? You need to read about the Sokal Affair because you are starting to sound like Social Text.

If a man steps out of a thirtieth floor window, he falls and makes a mess on the pavement below. It does not matter whether he prefers to float upwards, what religion he espouses, or what his opinions are on the economic modalities in (especially) the southern colonies prior to the Revolutionary War.

Similarly, the utility (or non-utility) of static typing does not depend on whether you are terminally allergic to types or want to have desperate, passionate sex with them.

Having spent many years in a research environment, I'm not sure I can completely agree with that. ;-)

No, it doesn't! I don't understand how you can make such an unwarranted claim, much less so abuse the word `precisely'!

If you add "in the specific way that I'm using it for the sake of this discussion" to that sentence (I would have thought that was implicit) precisely works quite as intended.

How do you, or they, know that they don't need it? Most programmers are barely even aware of the existence of languages besides C++, Java and Perl.

My premise is that if they really needed it, they would be highly motivated to find it. That they don't seek it out suggests they don't really need it that badly.

The quality of an object does not depend on its price, your circumstances, the phase of the moon, how you feel about gay marriage or Dolly Parton's hairstyle; it is an intrinsic characteristic.

So here is the crux of our disagreement. I reject the notion of a universally applicable, inherent notion of "quality" or "good", that only a few really smart people can perceive.

Good or quality is relative to a particular purpose. (I have mentioned that I'm a philosophical Pragmatist before...)

I have my own "inherent" sense of "quality" that is 100% valid for me, and I happily live by it (even at the expense of the values of others if necessary), but I don't feel I can claim that this is the "one true objective quality" that all intelligent and suitably-educated people would obviously agree with. (Though ironically, we would probably agree more on these things than it might seem ;-) )

Similarly, the utility (or non-utility) of static typing does not depend on whether you are terminally allergic to types or want to have desperate, passionate sex with them.

Perhaps we are arguing terminology. I would agree with you that static typing has more "capacity", i.e. it COULD be used to do "more things".

Whether it is always useful for all programming tasks is the point that I think could be (and usually is) debated.

Marc: "My premise is that if they really needed it, they would be highly motivated to find it. That they don't seek it out suggests they don't really need it that badly."

Perhaps someone has a curable illness which is silently and systematically destroying their mind and body. If they knew they had the illness they would be highly motivated to find and take the cure. But they won't know until it's too late.

It's an unknown unknown.

Are there even any convincing measurements of the popularity of a PL?

Who knew not knowing Haskell was so bad for your health? ;-)

Two economists are walking down the street when they both spot a twenty-dollar bill lying in the gutter. One of them stops to pick it up and the other one says, "What are you doing, George?" "Well, there's this twenty here in the gutter." "C'mon, George, don't be silly -- if it were really there someone would have picked it up already."

(From George Stigler's autobiography, which must have told it better.)

(what follows is all based on intuition, no proofs)

There are two types of programmers: those who like to solve problems, and thos who just like to be in control.

Problem solvers like bugs, it's a new problem to set your teeth into. Control freaks hate bugs, they just lost a bit of control.

Problem solvers like high level languages, those languages let them focus on the problem better. Control freaks like low level languages, those languages give them a finer grained control.

And now the relevant part to this discussion:

Problem solvers like learning other languages. New languages give opportunities to solve more problems than before. Control freaks hate learning other languages. Switching to a new language means a huge drop in control.

Historically programming was only for the control freaks, code needed fine tuning to be of any use. That's why a lot of senior programmers stick to C, they are control freaks.

Hardware gives the problem solvers more and more the advantage, so ultimately the low level languages will mostly disappear. But this will take a while as long as the older control freak generation make the important decisions.

Conclusion: language popularity is not a good metric.

Certainly, there are programmers that get more of a charge out of solving problems than other programmers do. And there are programmers who are more strongly motivated by controlling the solution. But those seem like two orthogonal dimensions, not the endpoints of a single dimension.

So there are really four kinds of programmers. :)

And some of your conclusions seem wrong:

Problem solvers like bugs, it's a new problem to set your teeth into. Control freaks hate bugs, they just lost a bit of control.

And I would characterize "control freaks" as the sort of people that would rail against any attempt to prevent bugs, because it necessarily removes some of their control. Have you ever heard programmers say things like "I shouldn't have to suffer because of idiots who write buggy code", or "really, the compiler should trust me that this is correct, not the other way around".

Problem solvers like high level languages, those languages let them focus on the problem better. Control freaks like low level languages, those languages give them a finer grained control.

C is a good example of a language that does not respect a programmers abstractions (removing control from the programmer). The language doesn't even respect its own abstractions. So the latter sort of "control freak" is the type who would need a very very good reason to develop in C.