Lambda the Ultimate

(Alludes to a story recounted by the professor in my college compilers class, about wanting to fill in an error entry in an LR parser table with "error in language design".)

Isn't this an error in the design of the language syntax?

if (foo())
  goto end;
  goto end;

important and useful stuff...

end: cleanup

The syntax of the language ought to require some sort of explicit, clear terminator for the consequent clause. If it did, the language processor would object to the missing clause-terminator — if, indeed, the obviousness of the missing clause-terminator didn't prevent the programmer from making this mistake in the first place. Evidently the desirability of such a clause-terminator is orthogonal to whether the language happens to have goto. (Dangerous things should be difficult to do by accident.)

The error was conducted using mature technologies and processes that are supposedly larded up with all kinds of safeguards against stupid mistakes like this doubled "goto". The safeguards failed and failed with a huge footprint.

I don't think there is a lesson here for how to tweak language design. I think it is arguably socially irresponsible to suggest that the problem is just a language design defect.

I think the lesson should be (but won't be) that we are far too reliant on the non-existent robustness of these systems and that there is no practical solution to that other than to stop relying on them in the ways that we do.

I don't think there is a lesson here for how to tweak language design. I think it is arguably socially irresponsible to suggest that the problem is just a language design defect.

Depends perhaps what one considers a language defect. Certainly if your language required a logical specification along with the implementation, these sorts of errors would be virtually impossible, particularly if two different people implemented each one.

Certainly if your language required a logical specification along with the implementation, these sorts of errors would be virtually impossible, particularly if two different people implemented each one.

I think what you've said amounts to this: "We can design tools and practices that are much more expensive to use than the ones Apple was using and in theory those tools and practices will make bugs a bit like this less likely."

We already knew that, though.

What we're seeing here, though, is what the market actually chooses and how that actually makes so much of our economy and society extremely fragile to the extent it relies on this software.

So I stand by my conclusion: the lesson is that we're using computers wrong.

It doesn't change anything to say that a much more expensive-to-use programming environment could avoid certain errors because if there was a massive shift to raise the costs of software that way, it would entail us using computers a lot differently, I'm sure.

I wouldn't expect things to shift in favor of more-expensive, theoretically more robust tools. The rewards of keeping development costs down concentrate and accrue to software vendors. The costs of having the infrastructure fail is significant by diffuse.

I think what you've said amounts to this: "We can design tools and practices that are much more expensive to use than the ones Apple was using and in theory those tools and practices will make bugs a bit like this less likely."

A lot less likely, and not just bugs like this. As I'm sure you know, part of the research of the past 30 years has been reducing the specification effort to bring it within the bounds of an exploit cost for the software vendor. It's gotten much better, and the full specification of the L4 microkernel has shown that it's now within certain software budgets to do so. It's only going to improve from here. I don't think you can infer "don't rely on computers" when the practitioners are using methods that are 40 years out of date.

What we're seeing here, though, is what the market actually chooses and how that actually makes so much of our economy and society extremely fragile to the extent it relies on this software.

We already knew this too though. This lament probably goes back at least to Dijkstra's "GOTO Considered Harmful".

Not "just" a language design defect. I was making no remark, one way or another, about those "mature technologies and processes". My contention was, and is, that it's always better, when possible, to do something really simple at an earlier stage in the design that would make the whole system less likely to produce an error rather than depending on some additional measure to catch the error after it's produced.

Any language that requires the use of external safeguards to take care of simple things like unreachable code is broken by design.

Static analysis should be part of the language design, where all possible warnings are dealt as errors, and not relegated to an external tool.

As real world examples show, average developers tend to shy away of such tooling, with consequences like the one being discussed.

Exactly, and this whole separation of powers, judicial independence is a bad idea; let's do government the Chinese way where everything is under party control.

I'm the biggest advocate of language/tool codesign, but sometimes you can't do everything perfectly in one waterfall-like plop. Sometimes you just do the best you can, focus on a few areas, and leave it up to success and later tooling to handle the rest.

I don't think there is a lesson here for how to tweak language design.

I think there clearly is. Although it's arguably not "just" bad language design, that is a major culprit still. And the second lesson is that tooling, process, and other safe guards cannot make up for that.

First of all, the bug is slightly more subtle. The broken code actually was something like:

OSStatus err;

// ...lots of code

if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
    goto fail;
    goto fail;
if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
    goto fail;

// ...more checks

fail:
    SSLFreeBuffer(&hashCtx);
    return err;

So what happened here was that the first check was successful, setting err to success by side-effect, and then the remaining checks were skipped -- ultimately returning success.

Now, it seems pretty obvious to me that at the core, this exemplifies the bad interaction of two language features that are central to imperative programming and whose danger we've known for ages: mutation and jumps -- made worse by some poor design choices in C's syntax. (And as an aside, the uninitialised error variable doesn't make it any better.)

I don't think a bug like that could happen with a more functional language design, where data flow cannot arbitrarily be messed with by control flow. (And no, it's not just dreaded goto, the above would 'work' just as well with break or similar 'structured' jumps.)

Also, simple tooling fixes are not a solution. In particular, unreachable code warnings won't help in similar cases. Just imagine a slight variation:

if (some_condition) {
  // ...lots of code
  if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
      goto fail;
      goto fail;
}
if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
    goto fail;

No unreachable code, still boom. Notably, nested checks like this can be found in the same source file, even in the same function.

I agree that better languages won't ever remove all dangers. But they sure can remove a lot of them. But we keep being stuck in the 60s.

Thanks for the refined analysis, I had indeed missed the interaction with the assignment-as-error-report aspect, which makes the failure more interesting.

if (some_condition) {
  // ...lots of code
  if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
      goto fail;
      goto fail;
}

A warning about misleading indentation could raised here.

...although it won't if the indentation wasn't wrong (which seems a case just as (un)likely).

The GnuTLS demonstrates yet another failure that would be addressed by a saner language: silent coercions between integers and booleans.

...is implicit coercion between pointers and booleans. I have introduced the following bug more than once myself:

int g(Z, bool* ok);

int f(X x, Y y, bool* ok) {
  // ...
  int n = g(z, ok);
  if (ok) {
    // ...
  }
}

This essentially is the encoding of an error monad. You need this particularly crude style of emulating it because C has neither option types nor tuples nor exceptions nor sane out parameters (and even in C++ there are often reasons to avoid them because they are so kludgy). A bug like the above is very difficult to spot. Unfortunately, using null pointers as falsy values also is encouraged by the language, and so common that respective warnings tend to generate too many false positives to be useful.

I don't think there is a lesson here for how to tweak language design.

I think there clearly is. Although it's arguably not "just" bad language design, that is a major culprit still.

I think the underlying claim you make is that a language tweaked for this one kind of bug is better than a language not so tweaked.

Apple's bug and bugs like it are socially disastrous.

Do you think any tweak to languages will make such socially disastrous bugs significantly less probable anytime soon and in an economically effective way?

I don't and don't see why anyone would.

Language design that resorts to "this is better in theory, if only people would resolve to act against their economic interests and use these tools differently" is pointless or worse.

if only people would resolve to act against their economic interests

Socially disastrous bugs aren't in a company's economic interest, either.

Apple's bug and bugs like it are socially disastrous.

That's one classification for it. OTOH, we could also call it a "confusing syntax" bug or a "dead code was not annotated" bug.

Do you think any tweak to languages will make such socially disastrous bugs significantly less probable anytime soon and in an economically effective way?

Yes. A tweak that makes any common class of bug (scope errors, uninitialized variable errors, unhandled condition errors, etc.) significantly less probable will have commensurate effect on socially disastrous bugs. Of course, it is difficult to 'see' the bugs that were avoided.

Socially disastrous bugs aren't in a company's economic interest, either.

The value that they place on that problem by the amount they spend on it which is more than $0 but is not large, some sub-segments of the industry being exceptions that prove the rule.

A problem here is that the costs of socially disastrous bugs tend to be socialized. For example, in recent news quite a few retailers are implicated in IT system breakdowns that are now so numerous that people are questioning whether or not the payment system as a whole will be able to take a huge surge in identity theft. You'll have trouble, though, finding losses on the balance sheets of those firms that are commensurate to the threatened damage.

A tweak that makes any common class of bug (scope errors, uninitialized variable errors, unhandled condition errors, etc.) significantly less probable will have commensurate effect on socially disastrous bugs.

You have made, there, some assumptions I don't share:

1) That disastrous bugs are by-in-large the result of common classes of bug.

2) That introducing a language feature that reduces the frequency of common classes of bug achieves a reduction of the bug rate in actually deployed code.

The PLT project of treating as precious every new language feature that addresses a common class of bug seems to me premised on those same assumptions.

value that they place on that problem by the amount they spend on it which is more than $0 but is not large

True. It is not unusual for companies to misjudge the value of such things. It is difficult, in an accounting book, to assess damage to reputation. It is difficult even to assess damage from slipping schedules and delays.

You seem to be making an assumption that I do not share: that companies - that people who make decisions - are rational actors with sufficient knowledge and expertise to judge what is in their own best interest (without even the benefit of hindsight). Is this a common assumption in economics?

I believe the opposite, that it is most unusual for people to accurately assess value, cost, or risk. Waste is common. People do not act in their own economic interests, and often unwittingly act against it. Price, cost, and value are at best loosely correlated.

[you assume] That disastrous bugs are by-in-large the result of common classes of bug.

There are a lot of flawed concepts and designs that cost societies a lot. The Y2K issue, for example. But, when speaking of actual programming bugs, those do fall into common classes. Socially damaging bugs are very frequently just plain old common bugs in locations where they can do a lot of social damage.

[you assume] That introducing a language feature that reduces the frequency of common classes of bug achieves a reduction of the bug rate in actually deployed code

Indeed it does. Most bugs in deployed code are common bugs - e.g. off-by-one errors, failed null checks, accidental masking/shadowing, missing cases, state corruption due to partial failure, or uninitialized variables. When you eliminate these, they don't pop up as new bugs elsewhere. They're just gone. Of course, it is difficult to account for the bugs that didn't happen, much less trace the not-happening to a particular cause.

Addressing common conceptual error is also feasible, and oft the domain of DSLs or frameworks. But this is even more difficult to account, and not readily achievable by 'tweaks'.

You seem to be making an assumption that I do not share: that companies - that people who make decisions - are rational actors with sufficient knowledge and expertise to judge what is in their own best interest (without even the benefit of hindsight). Is this a common assumption in economics?

As I recall, Alan Greenspan said this assumption was his mistake leading up to the recent economic, ah, unfortunate incident (you know, the one that sort of almost destroyed the global economy).

value that they place on that problem by the amount they spend on it which is more than $0 but is not large

True. It is not unusual for companies to misjudge the value of such things. It is difficult, in an accounting book, to assess damage to reputation. It is difficult even to assess damage from slipping schedules and delays.

I think you are illustrating "PLT blinders" very nicely:

I think you are starting from the premise that PLT work on langauge safety features must be quite valuable.

You also assume that PLT work on safety features is of a nature that its adoption or non-adoption will be reflected in measurable economic phenomena like better or worse corporate reputation and better or worse adherence to schedules.

Confronted with the empirical fact businesses (with some niches excepted) aren't much interested in this kind of PLT work, you are left having to conclude that their accountants must be keeping the books the wrong way.

I'm looking at it a bit differently.

I think most contemporary PLT work is inconsequential outside the field. I don't think that is a controversial claim. A high degree of inconsequentiality is not, in and of itself, a criticism.

I don't think contemporary PLT has any useful understanding of what makes for its occasional hits, developments that are actually of significant consequence outside the field.

Nevertheless, contemporary PLT has, internally, some intrinsic theories of relative value in PLT research.

For example, in some PLT contexts, advances that help facilitate machine-assisted proofs about programs are valued.

In other contexts, langauge safety features may be favored.

So there are internal working theories of "value" here.

Of course it is inevitable that PLT will have some internal sense of what is valuable and what is not but...

I'm highly suspect when PLT practitioners express values internal to the field that seem to assume PLT is consequential in ways which it is not.

Remember this started with John Schutt, looking at the source code at the heart of the Apple bug, suggested that it pointed out a language design error.

He specifically suggested a purported bug in syntax design. Implicitly he argued that a language without the purported bug is better than a language with the purported bug as evidenced by the disastrous Apple bug.

Socially disastrous bugs like the Apple bug are thus said to be something about which PLT can make a significant, consequential contribution. Yet in reality I don't see that happening both because PLT has trouble getting new ideas deployed and because PLT offers no assurance that the end is anywhere in sight as we knock down language design errors one by one.

You know if a prep chef cuts his finger chopping an onion we generally don't say that the knife has a design bug (unless it is a very unusual knife). Instead we're much more likely to say that the junior kitchen staff was handling the knife improperly -- needlessly holding his fingers so as to put them at such risk.

A different sense of value in the PLT world might look at the Apple bug and say "Hey, Apple was wielding that knife incorrectly -- needlessly putting so many people's personal data at risk." Perhaps the best way to view the bug is that Apple is selling computer systems that purport to do deserve a level of trust they can't ever possibly hope to earn, syntax tweaks or no.

Socially disastrous bugs like the Apple bug are thus said to be something about which PLT can make a significant, consequential contribution. Yet in reality I don't see that happening both because PLT has trouble getting new ideas deployed

I don't think this is the responsibility of PLT theorists. Why should it be?

Theoretical research is about what you can do. Such research doesn't set policy about what you should do. What measures actually yield positive results is largely an empirical queston, not a theoretical one.

That said, theoretical properties can give us good reasons to believe that particular measures will lead to certain types of results. Certainly more reason to believe in a reduction in bugs than doing nothing and repeating the same mistakes over and over again.

You know if a prep chef cuts his finger chopping an onion we generally don't say that the knife has a design bug (unless it is a very unusual knife).

Isn't C a very unusual knife? It's essentially a knife without a handle that's all sharp edges. You could certainly build yourself a handle, but why should you have to? Sounds like a design bug to me. ;-)

I don't think [addressing socially disastrous bugs like Apple's] is the responsibility of PLT theorists. Why should it be?

I hope it is clear that I also don't think it should be the responsibility of PLT. What I'm saying beyond that is that, because that isn't a PLT responsibility, PLT's internal concept of value should be different from what it is.

The recognition that bugs like this are not PLT's responsibility should be (but isn't) liberating.

I hope it is clear that I also don't think it should be the responsibility of PLT. What I'm saying beyond that is that, because that isn't a PLT responsibility, PLT's internal concept of value should be different from what it is.

I should have been more clear. I was specifically objecting to the last comment you made in the text I quoted, "Yet in reality I don't see [significant, consequential contributions] happening both because PLT has trouble getting new ideas deployed".

It isn't PLT's responsibility to market and deploy advances in its fields, its purpose is discovery. Would you criticize materials science because a car was made out of shoddy materials that are 40 years out of date, or would you blame the car manufacturer and the engineers for not using reasonably current materials science to make their vehicles safer?

The problem with industry programming is not with PLT, which can and does make languages safer, the problem lies elsewhere.

Would you criticize materials science because a car was made out of shoddy materials that are 40 years out of date,

I think this is a poor analogy. Materials science seems to be profoundly engineering-driven with a keen and accurate sense of real world impact. It has repeatedly successful, well understood pathways from theory to widespread practice. Some of its branches are very much concerned with automotive design and as far as I can tell the concept of valuable research internal to the field of materials science is well and effectively informed to how it is valued outside the field.

The problem with industry programming is not with PLT, which can and does make languages safer, the problem lies elsewhere.

Whence PLT's seeming, ongoing, emphatic obsession with safety, given that actual industry practice is not PLT's job?

It has repeatedly successful, well understood pathways from theory to widespread practice.

These pathways aren't part of materials science itself, as you seem to think PLT requires. Industry pays engineers to keep abreast of developments in materials science because it provides competitive advantage.

Where is the analogous incentive to do so with PLT, ie. paying software engineers to research, adopt and deploy PLT improvements? This is the point of my poor analogy: when a car fails in a way that was avoidable given knowledge at the time, we know who's at fault and who pays. There is thus incentive to use newer materials, and disincentive to use older materials, but there are paltry disincentives to using outdated languages because the costs are negative externalities which the software vendor doesn't pay (ie. the software consumers pay the price).

This isn't a failing of PLT where their developments have little utility to industry, but a failing of industry because there are little to no consequences for failures. In every other engineering discipline there's a regulatory framework that internalizes these externalities and makes the costs of compliance failure quite clear, a cost that the accountants you cite can now be made aware of. Is it any surprise the costs aren't so apparent when software lacks a corresponding framework? I'm not sure why you'd lay this at PLT's feet though.

Whence PLT's seeming, ongoing, emphatic obsession with safety, given that actual industry practice is not PLT's job?

Inspiration comes from many places. Safety of industry practices is of theoretical interest to some. Why wouldn't it be?

Inspiration comes from many places. Safety of industry practices is of theoretical interest to some. Why wouldn't it be?

If you want to say that PLT has evolved into a clique and that PLT has come to be mostly about some theoretical interests held mainly by its own members, then I agree. It is lately relevant mostly to itself.

It wasn't always so.

If you want to say that PLT has evolved into a clique and that PLT has come to be mostly about some theoretical interests held mainly by its own members, then I agree. It wasn't always so.

I haven't seen any evidence that PLT has any more or less industry focus than it has always had. Research always precedes application. Prior research just happens to look like current application because current application has stagnated, not because research is out of touch with the how it ought to be done. There is little incentive for application to follow research. This doesn't make PLT cliquey anymore than any other theoretical field when compared to their corresponding applied fields.

Research always precedes application.

_{(fans of hylomorphic deforestation (theory ca 1990) may enjoy reading $5.4 of Iverson's 1962 "A Programming Language"; on p.173 he notes:}

Each of the translation programs considered produces the resulting L-formula in reverse order. This is the order in which it is most easily evaluated and, consequently, the order in which the synthesis of a corresponding function program is most easily performed. Synthesis may therefore proceed concurrently with analysis. The analysis may, on the other hand, be completed first, and the output file Φ2 rewound before the synthesis. The latter alternative allows the use of separate metaprograms for analysis and synthesis, and hence makes lesser demands for metaprogram storage. It also allows the application of an intervening transformation of the L-formula to some preferred equivalent form. However, as shown by Program 5.4, the transformation to minimax form also treats the L-formula in reverse order. It can therefore be performed concurrently with the translation from parenthesis notation.

I think this counterexample equivocates on "research". Newcomen himself did "research" before achieving something workable. Just because it isn't published, or isn't theory-driven, doesn't mean it isn't research. If you wish to define "research" in a narrower sense requiring theory or publication, then "always" is indeed an inappropriate word choice, but that's not the meaning I intended.

Confronted with the empirical fact businesses (with some niches excepted) aren't much interested in this kind of PLT work, you are left having to conclude that their accountants must be keeping the books the wrong way.

Accountants misjudging value of things is empirical fact. We don't need PL theory to explain that. The impact of choosing different programming languages, i.e. whether this language would have led to more overheads than that one (and thus the true cost of the PL) is just one of the many things accountants cannot correctly account for.

I don't think contemporary PLT has any useful understanding of what makes for its occasional hits, developments that are actually of significant consequence outside the field.

Yes. I agree with this.

It would be the case even if technical merit was a primary arbiter of PL success. Of course, impact of a PL and PL theory is also determined by many things unrelated to PL - path dependence, network effects, licensing, accessibility, human resistance to change, etc.. Many of these factors are also economic in some aspect, and add noise around any signal from economic impact of technology.

I'm highly suspect when PLT practitioners express values internal to the field that seem to assume PLT is consequential in ways which it is not.

I'm okay with rejecting assumptions.

But to go the other direction - to assume PL decisions are inconsequential without proof - is far more absurd. In reality, most decisions have consequences. Sometimes they're subtle or latent or emergent. "There will be consequences!" is just a slightly sinister way to say "true".

Further, I would not characterize every argument about PL consequences as assumptions. Many of them do have considerable evidence or rational argument.

PLT offers no assurance that the end is anywhere in sight as we knock down language design errors one by one.

Two points:

• We don't need an 'end in sight' to have progress.
• Whether PL tweaks are the best approach was not under discussion

Tweaks can address some problems effectively. But not all problems, and not always most effectively. And deployment of incremental tweaks can (for long term costs) be a lot more expensive than making a painful change all at once.

You know if a prep chef cuts his finger chopping an onion we generally don't say that the knife has a design bug (unless it is a very unusual knife).

That's an interesting choice of analogies. To the best of my knowledge, the most common cause of people cutting their hands and fingers is use of a dull knife.

Perhaps the best way to view the bug is that Apple is selling computer systems that purport to do deserve a level of trust they can't ever possibly hope to earn

There are PL designs that can support a much greater level of trust. Sadly, these are not readily accessible by 'tweaking'.

Today, it is probably easier to account for a large team of lawyers and lobbyists and a big portfolio of patents as MAD deterrents, than it is to account for fixing and avoiding the technological errors.

I'll drop this particular thread between us, I think, because it's pretty played out but two last quick things:

technical merit

Paraphrasing myself again: I don't think contemporary PLT has a well grounded, coherent notion of "technical merit" but I also think it pretends that it does. That's the problem I've been trying to point at.

To the best of my knowledge, the most common cause of people cutting their hands and fingers is use of a dull knife.

In a kitchen where there is junior staff with the title "prep chef" the knives are usually very sharp. Knife accidents are most often the result of improper handling from poor training or inattentiveness.

Cutting yourself with a sharp knife while chopping an onion is, in general, caused by sticking a finger underneath the blade as it descends to the cutting board. More simply, you cut your finger by sticking your finger under the knife.

Proper handling is a variety of simple techniques that should be habitual and that have the effect of keeping fingers out from underneath blades.

I will guess that if you search the web for terms like "knife handling finger curl" you can probably find some basics explained.

I don't think contemporary PLT has a well grounded, coherent notion of "technical merit"

Technical merit is vague and heuristic, I'll grant, in part because it aggregates a lot of more precise concerns that may have different weights in different contexts. There is no strong, obvious total order between languages. But this doesn't keep it from being real or consequential. Technical merit is convincingly demonstrated by the existence of languages we classify as esoteric Turing tarpits.

My arguments do not assume that technical merit can be measured. My point is that such merit is not a primary factor in language adoption. Network effects and path dependence are much more significant.

No, I don't think that mere "tweaks" would change much. (Focus on largely irrelevant tweaks instead of fundamental improvements is what keeps disappointing me about most new languages coming out of industry.)

On the other hand, the logical conclusion of your argument is that software would be no worse if we were still hand-wrinting it in machine code. Do you seriously mean that?

Much of today's software development approach hasn't evolved very far from assembly language, unfortunately. I think there can be little doubt that this sad state of affairs has a significant economic (and social) cost, well beyond what's unavoidable by technical means.

On the other hand, the logical conclusion of your argument is that software would be no worse if we were still hand-wrinting it in machine code. Do you seriously mean that?

No that is not what I mean.

I mean that PLT as practiced seems to be lacking a good account of the value of proposed language features.

PLT is famously tongue tied when it comes to providing a working theory of the popularity of theoretically inferior languages.

I'm suggesting that the PLT field additionally over-values some directions of research that are somehow theoretically better in the internal PLT logic of what research gets done and published -- but somehow in practice not really better.

I will say this: in a kind of history of ideas departing from the starting point of machine language there are a lot of directions to go and PLT, so far, has gone deep but narrow in just a few directions.

I mean that PLT as practiced seems to be lacking a good account of the value of proposed language features.

Agreed. Measuring this is an unsolved problem (and will very likely remain one).

PLT is famously tongue tied when it comes to providing a working theory of the popularity of theoretically inferior languages.

The lack of correlation between popularity/success and technical merit is not at all exclusive to PLs or even IT. Just consider OSes, CPUs, file systems, video systems, you name it.

I'm suggesting that the PLT field additionally over-values some directions of research that are somehow theoretically better in the internal PLT logic of what research gets done and published -- but somehow in practice not really better.

It may perhaps be over-valuing some goods, but I don't see the evidence disproving practical value altogether. Admittedly, there also is not much besides individual and anecdotal evidence proving it either -- in part, because new ways are rarely even tried, and in part because it is practically impossible to make sound measurements, see above.

I will say this: in a kind of history of ideas departing from the starting point of machine language there are a lot of directions to go and PLT, so far, has gone deep but narrow in just a few directions.

What other "directions" do you think we miss out on?

I think PL research overvalues language semantics (in particular type systems) and undervalues standard library design and the design of how the programmer interacts with the language.

If you take the PLT viewpoint and ask "What is Smalltalk?", the answer would be a sheet of evaluation rules that specify the semantics. If you asked a Smalltalk programmer "What is Smalltalk?", he will definitely tell you about the semantics, but he will also tell you about the class browser, the ability to evaluate any expression right in the editor, he will tell you that Smalltalk has a collections hierarchy with lists, sets, maps, bags and more, that all follow a coherent API. Sure, these things are old hat nowadays, but there was a time when they weren't. If the designers of Smalltalk had solely focused on that sheet of evaluation rules, they would have missed a huge part of the value in Smalltalk. Furthermore, the design of the standard library and design of the interaction greatly informs how the semantics should be designed.

Amen.

I would argue that the sort of practical library design you are referring to is certainly an important area, but not really the topic of Programming Language Theory (although the field certainly does care about applicability to libraries). Would you blame compiler people for not researching GUIs?

Show me a language, even research one, that has no standard library, at least "core" one. It is also not unusual that a primitive construct of some language can be achieved in library code of some other, more powerful language. How can a PL be designed without primary concern of how the code (the "library") will be like in this language.

While compilers function without GUI quite well.

OK, let's hear a concrete example: what research problem would you like to see addressed regarding libraries? Honest question.

As I see it the problems are very similar to the ones for PL design. So there is abundance of problems to be addressed. For me there is no strong boundary between a programming language and a library, it's more like continuous "gray-scale"" transition. Libraries just continue from where the PL left of. They build additional abstractions using capabilities given by language (typically functions and types). Whereas languages can build arbitrary abstractions using keywords. One part of evolution of programing languages is to take often repeater patterns in libraries and provide better support for them in the languages. For practical reasons (when the goal is to develop an working app) I judge PL + Standard library + availability of app-domain library as one thing.

there is no strong boundary between a programming language and a library, it's more like continuous "gray-scale"" transition. Libraries just continue from where the PL left off

I agree with this sentiment.

In some ways, I feel that "library" is a poor metaphor, as it does not emphasize how the different modular elements extend the language. I'm growing fond of the 'dictionary' metaphor, with 'words' as the fine-grained modular component.

The division of fields is largely an accident of history, so I don't see any utility in strictly adhering to those divisions. It would in fact be quite silly to do so simply because of the name of a field. The line between what is a library and what is a language is quite blurry to begin with, and even in those cases where it is not blurry there are a lot of inter-dependencies. If you had to pick any field that is best equipped to design standard libraries, it would certainly be PLT, wouldn't it? Keeping them separate is like having one group of people design a lock and another group of people design a key. "We are lock theorists, we don't deal with keys"

The divisions of fields is largely an accident of history

I don't think I agree with that. Language theory is mostly concerned with identifying the fundamental building blocks of programming logic, and understanding their properties in depth. Library design on the other hand is far more engineering-oriented, and pretty much an open-ended topic. Sure, the distinction isn't always sharp, but what you expect from a library today is mostly tons of stuff that has little to do with theoretical questions.

It's more like one group doing car engines and the other coachwork and interior.

In many cases this mundane engineering has more depth to it than it seems at first sight. For example GUIs -> FRP, IO -> iteratees. There are a lot of interesting questions and problems in library design with a distinctly PLT flavor.

Language theory is mostly concerned with identifying the fundamental building blocks of programming logic, and understanding their properties in depth.

That explains why many POPL papers have very little to do with programming.

Library design on the other hand is far more engineering-oriented, and pretty much an open-ended topic.

It is actually a design discipline, but I digress. Basically, you are claiming that since language theorists have no tools to deal with library design, so its not a topic for them, even if it is a crucial topic for programmers who would never benefit from POPL papers anyways.

Perhaps you are right, and this is why I barely feel like a PL academic anymore.

I think PL research overvalues language semantics (in particular type systems) and undervalues standard library design and the design of how the programmer interacts with the language.

I disagree that PLT overvalues semantics, as virtually all of the most exciting languages focus almost exclusively on semantics, (a language such as Mercury would be a case in point) and there are a lot of useful innovations that come from modern type systems (I find Idris quite exciting).

That being said, I agree strongly about the value of library design. I didn't find Haskell (just as an example) very useful until I discovered FRP libraries, and arrows in particular. I've never been happy with the way event driven programming is handled in most languages, and the idea of dataflow programming fits well with my intuition of how interactive applications should work.

Instead of just offering a cutting edge feature that I would have no idea of how to utilize in production software, it was a selling point for using Haskell in more projects.

I disagree that PLT overvalues semantics, as virtually all of the most exciting languages focus almost exclusively on semantics

This is a self fulfilling prophecy ;-)

What other "directions" do you think we miss out on?

I'm sure there are vastly many more than I can think of but here are five off the top of my head:

1. Programmers sometimes informally describe programming languages with sensual terms. A language may be fun to program in, or luxorious. Or it may be annoying or tedious. Languages maybe easier or harder. They may be intuitive or mind-bending.

I don't think contemporary PLT places much ostensible value or even much recognizes advances that make a language easier and more fun. (Of course there are exceptions that prove the rule. For exmaple, some contemporary PLT practice concerns trying to make fun, easy languages and environments for young kids.)

2. Historically early programmers often relied of necessity on very literal operational models of their programs. They were aware as they coded or read code of the literal layout of data in registers and memory, for example. Even as "higher level" programming languages came into use, a deeply literal operational model was still at the core of how programmers often thought about their programs.

I want to mention an example for which I am hard pressed to remember the actual citation -- and I think there may be more than one. I remember reading very old JACM issues with algorithms using an algol-style notation and, nevertheless, illustrating a recursive algorithm by some non-recursive pseudo-code containing a loop and an explicitly managed data stack!

Contemporary PLT seems to be downright hostile to such literal operational thinking and I wonder what has been lost. It might be interesting to see PLT work that aims to assist that operational style programming. (Again there are exceptions in contemporary PLT and again the ones that come to mind are simple toy languages for kids.)

3. Often programming languages carry implicit assumptions or theories of the appropriate division of labor in software development. For example, the topic of modularity in contemporary PLT is often ostensibly presented as a decoupling of proofs about various components of a program but implicit in this seems to be an intention to decouple the labor of developing those components.

Traditionally and in contemporary PLT it seems to me the division of labor contemplated is predominately hierarchical and siloed. The library module developers are each on their own and under a master planner for the libraries, to exaggerate just a bit. There is a lot of emphasis on not only reasoning about parts of programs as "black boxes" but as treating them as such in the development process.

In the world of libre software (aka open source) it is not uncommon to encounter a programming problem that cuts across these boundaries of modularity. Yet, there are high practical costs because the organizing structure of public projects reflects the traditional hierarchical, siloed idea of development. Thus, if adding my new feature involves changes to 5 library modules and a main program, I may have to awkwardly intereact with 6 entirely separate projects.

In a "flattened" world where software development is naturally less hierarchical, less compartmentalized -- are there alternatives to traditional concepts of modularity?

4. Early in the history of programming it was recognized that (so to speak): the computer can help check the correctness of the program. Compiler warnings, static checking, computer-assisted correctness proofs... Contemporary PLT seems obsessed with this concept, to me.

An alternative approach to building robust systems is:

(a) emphasize stupid-simple components whose correctness can quickly be checked by inspection or with pencil and paper

(b) emphasize methods of composition that are similarly stupid-simple

What programming language features are most supportive to this kind of "rough and ready" style of highly pragmatic engineering? Can programming systems be developed in this style that are good for seemingly complex applications like web browsers?

5. Historically, some languages were first recognized as valuable because of their extreme simplicity of implementation. Forth, for example, admitted a tiny implementation yet had many "high level" qualities. I think the early history of Scheme can be recognized as a very practical PLT breakthrough towards simpler compilers and interpreters.

Contemporary PLT seems relatively uninterested in new languages that emphasize ease of implementation (Lua, perhaps, excepted).

(a) emphasize stupid-simple components whose correctness can quickly be checked by inspection or with pencil and paper

I don't think this is new territory. Stupid-simple components end up requiring not-so-stupid-simple glue. What set of stupid-simple components is going to implement distributed consensus like Paxos/Raft? There's a proper balance between consolidating and scattering logic. Pencil and paper don't always suffice.

(b) emphasize methods of composition that are similarly stupid-simple

I agree. Fortunately, it's also a subject being studied in PLT.

Stupid-simple components end up requiring not-so-stupid-simple glue.

That's an oddly strong claim.

What set of stupid-simple components is going to implement distributed consensus like Paxos/Raft? There's a proper balance between consolidating and scattering logic.

I'm not conversant in those protocols so maybe you can unpack that a bit?

I did note in a quick check that Paxos was devised in the late 1980s by Leslie Lamport et al. which leads me to guess that it's (a) damn clever; (b) approachable and comprehensible using paper-and-pencil reasoning; (c) based on stupid-simple mechanisms, cleverly arranged in composition. (My guesses are based on the line of reasoning that goes: "because Lamport".)

So I'm not sure what you're getting at.

I did note in a quick check that Paxos was devised in the late 1980s by Leslie Lamport et al. which leads me to guess that it's (a) damn clever; (b) approachable and comprehensible using paper-and-pencil reasoning; (c) based on stupid-simple mechanisms, cleverly arranged in composition. (My guesses are based on the line of reasoning that goes: "because Lamport".)

There were at least 3 papers published in the past few years just trying to explain Paxos. Raft is a new distributed consensus algorithm designed because Paxos was so complicated. It's not nearly so simple as pencil and paper, but perhaps your pencil and paper threshold is much higher than mine.

I don't know what to conclude from this. It seems like most of your points can be paraphrased as: "in old times, programming used to be simple and direct, and now that's not the case anymore". And between the lines you seem to imply that this development is (1) a mistake, and (2) the fault of PL research.

But the software that's being built today is 10000 times more complex. To keep it manageable at all, more and higher-level abstractions are crucial. That naturally includes languages themselves. Whatever directions language research has taken is not the cause of a changing world, it is the reaction to it. And as such, it has rather moved too little than too much.

But the software that's being built today is 10000 times more complex.

That seems false to me but let me begin by asking what you mean by "10000 times more complex" and if you have in mind particular sectors or if you think this is true of software in general.

It's true of software in general. Do you think that observation is controversial?

But the software that's being built today is 10000 times more complex.

That seems false to me but let me begin by asking what you mean by "10000 times more complex"

It's true of software in general. Do you think that observation is controversial?

"Controversial" isn't the word I'd use. It seems like a greatly exaggerated, hand-wavy thing to say.

Trying to make sense of it one way I come to the idea that software today (contrasted with "yore") is often built from much larger numbers of components that are more or less developed independently. Yet, I don't think that there is any kind of exponential effect in complexity here. In fact, in successful programs with many components, if "complexity" could be quantified, it seems to me to grow at sub-linear rates (in the number of components). And I wouldn't attribute successful complexity avoidance like that much to building ever higher levels of abstraction, as you originally suggested. Rather, I think it comes from keep to relatively simple architectures.

As code size increases, how, in your view, does "keep[ing] to relatively simple architectures" differ from "building ever higher levels of abstraction"?

"keep[ing] to relatively simple architectures" differ from "building ever higher levels of abstraction"?

Repetition rather than subsumption.

For example, if you start with a game engine and three kinds of monster, but through a standardized interface then add 1,000 kind of monster, your program is much larger but a simple architecture has kept it from becoming more complex. That's repetition.

A case of subsumption might be the introduction of JQuery over various implementations of the DOM model. (And there, lo, we do have a bona fide PLT contribution with real world impact!)

Food for thought, certainly. Seems to relate to the point about what "10000 times more complex" means (which I wondered too, but didn't want to skew the discussion by commenting on). If you go from 3 kinds of monsters to 1000, intuitively it seems things have gotten somewhat more complex. Do different kinds of monsters have things in common that ought to be factored out of their definitions? If so, that would seem to be a sort of abstraction... though what it does to copmlexity, intuitively, is an interesting question since, depending on how the factoring is done, it might complicate the task of creating new kinds of monsters. (One might suppose that if anything needs factoring out, that's a flaw in the engine design.)

Developing an architecture is ultimately an exercise in abstraction. The issue of reusability remains: is the architecture reusable or project specific? If not reusable, how can it be right? It has certainly not been exercised sufficiently to work out the design flaws.

If you have three kinds of monsters, you do not know enough to create a standardized interface for the next nine hundred. There will be something you missed, e.g. with respect to AI, tactics, pathfinding, leadership or swarm logic, animation, etc..

I think that architecture doesn't save you any complexity... but it might be a better way to organize the essential complexity, to seek reusable and composable architectures.

Developing an architecture is ultimately an exercise in abstraction. The issue of reusability remains: is the architecture reusable or project specific? If not reusable, how can it be right? It has certainly not been exercised sufficiently to work out the design flaws.

This is satire? (E.g., the unqualified equivocation of "right" with a PLT-self-serving concept of "reusable".)

I did not equate or equivocate reuse with rightness. But I believe that reuse of an architecture is necessary to achieve rightness (by any reasonable definition of 'rightness' - simplicity, avoiding discontinuities and rewrites, etc.). There relationship is indirect, but I did clarify it in that last sentence you quoted.

When you first develop an architecture, it will have flaws like any other software you initially develop. Those flaws include awkward corner cases, poor divisions of labor... epicycles, in a sense, that create a lot of unforeseen complexity for the users. To work out these flaws requires testing the architecture in a lot of different scenarios and use-cases. A single project will exercise an architecture only a few ways. An architecture developed for only one project is generally incapable even of anticipating the needs for future versions of that project.

So reuse of an architecture across projects is essential to create 'right' or 'simple' architectures. It's a prerequisite of process, given the inherent limitations of foresight. But reusable and right are not at all the same concept.

Why did you interpret it as satire? I'm a little upset that you did so.

Why not just say that reuse can help to produce reusability? How did "rightness" get into it?

Reuse is necessary for simplicity, for productivity, not just for reusability (though that too). By 'rightness' I mean whichever of these (or many other) nice properties you're seeking.

Even your 1000 monsters example won't happen right away. You don't find a good architecture for creating 1000 different kinds of monsters until you've developed a lot of video games and worked out the kinks in the architecture across many projects - e.g. reducing it to something like component-entity systems.

A new, project-specific architecture will never have that refinement that results from reuse across projects.

The problem of creating simple architectures is very little different, in practice, than that of creating reusable abstractions.

Reuse is necessary for simplicity, for productivity, not just for reusability (though that too). By 'rightness' I mean whichever of these (or many other) nice properties you're seeking.

I don't see any justification for the connective "is necessary" in your argument, which is kind of my point to you in these last few messages.

I guess I'd refer you to what Crista Lopes talks about as "honesty" in this other blog post of hers

I already provided justification and argument. I'm inclined to agree that you don't see it - my impression is that you haven't been reading my words, beyond picking a few things out of context.

Rightness is when it does what I want without undue trouble. It seems justified to say that it isn't usually right until it's been smacked around enough to settle into a good form, because that smacking around forces me to think about it from different places.

I agree with a lot of your complaints here, at least partially.

I don't think contemporary PLT places much ostensible value or even much recognizes advances that make a language easier and more fun.

I think there has been a lot of PL work towards this end, if you look for it. They don't use "easier and more fun" in paper names, of course. Rather, of smooth transitions, avoiding discontinuities, supporting context-sensitive analysis and auto-completion, generative programming, live programming, conversational programming, self-explaining computation, automated debugging.

I think there's a lot of "you can't get there from here" issues in modern PL design. A lot of interesting ideas are difficult to adapt to mainstream languages.

aware as they coded or read code of the literal layout of data in registers and memory

Dave Griffith had something interesting to say on this subject recently.

Operational reasoning has a nice properties of being relatively concrete. But the PDP-11 mental model of the machine isn't very nice - e.f. it isn't secure, interacts poorly with parallelism, doesn't map well to FPGA or GPU.

I believe we should support operational reasoning, but do so in a way that simultaneously supports denotational reasoning. I believe this requires creating an alternative machine model - one with nicer properties - and a machine code (or bytecode) to control it. My Awelon bytecode (ABC) is one such attempt.

in contemporary PLT it seems to me the division of labor contemplated is predominately hierarchical and siloed

I agree. I've argued many times for flat namespaces and more social approaches to software development - e.g. based on wikis, or some variation on app-stores except favoring reusable software components. A single wiki might host thousands of projects, thus enabling a high level of cross-project refactoring and maintenance (and reducing the burden for backwards compatibility, since you can directly refactor all clients).

My AO language has been carefully designed for use in such a wiki-like environment. And ABC is suitable for sharing software components directly - i.e. as a blob of bytecode named by secure hash. So I'll be able to explore some of these alternatives within the next couple years.

are there alternatives to traditional concepts of modularity?

Yeah, quite a few. Reusable component models. Overlay network models. Constraint-based linking models. Formalization of information hiding / ADTs via sealers or type generation.

the computer can help check the correctness of the program. Compiler warnings, static checking, computer-assisted correctness proofs... Contemporary PLT seems obsessed with this concept, to me.

It's a pretty good idea to automate such things where they can be automated, and to catch errors as early as possible. Even those favoring agile models focus heavily on automatic testing, and tools like Erlang's dialysis tool are welcome. Automated systematic debugging is an interesting subject that I haven't studied, but also seems widely useful.

The main contention here seems to be: how much should ensuring correctness interfere with expression? Can we do a better job of separating concerns here?

You mention focusing on software components that are simple to check individually. This works to some extent, but isn't so different from checking functions. As complexity builds through branching, state, and sheer size, it becomes more difficult to check.

I do favor focusing on component software, but in general it becomes more difficult to check correctness of larger components. We can focus on more compositional reasoning (which is helpful for preventing many kinds of errors). But correctness is fundamentally contextual, not compositional. As programs scale - more logic, more state - they simply become more difficult to validate.

Your idea of focusing on 'stupidly simple' components would only scale to stupidly simple software.

Contemporary PLT seems relatively uninterested in new languages that emphasize ease of implementation

I believe it's hugely important. Relating to my earlier mention of activation energy, I think that keeping syntax and implementation simple is critical to reduce the cost of tooling and improve development tools.

GCC has an option that makes it complain about unreachable code -Wunreachable-code - with this option it should have said something about such a goto;

For whatever reasons -Wunreachagle-code is not turned on by -Wall , go figure. Maybe that's the real conspiracy ;-)

However i think the real 'fix' should have been unit tests, there is nothing that could replace them.

I will look at this.