some critiques of the Semat initiative

In catching up with my Bloglines subscriptions, I came across the the following piece by Ivаr Jаcobson, Bertrаnd Mеyer, and Richаrd Solеy:

Given the ambition of the Semat project it would be surprising if it did not cause some controversy. Alistair Cockburn wrote a long critique. Martin Fowler echoes Cockburn and states that the effort is “fundamentally doomed”.

Scott Ambler thinks SEMAT is good. What does Dаniel Bеrry think?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

From Cockburn's critique: I

From Cockburn's critique:

I look for a theory to provide explanation and prediction of what happens on a project. The organizers of SEMAT are looking for a theory to be mathematical and composable. I do not think it is possible to have both, and I think that having accurate predictions from a theory is the more important of the two.

This struck me as a little odd. Doesn't a mathematical and composable "theory" of software development provide ultimate explanatory and predictive power? Is a process that does not have formal description truly understood? How would one make predictions if not by some formal technique?

He's saying that a

He's saying that a mathematical theory of people's behavior is not going to work.

But he's also saying he

But he's also saying he wants predictive and explanatory power regardless. How are those two propositions not contradictory?

non-formalized predictive theories

Re: comment-60097:

How would one make predictions if not by some formal technique?

As a f'rinstance, in Partition–Edit–Count: Naive Extensional Reasoning in Judgment of Conditional Probability, Craig R. Fox and Jonathan Levav build a rather handwavy and manifestly non-mathematical theory of how most people solve conditional-probability problems. The theory is clearly falsifiable in Popper's sense and allows some qualitativie predictions to be made and verified.

BTW, I didn't get a chance to read Cockburn's (/koʊbərnz/) piece until after I had read SEMAT's rebuttal. (Cockburn's server was giving me a 503 yesterday.) I have to say, his argument is a lot less dumb and self-contradictory that I had been led to believe by Jаcobson's, Mеyer's, and Solеy's response. Emotionally charged? Yes. Self-contradictory? No.

Re: non-formalized predictive theories

Re: comment-60117…

dmbarbour did a better job than I arguing against mathematical chauvinism. Grep his post for the words “How absurd! How patently untrue!”

Speaking of which…  Does anyone have a copy of Brittle Software: A Programming Paradox by Paul G. Bassett (Information Systems Management, vol. 4, no. 3, 1987, pp.8-14). In it, he seems to argue that “a mathematical chauvinism in computer science … has contributed directly to the software brittleness paradox.”  Sounds like an interesting argument but not one that's worth $37 to me.

I have Framing Software Reuse, an elongated book format diatribe

Paul's kind of got a weird notion of how to design systems (IMHO), and the objections he answers in his writing are not necessarily the ones I raise. But, at the time, the objections he was answering to were suprisingly common in software development. It is a time capsule of software engineering history, that people used to argue about certain weird stuff like this.

If you want to borrow the book, shoot me an e-mail.

Since when is engineering

Since when is engineering (as a science) involved with process methodology? Isn't the science entirely related to the product, its physical parameters and its observable properties, rather than the ways arbitrary companies ( hallo BP! ) are used to build them?

When to draw an apt analogy it would be that SE attempts to equal "scientific management", something I cannot say anything about except that it hasn't decreased hype, fashion and in particular politics over the time.

engineering is a process - or - so what?

The process of (software) engineering, quantitatively demonstrated by many studies such as those by Christian Bird and Brendan Murphy, is a predictable social (or sociotechnical) process. Furthermore, as shown by their work, their predictions can also suggest where faults lie, and thus how to correct the engineering process.

Whether this is interesting to you is a different matter. I'm interested in building better programs, so, to me, understanding and improving the process of building is a fair (and fascinating) target. It seems like you're making some sort of distinction by using the word 'science' but I don't follow it (and I am using a lax intuition of software engineering).

product and process

I'm interested in building better programs, so, to me, understanding and improving the process of building is a fair (and fascinating) target.

You could say much the same thing about almost any branch of engineering. I think what Kay was getting at is that most other branches of engineering focus most of their efforts towards "building better $PRODUCT" on understanding their building materials, understanding how to combine those materials into robust and efficient designs, and understanding how to analyze those designs to ensure they meet their requirements. Sure, they may give some consideration to "the process of building". But not to anywhere near the same degree that the software community -- or at least certain elements of it -- does. Do we already have all of the answers when it comes to "materials", design techniques, and analysis, meaning that all we have left is fiddling with process? Or are we spending so much effort on process and methodology because we actually have no idea how to tackle the rest of the problem?

I'd say it's the reverse.

I'd say it's the reverse. There is shockingly little rigorous (analytic or quantitative) and relevant work in the research community relative to, for example, model checking, logic, type theory, functional programming, and many other topics discussed here.

While there seems to be an emphasis on process in industry (scrum, etc.,), I'm not sure how meaningful it is outside of your manager / team communicating and otherwise paying attention. The current emphasis in industry often seems like one of (well-meaning) lemmings under some strange Darwinian process, providing no understanding of how the social system forms the end product. We still have basic surface questions like how interchangeable are these fads and many deeper ones such as from Mark Wegman (how do we debug the processes and what are tools to do so?). Perhaps a useful way to interpret the seeming emphasis in industry is that a symptom of some deep problem(s) are emerging.

Software engineering != civil engineering

Just to be clear, analogies between software engineering and civil engineering break down very rapidly.

For example, in civil engineering one might say "they don't build 'em like they used to", indicating that buildings have actually decreased in construction quality. In general, in the Boston area, you will find:

  • Improved building codes for electrical; for examples, (1) replacing a two-pronged outlet with a grounded outlet or GFCI outlet (2) mandated placing electrical outlets every 6 feet in new buildings, because it is safer to plan for people to need extra plugs than for them to overload some surge protector.
  • Minimum height off the floor for second story and above windows, in all new construction; omittance requires filing for and being approved for a variance, because the code exists to protect children from falling out of windows - Bottom line: some aesthetic beauty is lost by significantly limiting the height of oversized windows; for example, you cannot see out the window just by sitting at the dinner table
  • Houses have better foundations 100 years ago than they did today; computers are the only invention that has continuously gone down in cost every year since their inception - you cannot say the same thing, for example, with concrete, since right now China has cities with 100 million people in it (3x the population size of all of California) and they are focused on building up those places into mega-cities

Generally speaking, in software, programs actually get better when good processes are enforced. If software engineers say "they don't build 'em like they used to", it generally means the old way was crap! The only exception might be the quasi-elegance of a one-pass compiler that fits in a few KB of memory. But that will gradually be knocked down, too, I think.

Re: Software engineering ≠ civil engineering

Re: comment-60139…

Perhaps software engineering is nothing like civil engineering but there is no denying that it is very much like plumbing.


Does Distributed Development Affect Software Quality?:An Empirical Case Study of Windows Vista by Christian Bird

The abstract states it is a myth that geography affects the engineering process, measured in terms of post-release failures.

I think I am going to request the raw data set from this project and see what I can determine on my own.

devil in the details

The same group has shown code module boundaries where developers across them communicate less are more prone to bugs. I believe the point of this particular study work was that by focusing on breaking down geographic communication boundaries -- which most companies fail at -- Microsoft fixed (part of) the typically found problem. So: untreated, geographic distance leads to communication distance which leads to bugs on module boundaries. The win here is that, while geographic distance may be desirable, the communication distance may be (automatically) detected and (manually) treated, diminishing a category of software fault.

Another interesting one was from Philip Guo last summer where he tracked how geographic and work hierarchy effects, such as being in different buildings or teams or managerial level, impact the likeliness of a bug making it through triage. I don't recall any proposed constructive interventions from this work, however.

Btw, I doubt you can get their stats. OTOH, you might have better luck with open source projects like Mozilla and Webkit or repos like Sourceforge and Ohloh (I've had luck with the former but not the latter).

Edit: Philip's paper is Characterizing and Predicting Which Bugs Get Fixed

Process as a domain of engineering discourse

Since the late 1800s: In some ways industrial engineering is the closest of the traditional engineering disciplines to software development. You are correct to note that human factors complicate process engineering, but that doesn't make the subject completely untenable.

design versus production

In software we are focused on what is basically a design process because the production process is comparatively trivial (the source code is the design). In Industrial engineering it is mostly about the production process although there is a feedback loop into the design process to optimize production. So while they both are a lot more process oriented than other engineering disciplines there is still a world of difference between the two.

the engineering of process and the process of engineering

Re: comment-60122:

… human factors complicate process engineering, but that doesn't make the subject completely untenable.

Has much progress been made since the days of Speedy Taylor?