## some critiques of the Semat initiative

In catching up with my Bloglines subscriptions, I came across the the following piece by IvÐ°r JÐ°cobson, BertrÐ°nd MÐµyer, and RichÐ°rd SolÐµy:

Given the ambition of the Semat project it would be surprising if it did not cause some controversy. Alistair Cockburn wrote a long critique. Martin Fowler echoes Cockburn and states that the effort is â€œfundamentally doomedâ€.

Scott Ambler thinks SEMAT is good. What does DÐ°niel BÐµrry think?

## Comment viewing options

### From Cockburn's critique: I

From Cockburn's critique:

I look for a theory to provide explanation and prediction of what happens on a project. The organizers of SEMAT are looking for a theory to be mathematical and composable. I do not think it is possible to have both, and I think that having accurate predictions from a theory is the more important of the two.

This struck me as a little odd. Doesn't a mathematical and composable "theory" of software development provide ultimate explanatory and predictive power? Is a process that does not have formal description truly understood? How would one make predictions if not by some formal technique?

### He's saying that a

He's saying that a mathematical theory of people's behavior is not going to work.

### But he's also saying he

But he's also saying he wants predictive and explanatory power regardless. How are those two propositions not contradictory?

### non-formalized predictive theories

Re: comment-60097:

How would one make predictions if not by some formal technique?

As a f'rinstance, in Partitionâ€“Editâ€“Count: Naive Extensional Reasoning in Judgment of Conditional Probability, Craig R. Fox and Jonathan Levav build a rather handwavy and manifestly non-mathematical theory of how most people solve conditional-probability problems. The theory is clearly falsifiable in Popper's sense and allows some qualitativie predictions to be made and verified.

BTW, I didn't get a chance to read Cockburn's (/koÊŠbÉ™rnz/) piece until after I had read SEMAT's rebuttal. (Cockburn's server was giving me a 503 yesterday.) I have to say, his argument is a lot less dumb and self-contradictory that I had been led to believe by JÐ°cobson's, MÐµyer's, and SolÐµy's response. Emotionally charged? Yes. Self-contradictory? No.

### Re: non-formalized predictive theories

Re: comment-60117â€¦

dmbarbour did a better job than I arguing against mathematical chauvinism. Grep his post for the words â€œHow absurd! How patently untrue!â€

### I'd say it's the reverse.

I'd say it's the reverse. There is shockingly little rigorous (analytic or quantitative) and relevant work in the research community relative to, for example, model checking, logic, type theory, functional programming, and many other topics discussed here.

While there seems to be an emphasis on process in industry (scrum, etc.,), I'm not sure how meaningful it is outside of your manager / team communicating and otherwise paying attention. The current emphasis in industry often seems like one of (well-meaning) lemmings under some strange Darwinian process, providing no understanding of how the social system forms the end product. We still have basic surface questions like how interchangeable are these fads and many deeper ones such as from Mark Wegman (how do we debug the processes and what are tools to do so?). Perhaps a useful way to interpret the seeming emphasis in industry is that a symptom of some deep problem(s) are emerging.

### Software engineering != civil engineering

Just to be clear, analogies between software engineering and civil engineering break down very rapidly.

For example, in civil engineering one might say "they don't build 'em like they used to", indicating that buildings have actually decreased in construction quality. In general, in the Boston area, you will find:

• Improved building codes for electrical; for examples, (1) replacing a two-pronged outlet with a grounded outlet or GFCI outlet (2) mandated placing electrical outlets every 6 feet in new buildings, because it is safer to plan for people to need extra plugs than for them to overload some surge protector.
• Minimum height off the floor for second story and above windows, in all new construction; omittance requires filing for and being approved for a variance, because the code exists to protect children from falling out of windows - Bottom line: some aesthetic beauty is lost by significantly limiting the height of oversized windows; for example, you cannot see out the window just by sitting at the dinner table
• Houses have better foundations 100 years ago than they did today; computers are the only invention that has continuously gone down in cost every year since their inception - you cannot say the same thing, for example, with concrete, since right now China has cities with 100 million people in it (3x the population size of all of California) and they are focused on building up those places into mega-cities

Generally speaking, in software, programs actually get better when good processes are enforced. If software engineers say "they don't build 'em like they used to", it generally means the old way was crap! The only exception might be the quasi-elegance of a one-pass compiler that fits in a few KB of memory. But that will gradually be knocked down, too, I think.

### Re: Software engineering â‰  civil engineering

Re: comment-60139â€¦

Perhaps software engineering is nothing like civil engineering but there is no denying that it is very much like plumbing.

The abstract states it is a myth that geography affects the engineering process, measured in terms of post-release failures.

I think I am going to request the raw data set from this project and see what I can determine on my own.

### devil in the details

The same group has shown code module boundaries where developers across them communicate less are more prone to bugs. I believe the point of this particular study work was that by focusing on breaking down geographic communication boundaries -- which most companies fail at -- Microsoft fixed (part of) the typically found problem. So: untreated, geographic distance leads to communication distance which leads to bugs on module boundaries. The win here is that, while geographic distance may be desirable, the communication distance may be (automatically) detected and (manually) treated, diminishing a category of software fault.

Another interesting one was from Philip Guo last summer where he tracked how geographic and work hierarchy effects, such as being in different buildings or teams or managerial level, impact the likeliness of a bug making it through triage. I don't recall any proposed constructive interventions from this work, however.

Btw, I doubt you can get their stats. OTOH, you might have better luck with open source projects like Mozilla and Webkit or repos like Sourceforge and Ohloh (I've had luck with the former but not the latter).

Edit: Philip's paper is Characterizing and Predicting Which Bugs Get Fixed

### Process as a domain of engineering discourse

Since the late 1800s: http://en.wikipedia.org/wiki/Industrial_engineering. In some ways industrial engineering is the closest of the traditional engineering disciplines to software development. You are correct to note that human factors complicate process engineering, but that doesn't make the subject completely untenable.

### design versus production

In software we are focused on what is basically a design process because the production process is comparatively trivial (the source code is the design). In Industrial engineering it is mostly about the production process although there is a feedback loop into the design process to optimize production. So while they both are a lot more process oriented than other engineering disciplines there is still a world of difference between the two.

### the engineering of process and the process of engineering

Re: comment-60122:

â€¦ human factors complicate process engineering, but that doesn't make the subject completely untenable.

Has much progress been made since the days of Speedy Taylor?