Lambda the Ultimate

This was my response from when the article showed up on reddit.com:

I think the author's of this paper and several of the posters here completely miss the boat.

What this paper does do is pinpoint a specific area of misconception that many students coming into programming have. But the authors then go on to propose that it's a litmus test for telling those who 'can' from those who 'can't'. They mention in passing that they 'might' get better results by, surprise, trying to correct this in student's minds.

THAT'S THE WHOLE POINT. That shouldn't be some minor sentence burried deep in the paper. The paper spends paragraphs talking about how all attempts at improved programming education have failed. It also points out this precise thing that if students have wrong hurts them. But the authors completely fail to realize then that the reason all previous attempts have failed is because they haven't targetted this precise thing, the mental model for assignment/sequence, not because of some inherent property of students.

Commenters above posit that that engineering and comp sci schools are just for weeding people out. They are mostly correct. That's not because the students suck. That's because the type of people who succeed in these areas tend to have the communicative skills of a gerbil. Even when they have great command of the English language (which is a small subset in itself), engineering types are often at a total loss when it comes to seeing a problem from a non-hyper-logical perspective and pinpointing what's wrong with a student's reasoning, i.e., empathizing, demonstrating EQ not just IQ.

I have seen countless numbers of teachers tell students to just "think a little more" about a problem when it is quite clear they've given up on the student being able to solve a problem. They convince themselves that the student is incapable of understanding because of their own inability to explain. The authors state that teaching the consistent group is "easier." Well, yeah, if you take the group that already has the correct mental model coming into the course, yes, things are "easier." Easier in the sense that you are no longer functioning as a teacher but instead as a text-to-speech reader for a Java tutorial.

Overall, I'm glad the paper pinpointed a way of testing for a common source of errors coming into the course. But on the other hand I think their conclusions are flat out wrong.

I am entirely self taught. I code C++ now as a part time job, in and of itself an incredibly obfuscated language, but I also make use of things most C++ programmers find too esoteric, the boost libraries, heavy template usage, etc. I harbor considerably more skill than the hordes of UML diagram drawing software engineers out there. I am also fairly certain that I would have failed the test when I started, probably being in the 'inconsistent' group. I got over the hurdles by thinking about them over and over over long periods of time. A teacher who would have told me that "binary is just another way of writing the same numbers" or that "computers follow everything literally and always the same way" and demonstrated such probably would have cut 2 years off my learning time.

61 Subjects seems like a depressingly small sample size. It seems arrogant to assume a class of people is fundamentally incapable of learning a given discipline.

The paper also suggested that the language used to teach has no bearing on the student's learning, and yet the test they propose intentionally relies on Java's conflation of assignment with the mathematical symbol for equality. How might the same test have gone with Pascal's ':=' assignment operator or the completely foreign Scheme notation?

The research is interesting and corroberation would be nice but most CS programs have 2-3 cullings and intro is only the first one. While the notion of being able to take an aptitude test rather then sit through a survey for exploration seems attractive if there is no correlation at the later hurdles the value would be no more than saving some college students some money.

They pose questions along the lines:

int a = 10;
int b = 20;

a = b

What are the changed values of a and b?

Look at the company they keep. They are lumped in with those who correctly guess what JAVA does:
a = 20, b = 20
The tests at the end of the course don't seem to be an awful lot harder. It would be appalling if students who guessed the right answer consistently, whose intuitions were aligned with the language they were about to be taught, were messed up by the course and ended up failing.

The paper emphasises the current absense of effect aptitude tests. This is presumably a comment on the absense of double sided tests able to predict both success and failure. It seems obvious that testing students to see if they belong to the consistently correct subgroup of the consistent group is going to be a one-sded test for success. Pass and you will succeed. Fail and the test does not predict. It also seems obvious that such an aptitude test makes a mockery of a university level course - we can claim little credit as educators if we only admit those who can guess much of what is to be taught before they have attended any lectures.

The interest of the observations lies in the success of the consistent but incorrect group constrasted with the failure of the inconsistent group. The suggestion seems to be that those with an incorrect mental model fix it within the several-week timescale of the course, but those without a mental model do not build one within that timescale.

The authors have made a potentially very interesting observation, but until they split up the consistent group into correct and incorrect none of us know the quantitative detail.

I suspect it would be fairly easy to reproduce these results---give students the final exam coming in the door at the beginning of the semester and again going back out at the end, and don't teach them anything in the middle.

Aside from a sad attempt at humor, why do I think that is relevant? Statements like the following:

There was an attempt to administer the test to another 30 or so students on a foundation (predegree) programming course at Middlesex University. That administration failed, because the students – rightly, in our opinion – were incensed at the conduct of their teaching and the arrangements for their study, and simply refused to do anything that wasn’t directly beneficial to themselves.

and

Another group had very much lower than average A-level scores, and had been admitted to a maths and computing course largely to boost numbers in the mathematics department: we call this group the low achievers.

In short, I suspect there's something wrong in Middlesex.

Further, if the author's goals were to avoid multiple humps in the their grade graphs, I would have to suggest that they have not succeeded; see Figure 16, Figure 13, and Figure 10. They may be on to something with the consistency criteria, but I would be very hesitant to associate that with "natural aptitude".