Static vs. Dynamic Languages: A Literature Review

We've mentioned some empirical studies of programming languages a few times, but I haven't seen a comprehensive list we can use as a reference.

Fortunately, I just came across this pretty decent overview of existing literature on how types impact development. Agree or disagree with Dan Luu's position, the comprehensive list warrants a front-page post in my opinion.

One point worth noting is that all the studies used relatively inexpressive languages with bland type systems, like C and Java, and compared those against typed equivalents. A future study ought to compare a more expressive language, like OCaml, Haskell or F#, which should I think would yield more pertinent data to this age-old debate.

Part of the benefits of types allegedly surround documentation to help refactoring without violating invariants. So another future study I'd like to see is one where participants develop a program meeting certain requirements in their language of choice. They will have as much time as needed to satisfy a correctness test suite. They should then be asked many months later to add a new feature to the program they developed. I expect that the maintenance effort required of a language is more important than the effort required of initial development, because programs change more often than they are written from scratch.

This could be a good thread on how to test the various beliefs surrounding statically typed and dynamically languages. If you have any studies that aren't mentioned above, or some ideas on what would make a good study, let's hear it!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Cut and Paste Coding

Cut and paste coding using resources like stack-overflow is a big problem. People use code fragments without understanding what they do, and then have no idea how to fix it when it inevitably does not work as expected. On the other hand there are a lot of good learning resources on the internet that will help an intelligent person learn how to program well.

People use code fragments

People use code fragments without understanding what they do

Using libraries has the same problem. I still haven't figured out what all those OpenGL calls are doing under the hood...

Really? Depending in the

Really? Depending in the context, I would credit the student who just googled the answer with more intelligence, and the other student for actually being more stupid (the answer was out there, why did you waste your time?). As professor Scrooge McDuck once said: "work smarter, not harder."

Evidence of Intelligence

Note that I said Googling provides less evidence of intelligence. There is little evidence of intelligence when you Google the answer, anyone could do that, but in working out the solution for yourself you demonstrate you have the mental tools to solve the problem.

Knowing how to solve

Knowing how to solve problems in the very energy efficient ways is the hallmark of human evolution. Googling (or say, using prior research) is definitely a part of that!

There is little evidence of intelligence when you Google the answer, anyone could do that

You would be surprised....

but in working out the solution for yourself you demonstrate you have the mental tools to solve the problem.

I think what you really want to say is: if the answer isn't available on Google, we still need to solve the problem. Getting experience solving problems with existing answers (without looking at those answers) is a way to develop those skills. But you also have to spend time working on your Google-Fu to prepare yourself for the real world.

The most intelligent people I know can find the answer to almost any problem if they don't know the answer already. The ability to tap into the collective intelligence of humanity shouldn't be underrated, and I would argue is a form of intelligence in itself.

Argument doesn't support conclusion

We don't allow students to bring Wolfram alpha into a math exam for the same reason we don't allow them to bring Terence Tao. The reason isn't that it wouldn't increase their combined intelligence.

Edit: Looks like we've made it to page 2. Remember, civility is optional on page 2 and profanity is encouraged.

System 2

Is system 2 evidence of Intelligence, because system 1 looks a lot like applying knowledge and experience, which can be learned by rote, and does not necessarily involve any complex thought processes?

experience

Experience seems necessary to take full advantage of intelligence. Conceivably both the very best and the very worst people do may be in system 1, which would seriously mess up correlations along the system 1/2 distinction.

Therefore the system of

Therefore the system of student + Google demonstrates less evidence of intelligence than just the student (because intelligence is not just searching for a pre-defined answer using text matching?)

One more thought: a colleague of mine who does machine learning believes that humans are just really fancy pattern matchers...that intelligence is just a trick of pattern matching. In that case, even if you are working through a problem using "first" principles, you are pattern matching your way through it.

BTW, those first principles usually aren't! Newtonian mechanics didn't exist before Newton. We tend to build on existing knowledge over time, well, barring Ug the caveman when he learned hitting that saber-tooth tiger with his/her club harder led to a better survival result.

Pleasure and Pain.

That view of humans is overly simplistic. Temporal patterns can be unfolded from goals, for example you want the car to go faster triggers the unfolding of the change gear pattern, which sequences a series of coordinated muscle actions. But where do the goals come from? How can pattern matchers experience pleasure, pain, etc. We can feed heat sensor outputs into a pattern matcher, and this could even trigger a pattern matcher to recall a movement sequence so that a robot avoids hot things, but this robot would in no way experience pain.

feline

I think my cat can experience pleasure and pain, but I don't think he's sapient.

Sapience is a fuzzy thing,

Sapience is a fuzzy thing, it is not a binary attribute. The cat does have awareness of its self, but has a much more limited perspective about everything than humans do, so we tend to call it non-sapient.

Sapience is fuzzy, to some extent

Sapience is fuzzy, to some extent. The distinctions involved inevitably break down fractally if you look closely enough; but there's some approximately all-or-nothing stuff going on. If you've read Terrence Deacon's The Symbolic Species — which I'm more in agreement with than not — the way Deacon would put it is that my cat has no significant ability to form symbolic thoughts.

Symbolic thought comes with

Symbolic thought comes with language faculties, of which only a few animals have evolved in any significant way. Turns out when you can talk, you need to express concepts that are a bit more abstract.

Symbolic thought comes with

Symbolic thought comes with language facilities

A point on which Deacon and I might part company. Certainly symbolic thought in Deacon's sense (with he credits to CS Peirce) is not fundamentally linguistic; however, Deacon seems much more inclined than I to treat symbolic thought as an adjunct to language.

That view of humans is

That view of humans is overly simplistic.

That is why it is most likely to be true.

But where do the goals come from?

Evolution has no goals, natural selection just works to weed out things that can't compete. See this Quora question.

The robot would avoid heat simply because not doing so would cause it to be destroyed (the one that does avoid heat survives and replicates).

Goals

There are several different ideas and technologies all mixed up here, with nano-technology it may be possible to have something like this, but it would require designing an inheritance mechanism for goals, along with a mechanism for producing diversified offspring, otherwise all the machines would get destroyed by the heat and none would survive. The concept of a robot does not automatically include natural selection and nanomachines. In fact this is no longer a robot, but artificial life, and using existing DNA mechanisms might be an easier way to achieve this. If the replication mechanism is significantly more cumbersome than DNA, the artificial life will loose out to DNA life in the long term due to the difficulty in finding the correct conditions for replication (after all DNA lifeforms evolved spontaneously given this universes rules of physics). Such an endeavour is likely to be folly, and simply constructing robots to perform specific tasks is going to be far more useful, and give us usable and reliable tools.

Edit: There are ways around this, like evolving the goals in a virtual environment, and then transplanting the frozen goals to a physical robot. I would relate this to the other discussion on machine learning, and say that evolving goals is just another way of humans specifying the goals for robots, just like training a neural net can be seen as another way of programming a computer to perform a task.

I agree system 2 is taxing,

I agree system 2 is taxing, but what's really the ultimate goal of software development? Presumably to deliver a good result on a good timeline, which I believe you've expressed elsewhere too.

So to the extent proactively triggering system 2 can reduce the timeline, or deliver a better result on the same timeline, do you agree that proactively taxing you in this way is worth it?

If so, do you agree that there may exist certain critical parts of a software product's lifecycle where doing this proactively might shorten the timeline or deliver a better result on the same timeline, on average? Would you also agree that optimizing the mean is worth the wasted effort on the tail?

Assuming we can agree on that much, then it's simply a matter of finding those high value targets. I've named a few I think qualify, and ranked in order of impact I think they're: requirements gathering, core/high-value data types, contracts (sometimes), debugging (sometimes).

I agree system 2 is taxing,

I agree system 2 is taxing, but what's really the ultimate goal of software development? Presumably to deliver a good result on a good timeline, which I believe you've expressed elsewhere too.

Agreed: the goal of software development is to build things that improve our lives (good result), preferably before our competitors do (good timeline).

So to the extent proactively triggering system 2 can reduce the timeline, or deliver a better result on the same timeline, do you agree that proactively taxing you in this way is worth it?

Yes, if A implies B, and we want B, then give us A.

If so, do you agree that there may exist certain critical parts of a software product's lifecycle where doing this proactively might shorten the timeline or deliver a better result on the same timeline, on average? Would you also agree that optimizing the mean is worth the wasted effort on the tail?

There are definitely places where rigor is justified. Those places cannot be identified in general, and programming is not really a homogeneous activity. So the programs I write are vastly different from the ones you do, we maybe aren't even doing the same art, only some details are the same (we use languages to write code).

I'm coming from the design world, where System 1 rules and attempts to apply rigor to the design process have repeatedly failed but where programming is still found to be very useful.