Lambda the Ultimate

Watson is only funny because all the answers are known in advance and you can see where Watson and the human candidates are right or wrong. So there is always a critical mind which guarantees correctness even if it is not among the players but the Jury [1]. The key issue is the quality of the critical minds and their availability in human actions, not the speed of information retrieval.

[1] One could argue that uncritically relying on the Jury is the wrong attitude as well. We are merely conditioned to believe in this constellation as a guarantee for objectivity but it's a simulacrum after all. Another fun idea for an SF-story was that politicians stage human decision making processes, hiding the fact that they query Watson like machines for that purpose.

Trivia is of course about pre-existing knowledge. No one creates something. Being a medical doctor is basically about being good at trivia: you have to remember quickly answers pre-discovered given observed symptoms. Some doctors discover new information via experimentation and the critical mind, but this is relatively rare to knowledge regurgitation. Watson-like systems can near-term free humans up from dumb trivia tasks. This is what 80% of programming work is, of course, so it should be useful to us.

Mendelbaum recently posted here about his work with Fisher on data interchange formats and Lens-like parsing work. To me, I didn't understand the point of his work there. Especially for example his paper's example of parsing CPT HCPCS codes from medical files.

To me, the issues run deeper than that (from in the trenches experience).

What would be cool is if I could say to my IDE/language environment, "I need to tell somebody about the schema for my system. What XML format should I use?" And then on the other end, when the other dude gets the file he says, "crap, how do i read this file?" And the computer just figures it out for you.

The computer's advantage is that it could give you reasonable answers to questions of the form "I want to do A in the context of B" or "I want to do A for reason B." Watson style resolution then looks at a huge corpus of programs and tries to connect the dots for you, or at least proposes ways of connecting the dots. This eliminates Googling and going to stack overflow to solve routine programming problems.

As for another approach, one of my colleagues working in ML/speech recognition was talking about someone who trained a stateful neural network (one with feedback loops) on strings of characters. Given a string of characters, the network would predict the next character, and then the next one and the one after that, and so on. Now, train this neural network on wikipedia as a corpus; start a sentence and let the network complete it. The results are very freaky, the network was able to produce grammatically correct sentences that was able to match quotes correctly. The sentences were even meaningful given enough of an initial sentence to start with. Now, imagine apply such a technique to writing code, scary.

Well, let's see... object-oriented programming was sold as a bill of goods for software re-use by academics.

This sort of brute-force computation might be more effective than anything humans could do by themselves.

The story would be more PL-relevant if part of it was about what properties the AI program demanded from its programming language, or what kind of infrastructure the implementors were forced into implementing themselves because of the defects of available PLs.

Without that, it's hard for me to agree that the story says anything about hard problems that the PL community has failed to solve.

Strange, I thought it was the other way around: that ML was the more radical approach while the symbolic approach by itself was tried and not seen as very scalable. It basically requires a lot of programming to manipulate symbols, while Watson just needs to be trained on large amounts of data. Data-sets are added to Alpha very slowly, and they can't be integrated without explicit support in the system.

Something Rich Skrenta mentioned at the Search Summit, while trying to get a word inbetween Matt Cutts and Harry Shum's childish stupidity, was that MS Research published a paper a few years ago showing how the number of interesting data sets is vanishingly small (there are maybe 50 millionin "interesting" web sites), and that it might make sense to brute-force optimize search results for the interesting data sets, e.g. by slashing out content farms and also manually identifying interesting data sets.

Designing abstractions is at the core of PL. But increasingly, it is also a systems issue, a data mining issue a...well, let's just say that there are nice PL problems in many fields of applied computer science. There is a reason why Google hires so many PL PhDs (among others), yet these guys are mostly all working on abstraction and systems issues and not designing/implementing languages.

The holy grail of PL would be to figure out how the universe is programmed and design a language to program our own :). Barring that, getting a handle on knowledge is a nice intermediate goal.

I think the semantic web is well-suited to PL theory: formal ontologies as used in the web closely correspond to types, and logics used for the semantic web correspond to relations on those types that can be well or badly supported by the type system of a programming logic.

Something like OWL can be seen to suggest a type system for Prolog, say.