Home » forums » LtU Forum

IBM's Watson plays Jeopardy champion Ken Jennings in February

This has been floating around the developer sites for the past couples days and didn't see anything related to Watson mentioned on LtU before, so we can categorize this under fun, but maybe also get some more technical speculation on what's going on over at IBM.

IBM's Watson computer to play Jeopardy against champions

By Roy Batty at 2010-12-16 21:08 | LtU Forum | previous forum topic | next forum topic | other blogs | 6139 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Watson is brute force computation

Watson has been known for awhile... IBM isn't interested necessarily in really clever AI so much as figuring out high quality brute force methods.

Myself and some others really love the idea behind Watson, regardless of how inelegant it may be. Some killer apps have to be created to take advantage of all these cores, and the most likely candidate in Fortune x00 companies is the return of the dumb terminal and uber-intelligent "smart servers".

By Z-Bo at Thu, 2010-12-16 21:12 | login or register to post comments

I lost on Jeopardy, baby!

Watson took all in today's game. The implications for programming languages are vast. In particular:

  • How long before a supercomputer can just data mine a lot of code + documentation to come up with a reasonable answer to a software component request? All you have to do is form your request as an answer that contains all your requirements.
  • Should be great for diagnosis systems, including ones that diagnose programs. Just take the world + lots of log information and have it come up with answers.

Taking advantage of clever AI, machine learning, and huge amounts of data, perhaps this is the next frontier for PL.

By Sean McDirmid at Thu, 2011-02-17 05:24 | login or register to post comments

Watson is only funny because

Watson is only funny because all the answers are known in advance and you can see where Watson and the human candidates are right or wrong. So there is always a critical mind which guarantees correctness even if it is not among the players but the Jury [1]. The key issue is the quality of the critical minds and their availability in human actions, not the speed of information retrieval.

[1] One could argue that uncritically relying on the Jury is the wrong attitude as well. We are merely conditioned to believe in this constellation as a guarantee for objectivity but it's a simulacrum after all. Another fun idea for an SF-story was that politicians stage human decision making processes, hiding the fact that they query Watson like machines for that purpose.

By Kay Schluehr at Thu, 2011-02-17 06:25 | login or register to post comments

Trivia is of course about

Trivia is of course about pre-existing knowledge. No one creates something. Being a medical doctor is basically about being good at trivia: you have to remember quickly answers pre-discovered given observed symptoms. Some doctors discover new information via experimentation and the critical mind, but this is relatively rare to knowledge regurgitation. Watson-like systems can near-term free humans up from dumb trivia tasks. This is what 80% of programming work is, of course, so it should be useful to us.

By Sean McDirmid at Thu, 2011-02-17 11:43 | login or register to post comments

data interchange format trivia

Mendelbaum recently posted here about his work with Fisher on data interchange formats and Lens-like parsing work. To me, I didn't understand the point of his work there. Especially for example his paper's example of parsing CPT HCPCS codes from medical files.

To me, the issues run deeper than that (from in the trenches experience).

What would be cool is if I could say to my IDE/language environment, "I need to tell somebody about the schema for my system. What XML format should I use?" And then on the other end, when the other dude gets the file he says, "crap, how do i read this file?" And the computer just figures it out for you.

By Z-Bo at Thu, 2011-02-17 20:54 | login or register to post comments

The computer's advantage is

The computer's advantage is that it could give you reasonable answers to questions of the form "I want to do A in the context of B" or "I want to do A for reason B." Watson style resolution then looks at a huge corpus of programs and tries to connect the dots for you, or at least proposes ways of connecting the dots. This eliminates Googling and going to stack overflow to solve routine programming problems.

As for another approach, one of my colleagues working in ML/speech recognition was talking about someone who trained a stateful neural network (one with feedback loops) on strings of characters. Given a string of characters, the network would predict the next character, and then the next one and the one after that, and so on. Now, train this neural network on wikipedia as a corpus; start a sentence and let the network complete it. The results are very freaky, the network was able to produce grammatically correct sentences that was able to match quotes correctly. The sentences were even meaningful given enough of an initial sentence to start with. Now, imagine apply such a technique to writing code, scary.

By Sean McDirmid at Fri, 2011-02-18 14:42 | login or register to post comments

I must say I fail to see

I must say I fail to see the PL connection (Sean statement notwithstanding).

By Ehud Lamm at Thu, 2011-02-17 08:19 | login or register to post comments

Hard problems PL community has not effectively tackled

Well, let's see... object-oriented programming was sold as a bill of goods for software re-use by academics.

This sort of brute-force computation might be more effective than anything humans could do by themselves.

By Z-Bo at Thu, 2011-02-17 16:28 | login or register to post comments

Coefficient of relevance

The story would be more PL-relevant if part of it was about what properties the AI program demanded from its programming language, or what kind of infrastructure the implementors were forced into implementing themselves because of the defects of available PLs.

Without that, it's hard for me to agree that the story says anything about hard problems that the PL community has failed to solve.

By Charles Stewart at Sat, 2011-02-19 16:17 | login or register to post comments

Symbolic vs. ML AI

Wolfram has a nice post about Watson as it compares to Wolfram Alpha:

http://blog.stephenwolfram.com/2011/01/jeopardy-ibm-and-wolframalpha/

The IBM approach (and most search engines) are based on machine learning (ML), while Alpha is based on pure deterministic symbolic reasoning. An infatuation with symbolic reasoning is basically what caused the AI winter, while ML* has pulled AI out of that winter and has been on the up and up for a while now. Most programming languages (definitely Prolog) can be seen as being based on limited symbolic reasoning engines, while one language in particular (LISP) was designed to implement symbolic reasoning engines. In this regard, Alpha begins to look like a programming language that is based on flexible natural language text.

*[Unfortunate that ML is also the name of a programming language]

As always, Wolfram says some crazy things, like:

Wolfram|Alpha is a completely different kind of thing—something much more radical, based on a quite different paradigm. The key point is that Wolfram|Alpha is not dealing with documents, or anything derived from them. Instead, it is dealing directly with raw, precise, computable knowledge. And what’s inside it is not statistical representations of text, but actual representations of knowledge.

Strange, I thought it was the other way around: that ML was the more radical approach while the symbolic approach by itself was tried and not seen as very scalable. It basically requires a lot of programming to manipulate symbols, while Watson just needs to be trained on large amounts of data. Data-sets are added to Alpha very slowly, and they can't be integrated without explicit support in the system.

Two thoughts here:

  1. ML eventually puts PL out of business as it becomes the stronger approach. Its difficult to see how a computer might eventually learn and apply math on its own. But then again, we just might not know what the appropriate corpus looks like yet. Still, even if computers become smart, we still need ways of asking our questions or giving our commands, but this might just be NLP.
  2. How about programming languages designed to guide and/or harness ML? All those fuzzy connections learned by the system, not reliable at all but they have some level of accuracy. One way is to leverage ML in the IDE while writing code, sort of advanced auto-completion. The other way is by basing a language on a ML core, where libraries are trained data sets or additional corpora.
By Sean McDirmid at Thu, 2011-02-17 11:37 | login or register to post comments

Strange, I thought it was

Strange, I thought it was the other way around: that ML was the more radical approach while the symbolic approach by itself was tried and not seen as very scalable. It basically requires a lot of programming to manipulate symbols, while Watson just needs to be trained on large amounts of data. Data-sets are added to Alpha very slowly, and they can't be integrated without explicit support in the system.

Something Rich Skrenta mentioned at the Search Summit, while trying to get a word inbetween Matt Cutts and Harry Shum's childish stupidity, was that MS Research published a paper a few years ago showing how the number of interesting data sets is vanishingly small (there are maybe 50 millionin "interesting" web sites), and that it might make sense to brute-force optimize search results for the interesting data sets, e.g. by slashing out content farms and also manually identifying interesting data sets.

By Z-Bo at Thu, 2011-02-17 16:30 | login or register to post comments

is semantic web design a PLT issue?

Is semantic web design (mark-up standards, data format standards) a PLT issue? By definition, such standards of communication don't specify any completely specific class of operational meanings. They involve questions of syntax, of human connotation, of ease of use and processing, and, in some cases some partial operational meanings. Are questions about the design of those things PLT questions?

I am thinking along these lines:

Perhaps "the web" or "the net" is an interpreter for an unspecified and unspecifiable but partially knowable platform. Can it not be programmed? You publish data in some way and in some format on "the web" and interpretation happens as a result. Some aspects of what result are predictable, others aren't. Some have solid meaning, others don't. Is the uncertainty about the nature of the interpreter a barrier to PLT or an interesting frontier?

If it is PLT fair game, since some of the nature of how "programs" in this domain are interpreted is determined by and determines how humans act in unprecedentedly broad ways... what are the ethical issues?

By Thomas Lord at Fri, 2011-02-18 00:26 | login or register to post comments

Designing abstractions is at

Designing abstractions is at the core of PL. But increasingly, it is also a systems issue, a data mining issue a...well, let's just say that there are nice PL problems in many fields of applied computer science. There is a reason why Google hires so many PL PhDs (among others), yet these guys are mostly all working on abstraction and systems issues and not designing/implementing languages.

The holy grail of PL would be to figure out how the universe is programmed and design a language to program our own :). Barring that, getting a handle on knowledge is a nice intermediate goal.

By Sean McDirmid at Fri, 2011-02-18 14:34 | login or register to post comments

Web logic

I think the semantic web is well-suited to PL theory: formal ontologies as used in the web closely correspond to types, and logics used for the semantic web correspond to relations on those types that can be well or badly supported by the type system of a programming logic.

Something like OWL can be seen to suggest a type system for Prolog, say.

By Charles Stewart at Sat, 2011-02-19 16:25 | login or register to post comments