Synthèse

Learning from Watson February 19, 2011

Now that Watson has convincingly demonstrated that machines can perform some natural language tasks more effectively than humans can (see a rerun of part of Day 1 of the Jeopardy contest), what is the proper conclusion to be drawn from it?

Should we join hands with “confederates” like Brian Christian and rally against the invasion of smart machines? (See his recent piece in the Atlantic and listen to his recent radio interview on CBC)?

Or do we conclude that machines are now (or soon will be) sentient and deserve to be spoken to with respect for their moral standing (see Peter Singer’s article “Rights for Robots“)? Or should we, like NSERC Gold Medal Award winner Geoffrey Hinton,  be scared about the social consequences (in the long term) of intelligent robots designed replace soldiers (listen to his interview on the future of AI machines on CBC’s Quirk and Quarks).

Before coming to any definite conclusion about how “like” us machines can be, I think we should consider how these machines do what they do.  The survey paper in AI Magazine about the design of “DeepQA” by the Watson team gives some indications of the general approach:

DeepQA is a massively parallel, probabilistic evidence-based architecture. For the Jeopardy Challenge, we use more than 100 different techniques for analyzing natural language, identifying sources, finding and generating hypotheses, finding and scoring evidence, and merging and ranking hypotheses….

The overarching principles in DeepQA are massive parallelism, many experts, pervasive confi-dence estimation, and integration of shallow and deep knowledge.

Is this the right model for creating artificial cognition? Probably not. As Maarten van Emden and I argue in a recent paper on the chinese room argument and the “Human Window”, the question of whether a computer is simulating cognition cannot be decided by how effectively a computer solves a chess puzzle (for instance) but rather by the mechanism that it uses to achieve the end.

In this instance DeepQA uses and combines a number of different techniques from NLP, machine learning, distributed processing and decision theory – which is not likely to be an accurate representation of what humans actually do but it is undeniably successful at that task (see this talk on YouTube about how IBM addressed the Jeopardy problem).

Geoff Hinton (in the radio interview mentioned above) speculates that Watson is a feat of special-purpose engineering but that the general-purpose solution – a large neural network that simulates the learning abilities of the brain – is what the project of AI is really about.

What we suggest in our Human Window paper is that one criterion we can use to determine whether machines are performing adequate simulations of what humans do is whether or not humans are able to follow the steps that machine is undertaking. On that criterion, I think it’s safe to say that Watson – although very impressive – isn’t quite there yet.

P.S. If you have the patience, I recommend watching a BBC debate from 1973 between Sir James Lighthill, John McCarthy and Donald Michie about whether AI is possible. The context of this video is the “Lighthill Affair” in 1972, recently chronicled on van Emden’s blog (note that the audio on this thumbnail video is rather out of synch!).

It’s amazing how spectacularly wrong an amateur in artificial intelligence (Prof. Lighthill was an applied mathematician specializing in fluid dynamics) can be about the possibiliy of machines simulating intelligent behaviour. It is real tragedy that Sir Lighthill’s ideological biases had such disastrous consequences for AI research funding in the UK. The attitude of Sir Lighthill reminds me of Samuel Wilberforce‘s objections  to Darwin’s theory of evolution. I find it astonishing that this BBC debate was so civilized in its demeanour.