On Tue, February 15, 2011 16:24, John Sowa said:
> ...
> But the most common mistake that Watson makes is elementary
> category
errors.
On Day 2, it also was making category answers.
In Final Jeopardy the category was "U.S. Cities", yet the
answer it came up with was "Toronto????". Sure, it indicated
that it was guessing, and a city was guessed, but it was not
a U.S. city.
Looking at Watson's 2nd and 3rd choices, one often finds the
category violated. Sometimes it is the category specified in
the question, such as when "New Zealand's 2nd largest city"
was given (in the Jeopardy category ``"Church" and "State"'')
and neither its 2nd nor 3rd choice was a New Zealand city;
the 3rd choice being the "United States".
This really did seem like a battle of the clickers. It was often
obvious that the other two clicked, but Watson beat them on the
draw. At least twice Brian Ritter beat Watson on the click when
Watson was very sure of its answer. Another time Watson was
over it's threshold, but in
its "probable" range (indicated by
yellow) and Ken Jennings beat it. Likely it was working on
strengthening its certainty level before clicking.
To give humans a chance, Watson should be programmed to click
with the average delay that a person who clicks within a second
of the completion of the clue gives -- or to give an inconsistent
delay distributed according to the temporal distribution of such
clicks.
I would like to see figures for the number of times that the other
contestants clicked their buzzers, but Watson clicked first.
> ...
> So I'm surprised that Watson made such elementary mistakes.
It strongly suggests to me that Watson did not always determine the
category and then verify that its answer fit the category. I
suppose it did for quotation marks (which mean that that string
is in the answer).
In normal Jeopardy games, Alex often explains a category in
English
but he did not do so in either day one or day two, except for the
initialism "APB" (which Watson's dictionary would certainly have).
There were also a large number of questions which were quite
straight forward look-ups. These would be no challenge for
Watson's algorithms.
-- doug foxvog
> John
=============================================================
doug foxvog
doug@xxxxxxxxxx http://ProgressiveAustin.org"I speak as an American to the leaders of my own nation. The great
initiative in this war is ours. The initiative to stop it must be ours."
- Dr. Martin Luther King Jr.
=============================================================
_________________________________________________________________
Message Archives:
http://ontolog.cim3.net/forum/ontolog-forum/ Config Subscr:
http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/ Unsubscribe: mailto:
ontolog-forum-leave@xxxxxxxxxxxxxxxxShared Files:
http://ontolog.cim3.net/file/Community Wiki:
http://ontolog.cim3.net/wiki/ To join:
http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1JTo Post: mailto:
ontolog-forum@xxxxxxxxxxxxxxxx