I read somewhere that it (or its programmers) had decided in
early trials that the categories were not to be completely relied
upon, so it gave this a lesser weighting than if the question
itself had contained the string "US City". (01)
From what little I've seen it does rather seem to be a triumph
of statistics over semantics. (02)
Mike (03)
On 16/02/2011 01:47, doug foxvog wrote:
> On Tue, February 15, 2011 16:24, John Sowa said:
>> ...
>> But the most common mistake that Watson makes is elementary
>> category errors.
> On Day 2, it also was making category answers.
>
> In Final Jeopardy the category was "U.S. Cities", yet the
> answer it came up with was "Toronto????". Sure, it indicated
> that it was guessing, and a city was guessed, but it was not
> a U.S. city.
>
> Looking at Watson's 2nd and 3rd choices, one often finds the
> category violated. Sometimes it is the category specified in
> the question, such as when "New Zealand's 2nd largest city"
> was given (in the Jeopardy category ``"Church" and "State"'')
> and neither its 2nd nor 3rd choice was a New Zealand city;
> the 3rd choice being the "United States".
>
> This really did seem like a battle of the clickers. It was often
> obvious that the other two clicked, but Watson beat them on the
> draw. At least twice Brian Ritter beat Watson on the click when
> Watson was very sure of its answer. Another time Watson was
> over it's threshold, but in its "probable" range (indicated by
> yellow) and Ken Jennings beat it. Likely it was working on
> strengthening its certainty level before clicking.
>
> To give humans a chance, Watson should be programmed to click
> with the average delay that a person who clicks within a second
> of the completion of the clue gives -- or to give an inconsistent
> delay distributed according to the temporal distribution of such
> clicks.
>
> I would like to see figures for the number of times that the other
> contestants clicked their buzzers, but Watson clicked first.
>
>> ...
>> So I'm surprised that Watson made such elementary mistakes.
> It strongly suggests to me that Watson did not always determine the
> category and then verify that its answer fit the category. I
> suppose it did for quotation marks (which mean that that string
> is in the answer).
>
> In normal Jeopardy games, Alex often explains a category in English
> but he did not do so in either day one or day two, except for the
> initialism "APB" (which Watson's dictionary would certainly have).
>
> There were also a large number of questions which were quite
> straight forward look-ups. These would be no challenge for
> Watson's algorithms.
>
> -- doug foxvog
>
>> John
> =============================================================
> doug foxvog doug@xxxxxxxxxx http://ProgressiveAustin.org
>
> "I speak as an American to the leaders of my own nation. The great
> initiative in this war is ours. The initiative to stop it must be ours."
> - Dr. Martin Luther King Jr.
> =============================================================
>
>
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/
> Community Wiki: http://ontolog.cim3.net/wiki/
> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>
>
> (04)
--
Mike Bennett
Director
Hypercube Ltd.
89 Worship Street
London EC2A 2BF
Tel: +44 (0) 20 7917 9522
Mob: +44 (0) 7721 420 730
www.hypercube.co.uk
Registered in England and Wales No. 2461068 (05)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (06)
|