ontolog-forum
[Top] [All Lists]

Re: [ontolog-forum] IBM Watson & source code

To: doug@xxxxxxxxxx, "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: Mike Bennett <mbennett@xxxxxxxxxxxxxxx>
Date: Wed, 16 Feb 2011 02:02:21 +0000
Message-id: <4D5B302D.2000600@xxxxxxxxxxxxxxx>
I read somewhere that it (or its programmers) had decided in 
early trials that the categories were not to be completely relied 
upon, so it gave this a lesser weighting than if the question 
itself had contained the string "US City".    (01)

 From what little I've seen it does rather seem to be a triumph 
of statistics over semantics.    (02)

Mike    (03)

On 16/02/2011 01:47, doug foxvog wrote:
> On Tue, February 15, 2011 16:24, John Sowa said:
>> ...
>> But the most common mistake that Watson makes is elementary
>> category errors.
> On Day 2, it also was making category answers.
>
> In Final Jeopardy the category was "U.S. Cities", yet the
> answer it came up with was "Toronto????".  Sure, it indicated
> that it was guessing, and a city was guessed, but it was not
> a U.S. city.
>
> Looking at Watson's 2nd and 3rd choices, one often finds the
> category violated.  Sometimes it is the category specified in
> the question, such as when "New Zealand's 2nd largest city"
> was given (in the Jeopardy category ``"Church" and "State"'')
> and neither its 2nd nor 3rd choice was a New Zealand city;
> the 3rd choice being the "United States".
>
> This really did seem like a battle of the clickers.  It was often
> obvious that the other two clicked, but Watson beat them on the
> draw.  At least twice Brian Ritter beat Watson on the click when
> Watson was very sure of its answer.  Another time Watson was
> over it's threshold, but in its "probable" range (indicated by
> yellow) and Ken Jennings beat it.  Likely it was working on
> strengthening its certainty level before clicking.
>
> To give humans a chance, Watson should be programmed to click
> with the average delay that a person who clicks within a second
> of the completion of the clue gives -- or to give an inconsistent
> delay distributed according to the temporal distribution of such
> clicks.
>
> I would like to see figures for the number of times that the other
> contestants clicked their buzzers, but Watson clicked first.
>
>> ...
>> So I'm surprised that Watson made such elementary mistakes.
> It strongly suggests to me that Watson did not always determine the
> category and then verify that its answer fit the category.  I
> suppose it did for quotation marks (which mean that that string
> is in the answer).
>
> In normal Jeopardy games, Alex often explains a category in English
> but he did not do so in either day one or day two, except for the
> initialism "APB" (which Watson's dictionary would certainly have).
>
> There were also a large number of questions which were quite
> straight forward look-ups.  These would be no challenge for
> Watson's algorithms.
>
> -- doug foxvog
>
>> John
> =============================================================
> doug foxvog    doug@xxxxxxxxxx   http://ProgressiveAustin.org
>
> "I speak as an American to the leaders of my own nation. The great
> initiative in this war is ours. The initiative to stop it must be ours."
>      - Dr. Martin Luther King Jr.
> =============================================================
>
>
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/
> Community Wiki: http://ontolog.cim3.net/wiki/
> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>
>
>    (04)


-- 
Mike Bennett
Director
Hypercube Ltd.
89 Worship Street
London EC2A 2BF
Tel: +44 (0) 20 7917 9522
Mob: +44 (0) 7721 420 730
www.hypercube.co.uk
Registered in England and Wales No. 2461068    (05)


_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx    (06)

<Prev in Thread] Current Thread [Next in Thread>