[Top] [All Lists]

Re: [ontolog-forum] IBM Watson on Jeopardy

To: "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: Ed Barkmeyer <edbark@xxxxxxxx>
Date: Tue, 15 Feb 2011 11:27:54 -0500
Message-id: <4D5AA98A.3030004@xxxxxxxx>

Patrick Cassidy wrote:    (01)

> Also, Watson gave "the 1920's" when the
> answer ("the twenties") the decade) had already been declared wrong.  I
> wonder what lessons Watson's developers will derived from those blunders.
>       (02)

Bear in mind that Watson didn't have audio input.  Someone would have 
had to key in the response given by a previous contestant, and that 
would have doubtless introduced a significant delay.  Watson simply 
didn't know what the prior response had been.  It couldn't hear, and no 
one told it.    (03)

I'm not sure that a wrong answer is a 'blunder', where its information 
base was apparently faulty, as in the 1920s case.  Not everything one 
reads is accurate.    (04)

The problem with "finis" was that it knew the Latin word for "end", and 
it went with that, probably because it couldn't match "terminus" with 
"terminal".  That might mean it misunderstood "based on the Latin word 
for 'end', or didn't assign enough significance to the 'based on' part. 
I would be surprised if a NLP algorithm didn't know about 'stemming', 
but the direct reference to the Latin word may well have altered its 
preference.  (I didn't see what Watson's second best answer was when it 
guessed "finis".)  Further, Watson doesn't know about 'common speech' 
per se, it uses frequency of occurrence in its reference corpora as a 
gauge.  I think this example demonstrates the behavior of its 
probabilistic reasoning quite well, and in some sense matches the 
dilemmas of reasoning and credibility algorithms in intelligence circles.    (05)

-Ed    (06)

P.S. I'm sure many were reminded of the HAL9000.  I believe Clarke wrote 
the tale about 65 years ago.    (07)

Edward J. Barkmeyer                        Email: edbark@xxxxxxxx
National Institute of Standards & Technology
Manufacturing Systems Integration Division
100 Bureau Drive, Stop 8263                Tel: +1 301-975-3528
Gaithersburg, MD 20899-8263                Cel: +1 240-672-5800    (08)

"The opinions expressed above do not reflect consensus of NIST, 
 and have not been reviewed by any Government authority."    (09)

Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx    (010)

<Prev in Thread] Current Thread [Next in Thread>