John, (01)
> But the most common mistake that Watson makes is elementary
> category errors. For the example Jerry cites, the answer was
> supposed to be a disability. (02)
Actually I think the question asked for an "anatomical anomaly",
which could be a body part, like a sixth toe, rather than a disability.
The text Watson used referred to the athlete's "wooden leg". (03)
-- Jerry (04)
-------------------------------------------------------------------- (05)
Jerry and Doug, (06)
Those are problems that I thought they should have fixed: (07)
JH (08)
> very often an answer is wrong because it is incomplete, e.g.,
> Watson's "leg" when "missing a leg" was the right answer. (09)
DF (010)
> Watson does not "understand" its input text. It probabilistically
> finds its answer, often getting it right. (011)
One of the participants listed on the IBM web site is Michael McCord,
who had developed an excellent Prolog-based parser in the 1980s,
which he rewrote in C in the mid 1990s. I noticed that Michael
was listed on the IBM web site as one of the participants in the
Watson project. (012)
But the most common mistake that Watson makes is elementary
category errors. For the example Jerry cites, the answer was
supposed to be a disability. (013)
Any halfway decent type hierarchy (not even a deeply axiomatized
ontology) combined with a decent parser should have detected
the fact that Leg is not a subtype of Disability. But the
situation of a missing a body part would be a disability. (014)
As Doug mentioned, Watson depends very heavily on statistics.
I believe that it's important to use statistics for interpreting
NL text, but as a *supplement* to the parsing and semantic
analysis. (015)
For more info about Watson, following is an IBM technical report
written in 2009: (016)
http://domino.watson.ibm.com/library/cyberdig.nsf/papers/D12791EAA13BB952852575A1004A055C/$File/rc24789.pdf (017)
And following is an article in AI Magazine published in 2010: (018)
http://www.stanford.edu/class/cs124/AIMagzine-DeepQA.pdf (019)
Following is an excerpt from p. 11 of the article: (020)
> The DeepQA approach encourages a mixture of experts at this
> stage [question analysis], and in the Watson system we
> produce shallow parses, deep parses (McCord 1990), logical
> forms, semantic role labels, coreference, relations,
> named entities, and so on, as well as specific kinds
> of analysis for question answering. (021)
I believe that combination of "experts" working together is good.
Note that McCord is cited for the parser, they generate a logical
form, and they use semantic role labels. (022)
As a matter of fact, I was consulting on a project at IBM on the
West coast in the mid 1990s, which used McCord's parser. That
project also paid CSLI at Stanford to add semantic role labels
to the verbs McCord was using, and IBM later donated that list
of verbs with attached roles to open source: (023)
http://www-csli.stanford.edu/~arunm/ (024)
So I'm surprised that Watson made such elementary mistakes. (025)
John (026)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (027)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (028)
|