ontolog-forum
[Top] [All Lists]

Re: [ontolog-forum] IBM Watson's Final Jeopardy error "explanation"

To: ontolog-forum@xxxxxxxxxxxxxxxx
From: Ron Wheeler <rwheeler@xxxxxxxxxxxxxxxxxxxxx>
Date: Wed, 16 Feb 2011 17:42:24 -0500
Message-id: <4D5C52D0.6060809@xxxxxxxxxxxxxxxxxxxxx>
On 16/02/2011 2:33 PM, Mike Bennett wrote:
> Thanks. Very interesting. There's a few of us looking at using
> semantics to enable systemic risk management applications
> development. The use of this kind of thing by big investors is
> going to change the landscape considerably I think.
>
It is going to make it much harder to trace back the methodology used to 
develop a risk assessment.
The algorithms and logic used to analyze the impact of a Russian 
agricultural forecast on the price of oil may be hard for traditional 
stock brokers, analysts and traders to follow.    (01)

Would it have caught the fact that a lot of the incomes quoted in 
mortgage applications were too high for the job titles and historical 
incomes of the persons who were granted the mortgages?
The rating agencies did not (or chose not to consider it as a factor in 
rating the securities) so the bar for "better" is set pretty low.    (02)

At least it would have read all of the contracts!    (03)

The adoption rate will be interesting to watch. I suspect that once 1 
institution buys in the rest will follow quickly so as not to be left 
behind.
Unfortunately it could take a few years to see if Watson is really 
making better decisions and by, that time, no one will know how to go 
back to the "old" ways if it really turns out wrong.    (04)


Ron    (05)

> Mike
>
> On 16/02/2011 19:01, Ron Wheeler wrote:
>> On 16/02/2011 1:30 PM, Mike Bennett wrote:
>>> Agreed, I think there are some very interesting avenues to
>>> explore for systemic risk analysis.
>> IBM's video on the Financial industry's possible use of Watson
>> http://www.youtube.com/watch?v=IV6mfWWMIVg&feature=channel
>> A bit short on details but gives some ideas about where IBM sees this
>> applying.
>>
>> The comments section under the video is fun to look at as well.
>> Ron
>>> Mike
>>>
>>> On 16/02/2011 18:25, Ron Wheeler wrote:
>>>> On 16/02/2011 12:16 PM, Mike Bennett wrote:
>>>>> I think the financial industry will stick to semantics rather
>>>>> than statistics for now, at least for reporting and risk management.
>>>>>
>>>> Reporting at the authoring end but I suspect that at the receiving end
>>>> the government and NGOs are going to be very interested in tools that
>>>> help correlate reported data with other industry news and reports.
>>>>
>>>> Risk management is such a broad category and entails so much information
>>>> that I suspect that a system that can identify and explain the increased
>>>> risk to a portfolio caused by a flood in southern China and a
>>>> demonstration in Cairo on the same day will be hard to resist.
>>>>
>>>>> Stock picking is another matter, as noted by Ron.
>>>> The IBM material that was issued around the "Watson/Jeopardy event"
>>>> specifically mentioned M&A target identification and the feeling
>>>> expressed by one analyst in an interview, that analysts are inundated
>>>> with news, reports and trading patterns/events.
>>>>>       I wonder if
>>>>> this thing has significantly increased the risk (or scale) of
>>>>> another Flash Crash type of event?
>>>>>
>>>> Anything that can get everyone making the same decision at the same time
>>>> is bound to cause some very interesting "gold rushes" where there are
>>>> more losers than winners but lots of transactions for the
>>>> intermediaries. The guys selling logistics in a gold rush almost always
>>>> made more money than the miners.
>>>>
>>>> Ron
>>>>
>>>>> Mike
>>>>>
>>>>> On 16/02/2011 15:37, doug foxvog wrote:
>>>>>> Question answering by a machine such as Watson is not very useful unless
>>>>>> the system can explain its answers.  Thus i am disappointed that IBM's
>>>>>> "explanation" of its error in Final Jeopardy does not explain why it
>>>>>> chose its answer.
>>>>>>
>>>>>> On Tue, February 15, 2011 21:19, ZENG, MARCIA said:
>>>>>>> FYI:
>>>>>>> Watson's Final Jeopardy Blunder In Day 2 Of IBM Challenge was updated:
>>>>>>>
>>>>>>> UPDATE: IBM posted this explanation for the mistake on its "Smarter
>>>>>>> Planet" blog....
>>>>>>>
>>>>>>> 
>http://www.huffingtonpost.com/2011/02/15/watson-final-jeopardy_n_823795.html
>>>>>> This explanation is little more than "Jeopardy is hard":
>>>>>> * Category names are tricky and their significance is downgraded.
>>>>>> * There are cities named Toronto in the US
>>>>>> * Toronto, Ontario, has an American League baseball team.
>>>>>>
>>>>>> This does not explain why "Toronto" was selected as a response.
>>>>>>
>>>>>>
>>>>>> In the IBM ads accompanying the show, they discuss Watson being used
>>>>>> for diagnosis -- not just for pointing out information that a doctor
>>>>>> may want to look at for a case.  However, without explanation for
>>>>>> a diagnosis -- or even for a recommendation for reviewing some paper,
>>>>>> such use seems quite improper.
>>>>>>
>>>>>> -- doug foxvog
>>>>>>
>>>>>>> On 2/15/11 8:18 PM, "John F. Sowa"<sowa@xxxxxxxxxxx>       wrote:
>>>>>>>
>>>>>>> On 2/15/2011 4:24 PM, Jerry Hobbs wrote:
>>>>>>>> Actually I think the question asked for an "anatomical anomaly",
>>>>>>>> which could be a body part, like a sixth toe, rather than a disability.
>>>>>>>> The text Watson used referred to the athlete's "wooden leg".
>>>>>>> I agree.  It found the anomaly, but it made an error in not explaining
>>>>>>> that it was an anomaly.  So it should at least get partial credit.
>>>>>>>
>>>>>>> On tonight's round, Watson did very well.  But it made a category
>>>>>>> error for which there was no excuse.
>>>>>>>
>>>>>>> The category was US Cities, and all three contestants were given
>>>>>>> 30 seconds to write their answers and the amount they were willing
>>>>>>> to bet.
>>>>>>>
>>>>>>> Question:  A city whose largest airport is the name of a war hero
>>>>>>> and whose second largest airport is the name of a WW II battle.
>>>>>>>
>>>>>>> Both humans got the answer right -- Chicago -- and they bet
>>>>>>> the maximum or almost the maximum.
>>>>>>>
>>>>>>> That was tricky because Midway Airport happens to be the
>>>>>>> name of a battle, but it was not named for the battle.
>>>>>>>
>>>>>>> But Watson wrote Toronto and bet $947 (which was a small
>>>>>>> amount, indicating that it wasn't sure).
>>>>>>>
>>>>>>> But the category was "US Cities".  It certainly should
>>>>>>> have checked.
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>>
>> _________________________________________________________________
>> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
>> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
>> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
>> Shared Files: http://ontolog.cim3.net/file/
>> Community Wiki: http://ontolog.cim3.net/wiki/
>> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>>
>>
>>
>    (06)


_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx    (07)

<Prev in Thread] Current Thread [Next in Thread>