[Top] [All Lists]

Re: [ontolog-forum] Philosophy of science / ontology (was Dennett... )

To: ontolog-forum@xxxxxxxxxxxxxxxx
From: Tara Athan <taraathan@xxxxxxxxx>
Date: Thu, 09 May 2013 12:28:56 -0400
Message-id: <518BCEC8.5090206@xxxxxxxxxx>
 > But note that one of Popper's most famous points is that a scientific 
theory must be falsifiable.    (01)

The arguments about whether theories are disproved or just revised to 
account for new observations is not the one I am addressing, although 
that is an interesting discussion.    (02)

John, I think the crux of our disagreement is in regard to the meaning 
of "falsifiable". This is a characteristic of the theory, not the 
observations. I haven't seen a definition of this term that addresses 
the issue of uncertainty of observations, so I'll propose one based on 
how I have understood and used this principle in my own scientific career.    (03)

A theory is falsifiable if it makes a prediction that could be proved 
false if perfect information (with no uncertainty) about some 
observable(s) were available.    (04)

Ignoring or underestimating the uncertainty of observations is a pet 
peeve of mine - Start of rant.    (05)

No theory can mitigate the uncertainty of observations. And observations 
always have some uncertainty. In some cases, the uncertainty is so small 
that we consider it "negligible" - not worthy of even estimating the 
magnitude. But it is never zero.    (06)

Did the sun "rise" this morning? Most inhabitants of the earth (in the 
appropriate time zones and in a setting to make the observation) would 
agree that it did. Some persons - who are mentally ill , under the 
influence of hallucinogens, or overly pedantic (the sun didn't rise, the 
earth rotated) - might disagree. Is it physically possible that all the 
people who observed the sun to rise this morning actually hallucinated 
this event instead? Yes, although so unlikely that we consider it 
negligible.    (07)

The point I would like to emphasize is that there is no sharp boundary 
between the negligible uncertainties and those that are not negligible. 
The observations that are most likely to be of significance in 
evaluating the hypotheses on the frontiers of science have a lot of 
uncertainty.    (08)

Further, scientists (in general) tend to underestimate the uncertainty 
of their own observations. In one phase of my career, I was involved in 
a number of expert elicitations in order to quantify uncertainties of 
some observations that were used for Monte-Carlo based risk assessments 
of certain scenarios related to nuclear waste at Hanford. The experts 
that conducted these elicitations were always careful to conduct 
uncertainty training of their subjects beforehand.    (09)

The test is as follows: given 10 questions with numerical answers that 
are recorded in references with high accuracy but are not general 
knowledge (example: the length of the Nile river, the date Martin Luther 
King Jr. was assassinated), generate a 90%-credible interval for your 
own knowledge of this information. The correct answer should be within 
the 90%-credible interval for 9 of the 10 questions (on average).    (010)

(It is of course possible to cheat, giving wildly large intervals for 9 
questions, and a very small one for 1 question.)    (011)

The typical score, for highly-qualified Ph.D. scientists and 
statisticians, was 20-40%. Disclosure: I scored 20% on my first 
assessment, but improved to 80% on the the next round, and I didn't 
cheat. Statisticians generally did worse than physical scientists. 
Meteorologists tend to have the best performance - they get daily 
feedback on the accuracy of their predictions. Most scientists don't 
receive this kind of feedback, nor are such analyses routinely conducted 
regarding published results. Instead, there is a (scientific-)cultural 
discouragement for admitting a realistic level of uncertainty, and 
research quality is based on other metrics quite unrelated to this, such 
as number of citations, etc.    (012)

End of rant.    (013)

Tara    (014)

Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (015)

<Prev in Thread] Current Thread [Next in Thread>