> But note that one of Popper's most famous points is that a scientific
theory must be falsifiable. (01)
The arguments about whether theories are disproved or just revised to
account for new observations is not the one I am addressing, although
that is an interesting discussion. (02)
John, I think the crux of our disagreement is in regard to the meaning
of "falsifiable". This is a characteristic of the theory, not the
observations. I haven't seen a definition of this term that addresses
the issue of uncertainty of observations, so I'll propose one based on
how I have understood and used this principle in my own scientific career. (03)
A theory is falsifiable if it makes a prediction that could be proved
false if perfect information (with no uncertainty) about some
observable(s) were available. (04)
Ignoring or underestimating the uncertainty of observations is a pet
peeve of mine - Start of rant. (05)
No theory can mitigate the uncertainty of observations. And observations
always have some uncertainty. In some cases, the uncertainty is so small
that we consider it "negligible" - not worthy of even estimating the
magnitude. But it is never zero. (06)
Did the sun "rise" this morning? Most inhabitants of the earth (in the
appropriate time zones and in a setting to make the observation) would
agree that it did. Some persons - who are mentally ill , under the
influence of hallucinogens, or overly pedantic (the sun didn't rise, the
earth rotated) - might disagree. Is it physically possible that all the
people who observed the sun to rise this morning actually hallucinated
this event instead? Yes, although so unlikely that we consider it
negligible. (07)
The point I would like to emphasize is that there is no sharp boundary
between the negligible uncertainties and those that are not negligible.
The observations that are most likely to be of significance in
evaluating the hypotheses on the frontiers of science have a lot of
uncertainty. (08)
Further, scientists (in general) tend to underestimate the uncertainty
of their own observations. In one phase of my career, I was involved in
a number of expert elicitations in order to quantify uncertainties of
some observations that were used for Monte-Carlo based risk assessments
of certain scenarios related to nuclear waste at Hanford. The experts
that conducted these elicitations were always careful to conduct
uncertainty training of their subjects beforehand. (09)
The test is as follows: given 10 questions with numerical answers that
are recorded in references with high accuracy but are not general
knowledge (example: the length of the Nile river, the date Martin Luther
King Jr. was assassinated), generate a 90%-credible interval for your
own knowledge of this information. The correct answer should be within
the 90%-credible interval for 9 of the 10 questions (on average). (010)
(It is of course possible to cheat, giving wildly large intervals for 9
questions, and a very small one for 1 question.) (011)
The typical score, for highly-qualified Ph.D. scientists and
statisticians, was 20-40%. Disclosure: I scored 20% on my first
assessment, but improved to 80% on the the next round, and I didn't
cheat. Statisticians generally did worse than physical scientists.
Meteorologists tend to have the best performance - they get daily
feedback on the accuracy of their predictions. Most scientists don't
receive this kind of feedback, nor are such analyses routinely conducted
regarding published results. Instead, there is a (scientific-)cultural
discouragement for admitting a realistic level of uncertainty, and
research quality is based on other metrics quite unrelated to this, such
as number of citations, etc. (012)
End of rant. (013)
Tara (014)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J (015)
|