On Wed, Nov 9, 2011 10:07 AM EST Ali SH wrote: (01)
>Hello all,
>
>I recently came across this article:
>http://www.sciencenews.org/view/feature/id/335872/title/Odds_Are,_Its_Wrong
>
>Beyond the broader scientific issues it discusses, I think it is of
>particular note for ontologists. (02)
Ali,
Thank-you for the post. I am not "ontologist" but ontological questions are
inevitable if one is to make any sense of reality. It is also inevitable in any
scientific method. The real issue is efficiency and that was always an issue
with data analysis. The nature itself seem to have opted for statistical
processing first and logic following from it. The two sides can never be
completely separated but the balance is a matter of efficiency - not principle. (03)
>The basic thesis of the post is that many scientists have fundamentally
>misunderstood the meaning of statistical significance, and make several
>erroneous assumptions and conclusions while committing several logic errors
>(i.e. transposing the conditional). The article further highlights how an
>unexamined ontology can lead to a faulty meta-theory which fatally wounds
>the intended statistical interpretation of the results.
>
>It’s science’s dirtiest secret: The “scientific method” of testing
>> hypotheses by statistical analysis stands on a flimsy foundation.
>> Statistical tests are supposed to guide scientists in judging whether an
>> experimental result reflects some real effect or is merely a random fluke,
>> but the standard methods mix mutually inconsistent philosophies and offer
>> no meaningful basis for making such decisions. Even when performed
>> correctly, statistical tests are widely misunderstood and frequently
>> misinterpreted. As a result, countless conclusions in the scientific
>> literature are erroneous, and tests of medical dangers or treatments are
>> often contradictory and confusing.
>
>
>If there were ever a blatant, inadvertent call for the deployment of
>ontology throughout the sciences, this might be it. Without actually
>stating it, this article is largely about ontology and to some degree,
>collaborative belief revision.
>
>For one, it highlights the need of making implicit semantics explicit. The
>basic problem of misapplying inconsistent philosophies arises from a lack
>of understanding and due to the fact that many of the deployed assumptions
>remain implicit and their interactions are not properly evaluated or
>verified. Ontology would help by first explicating these assumptions and in
>so doing, clarify the intended meanings while highlighting potential
>inconsistencies. Mind you, I think this is applied ontology in a broader
>sense than the popular SemWeb implementations; I don't think a lightweight
>ontology would necessarily help, nor would a computational ontology be
>necessarily required. But ontological analysis, the process of making
>implicit assumptions explicit, would yield tangible benefits.
>
>The article further touches on the familiar issue of similar vocabularies
>but slight varieties in meaning; and how perhaps altogether different
>conceptualizations of procedures (methodologies) lead to problems when
>trying to combine results / hypotheses, as illustrated in the following
>passage:
>
>Another concern is the common strategy of combining results from many
>> trials into a single “meta-analysis,” a study of studies. In a single
>trial
>> with relatively few participants, statistical tests may not detect small
>> but real and possibly important effects. In principle, combining smaller
>> studies to create a larger sample would allow the tests to detect such
>> small effects. But statistical techniques for doing so are valid only if
>> certain criteria are met. *For one thing, all the studies conducted on
>> the drug must be included — published and unpublished. And all the studies
>> should have been performed in a similar way, using the same protocols,
>> definitions, types of patients and doses *[emphasis mine]*.* When
>> combining studies with differences, it is necessary first to show that
>> those differences would not affect the analysis, Goodman notes, but that
>> seldom happens. “That’s not a formal part of most meta-analyses,” he
>says.
>
> ...
>
>In principle, a proper statistical analysis can suggest an actual risk even
>> though the raw numbers show a benefit. But in this case the criteria
>> justifying such statistical manipulations were not met. In some of the
>> trials, Avandia was given along with other drugs. Sometimes the non-Avandia
>> group got placebo pills, while in other trials that group received another
>> drug. And there were no common definitions.
>
>
>
>> “Across the trials, there was no standard method for identifying or
>> validating outcomes; events ... may have been missed or misclassified,”
>> Bruce Psaty and Curt Furberg wrote in an editorial accompanying the New
>> England Journal report. “A few events either way might have changed the
>> findings.”
>
>
>Lastly, it seems to advocate for more of a Bayesian style belief revision
>when trying to combine results from multiple experiments, as opposed to the
>more AGM-like one that is implicitly being performed by many scientists (as
>described in the article). Namely it taps into the running debate as to
>whether the principle of "priority to the incoming information" (from the
>AGM postulates) should be dropped in ontology revision, i.e. [1]. Depending
>on how you read it, the article has important implications for ontology
>management, especially in collaborative socio-technical systems.
>
>[1] Mauro Mazzieri and Aldo Franco Dragoni. "Ontology Revision as
>Non-Prioritized Belief Revision." In Proceedings of International Workshop
>on Emergent Semantics and Ontology Evolution, 2007
>
>Best,
>Ali (04)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J (05)
|