Bill,
> - I would not expect the "gatekeeper" to exclude any particular
> ontology from an "open" repository.
There seems to be a misunderstanding. Gatekeeping -- at least in the
context of this thread -- is exactly about the minimal criteria that any
ontology needs to meet in order to be accepted as part of the repository. (01)
Best
Fabian (02)
Bill Bug wrote:
> On quick point I left out - I would not expect the "gatekeeper" to
> exclude any particular ontology from an "open" repository. Rather -
> as others have said - the gatekeepers would help to establish and
> populate the required metadata needed to provide for the vetting
> capability. In this way, the gatekeeper function we are discussing
> here is very much different from the process of peer review, where
> those manuscripts that don't pass muster never end up in the repository.
>
> On Mar 21, 2008, at 2:35 AM, Bill Bug wrote:
>> Mark is making a very import point regarding the problematic nature
>> of the analogy between journal peer reviewers and OOR gatekeepers.
>> To some extent this is an apples to oranges comparison, despite the
>> apparent similarity in the gatekeeper function.
>>
>> Having said that, I do believe there is another function journal
>> reviewers provide which is very relevant to what I'd hope to see
>> provided by an OOR gatekeeper - at least when it comes to ontologies
>> used to help add reliable, highly-scalable automation to
>> bioinformatic data processing. To go back to the realm of
>> advertising, this other gatekeeper role relates to the Google Scholar
>> "slogan" - "Stand on the shoulders of giants". The idea is the peer
>> review process in the literature helps provide a certain level of
>> distillation and reliable vetting to ensure every researcher need
>> only read, absorb, and collate vetted manuscripts, as opposed to all
>> the manuscripts submitted to the relevant journals. As imperfect as
>> this process, I still tend to transfer my confidence in the reviewers
>> having competently vetted the articles to the articles themselves, so
>> that I can accept - at a given point in time - the validity of the
>> assertions in a given article and build on them by performing
>> additional work on their "shoulders".
>>
>> Perhaps - as others have suggested - it is too early in the evolution
>> of ontology development practice for us to expect we can produce a
>> vetting process capable of functioning in this way, but until we do,
>> some of the expectations informaticists - and funding agencies - have
>> for the use of ontologies are probably not achievable. Until there is
>> a reliable vetting procedure, we cannot expect to re-use and extend
>> existing ontologies effectively or with confidence for the purpose of
>> bringing like data together in novel ways from across the biomedical
>> data diaspora. Without vetting, we cannot expect to provide other
>> developers with clear advise on what are the reliable ontological
>> shoulders to build on. If the OOR has 3 ontologies covering a domain
>> of interest at roughly the same scope and level of granularity such
>> as the physiology of mammalian electrolyte balance or the assembly of
>> peptides into functional multimeric macromolecular receptor
>> complexes, how would a bioinformatics application developer determine
>> which one to use? Even more importantly, if users pick at random from
>> amongst the 2 or more ontologies covering the same domain, who will
>> maintain the maps and software required to make deductions or
>> inferences across the annotated data repositories which use these
>> different ontologies to cover the same domain?
>>
>> Another one of my expectations for using ontologies in biomedical
>> informatics is as the data representation gets richer and more
>> expressive, the nature of the software each application developer
>> needs create can focus more on application-specific issues.
>> Community tools capable of parsing the expressivity (reasoners) can
>> provide more of the "smarts", so that the custom tools don't need to
>> hard code it - and can exclusively focus on the application specific
>> algorithms.
>>
>> This partly goes back to a point I made earlier in this thread.
>> Using ontologies to power broadly scoped, federated inferencing
>> brings with it a distinct set of requirements that differ from those
>> of applications focused on providing decision support built on a more
>> narrowly focussed data warehouse. In that case, it can be perfectly
>> acceptable for developers to pick the ontology they like the best
>> from the several covering the domain(s) they require, so long as
>> there is no expectation the resulting knowledge repository will be
>> easily shared with other informatics projects.
>>
>> These may all be expectations so narrowly linked to biomedical
>> informatics they do not hold sway in a more generic OOR. That's one
>> of the issues I'm hoping to better understand by participating in
>> this discussion and the upcoming summit.
>>
>> Thanks again to all for the stimulating and detailed dissection of
>> this important topic.
>>
>> Cheers,
>> Bill
>>
>>
>> On Mar 21, 2008, at 1:08 AM, Mark Musen wrote:
>>> On Mar 20, 2008, at 8:56 PM, John F. Sowa wrote:
>>>> There are two independent issues here: reviewing and publishing.
>>>> Everybody would agree that reviewing is important, but ideally,
>>>> the readers/users should have the option of making their own
>>>> choices based on the reviews. When publication was expensive,
>>>> the publishers became gatekeepers because it was economically
>>>> impractical to publish everything.
>>>
>>> The analogy between peer review of journal articles and peer review of
>>> ontologies has been applied too glibly, I believe.
>>>
>>> The best reviewers of a journal article are scientists who can
>>> evaluate the methods described in the paper, judge whether the data
>>> presented are plausibly consistent with the methods, and assess
>>> whether the authors' interpretations of the data are reasonable. This
>>> process is all done rather well by scientists who are experts in the
>>> field and who can understand the work that is described in the paper.
>>> Although the system does break down, sometimes in notorious ways, it
>>> generally works rather well.
>>>
>>> Ontologies are not journal articles. Although there are many surface-
>>> level distinctions that can be assessed purely by inspection (OBO-
>>> Foundry criteria regarding representation language, namespaces,
>>> textual definitions, and so on), the key question one wants answered
>>> before using an ontology concerns whether the ontology makes the right
>>> distinctions about the domain being modeled. This question cannot be
>>> answered by inspection of the ontology; it can be answered only by
>>> application of the ontology to some set of real-world problems and
>>> discovering where things break down. The people best suited for
>>> making the kinds of assessment that are needed are not necessarily the
>>> best experts in the field, but the mid-level practitioners who
>>> actually do the work. Any effective system of peer review has got to
>>> capture the opinions of ontology users, and not just those of renowned
>>> subject-matter experts or of curators.
>>>
>>> I think ontologies are much more like refrigerators than they are like
>>> journal articles. I view ontologies as artifacts. Not surprisingly,
>>> I am much more interested in the opinions of people who actually use
>>> refrigerators than I am of experts in thermodynamics, product
>>> manufacturing, or mechanical engineering. The latter are people who
>>> can inspect a particular refrigerator very carefully for surface-level
>>> flaws, but who may have no first-hand knowledge of what happens when
>>> you actually plug it in.
>>>
>>> Mark
>>>
>>> _________________________________________________________________
>>> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
>>> Subscribe/Config:
>>> http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
>>> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
>>> Community Wiki:
>>> http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
>>> Community Portal: http://ontolog.cim3.net/
>>>
>>
>> William Bug, M.S., M.Phil.
>> email: wbug@xxxxxxxxxxxxxx <mailto:wbug@xxxxxxxxxxxxxx>
>> Ontological Engineer work: (610) 457-0443
>> Biomedical Informatics Research Network (BIRN)
>> and
>> National Center for Microscopy &
>> Imaging Research (NCMIR)
>> Dept. of Neuroscience, School of Medicine
>> University of California, San Diego
>> 9500 Gilman Drive
>> La Jolla, CA 92093
>>
>> Please note my email has recently changed
>>
>>
>>
>
> William Bug, M.S., M.Phil.
> email: wbug@xxxxxxxxxxxxxx <mailto:wbug@xxxxxxxxxxxxxx>
> Ontological Engineer work: (610) 457-0443
> Biomedical Informatics Research Network (BIRN)
> and
> National Center for Microscopy &
> Imaging Research (NCMIR)
> Dept. of Neuroscience, School of Medicine
> University of California, San Diego
> 9500 Gilman Drive
> La Jolla, CA 92093
>
> Please note my email has recently changed
>
>
>
> ------------------------------------------------------------------------
>
>
> _________________________________________________________________
> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
> Community Portal: http://ontolog.cim3.net/
> (03)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
Community Portal: http://ontolog.cim3.net/ (04)
|