John,
Among the 'reviewers' is there any reason not to have an expert committee
that can create a binary distinction of, e.g. "well-structured" and "not
well-structured"? The imprimatur can be an alternative to absolute
exclusion, and still serve the legitimate concerns that Barry has about
poorly constructed ontologies. (01)
Pat (02)
Patrick Cassidy
MICRA, Inc.
908-561-3416
cell: 908-565-4053
cassidy@xxxxxxxxx (03)
> -----Original Message-----
> From: ontology-summit-bounces@xxxxxxxxxxxxxxxx [mailto:ontology-summit-
> bounces@xxxxxxxxxxxxxxxx] On Behalf Of John F. Sowa
> Sent: Thursday, March 20, 2008 11:56 PM
> To: Ontology Summit 2008
> Subject: Re: [ontology-summit] [Quality] What means
>
> Pat, Barry, Deborah, and Ed,
>
> Barry asked an important question that gets to the heart of
> the issues we have been discussing:
>
> BS> What are scientific journals for? Why do they employ a peer
> > review process?
>
> There are two independent issues here: reviewing and publishing.
> Everybody would agree that reviewing is important, but ideally,
> the readers/users should have the option of making their own
> choices based on the reviews. When publication was expensive,
> the publishers became gatekeepers because it was economically
> impractical to publish everything.
>
> But with the WWW, new options are available. Publication is
> almost free, and we have the luxury of decoupling the reviewing
> process from the gatekeeping process. Metadata enables that
> decoupling:
>
> 1. All submissions to the OOR can be made available as soon
> as they are submitted.
>
> 2. The metadata associated with each submission can indicate
> what tests were made, what the reviewers said, and what
> results the users, if any, obtained.
>
> 3. Users can choose to see ontologies sorted by any criteria
> they want: in the order of best reviews, most thorough
> testing, greatest usage, greatest relevance to a particular
> domain, or any weighted combination.
>
> PH> This is where I part company with Barry, and indeed where I
> > believe that the very idea of controlling the contents of an OOR
> > (noting that the first O means 'open') needs to be examined very,
> > very carefully. Of course we would not argue that majority voting
> > should be used to choose scientific theories; but ontologies,
> > even those used by scientists, are not themselves scientific
> > theories.
>
> Ontologies overlap philosophy, engineering, science, and mathematics.
> The closest model we have is the metadata registry, but new policies
> can and should be explored.
>
> BS>> While refrigerator manufacturers may allow democratic ranking
> >> to influence e.g. size and color, they would use other strategies
> >> e.g. in matters of thermodynamics.
>
> PH> Perhaps so: but we are here discussing matters of ontology, and
> > in the current state of the art, this may have more in common
> > with consumer product choice than with thermodynamics.
>
> That is the point I was trying to emphasize. The application
> developers have deeper understanding of their specific needs and
> problems than any general gatekeeper or committee of gatekeepers.
>
> DM> CSI, the specification writing organization for building
> > architecture, says quality is "a mirror of the requirements."
>
> That's a good point, which implies that different set of
> requirements might lead to a different ranking of the same
> ontologies. No gatekeeper can anticipate the requirements
> of all possible users.
>
> DM> Do you think the gatekeepers can help define the OOR requirements
> > and set up the dynamic tests?
>
> I'd prefer to keep the reviewers and replace the gatekeepers with
> caretakers who have a broader role along the lines you suggested.
>
> EB> I'm thinking about bureaucrats. I think that many ontologies
> > (and more broadly, concept systems including thesauri, taxonomies,
> > etc.) have been and will be developed for use within the mission
> > areas of government agencies. There can be a vetting process to
> > "approve" a concept system/ontology for use within a community
> > of interest.
>
> That suggests a further refinement of the roles of reviewers and
> gatekeepers/caretakers. At the source, there are individuals and/or
> organizations, who develop ontologies and make them available.
> Among the users, there may be organizations, coalitions, or
> bureaucracies that evaluate the ontologies and determine which
> of them are best suited to their groups of users.
>
> That is another reason for replacing the gatekeepers in the OOR
> with caretakers. Any gatekeeping that might be useful would be
> better done by user groups at a level close to the applications
> than by any gatekeeper that is close to the ontology providers.
>
> John
>
>
>
> _________________________________________________________________
> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-
> summit/
> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
> Community Wiki: http://ontolog.cim3.net/cgi-
> bin/wiki.pl?OntologySummit2008
> Community Portal: http://ontolog.cim3.net/ (04)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
Community Portal: http://ontolog.cim3.net/ (05)
|