Pat, Barry, Deborah, and Ed, (01)
Barry asked an important question that gets to the heart of
the issues we have been discussing: (02)
BS> What are scientific journals for? Why do they employ a peer
> review process? (03)
There are two independent issues here: reviewing and publishing.
Everybody would agree that reviewing is important, but ideally,
the readers/users should have the option of making their own
choices based on the reviews. When publication was expensive,
the publishers became gatekeepers because it was economically
impractical to publish everything. (04)
But with the WWW, new options are available. Publication is
almost free, and we have the luxury of decoupling the reviewing
process from the gatekeeping process. Metadata enables that
decoupling: (05)
1. All submissions to the OOR can be made available as soon
as they are submitted. (06)
2. The metadata associated with each submission can indicate
what tests were made, what the reviewers said, and what
results the users, if any, obtained. (07)
3. Users can choose to see ontologies sorted by any criteria
they want: in the order of best reviews, most thorough
testing, greatest usage, greatest relevance to a particular
domain, or any weighted combination. (08)
PH> This is where I part company with Barry, and indeed where I
> believe that the very idea of controlling the contents of an OOR
> (noting that the first O means 'open') needs to be examined very,
> very carefully. Of course we would not argue that majority voting
> should be used to choose scientific theories; but ontologies,
> even those used by scientists, are not themselves scientific
> theories. (09)
Ontologies overlap philosophy, engineering, science, and mathematics.
The closest model we have is the metadata registry, but new policies
can and should be explored. (010)
BS>> While refrigerator manufacturers may allow democratic ranking
>> to influence e.g. size and color, they would use other strategies
>> e.g. in matters of thermodynamics. (011)
PH> Perhaps so: but we are here discussing matters of ontology, and
> in the current state of the art, this may have more in common
> with consumer product choice than with thermodynamics. (012)
That is the point I was trying to emphasize. The application
developers have deeper understanding of their specific needs and
problems than any general gatekeeper or committee of gatekeepers. (013)
DM> CSI, the specification writing organization for building
> architecture, says quality is "a mirror of the requirements." (014)
That's a good point, which implies that different set of
requirements might lead to a different ranking of the same
ontologies. No gatekeeper can anticipate the requirements
of all possible users. (015)
DM> Do you think the gatekeepers can help define the OOR requirements
> and set up the dynamic tests? (016)
I'd prefer to keep the reviewers and replace the gatekeepers with
caretakers who have a broader role along the lines you suggested. (017)
EB> I’m thinking about bureaucrats. I think that many ontologies
> (and more broadly, concept systems including thesauri, taxonomies,
> etc.) have been and will be developed for use within the mission
> areas of government agencies. There can be a vetting process to
> "approve" a concept system/ontology for use within a community
> of interest. (018)
That suggests a further refinement of the roles of reviewers and
gatekeepers/caretakers. At the source, there are individuals and/or
organizations, who develop ontologies and make them available.
Among the users, there may be organizations, coalitions, or
bureaucracies that evaluate the ontologies and determine which
of them are best suited to their groups of users. (019)
That is another reason for replacing the gatekeepers in the OOR
with caretakers. Any gatekeeping that might be useful would be
better done by user groups at a level close to the applications
than by any gatekeeper that is close to the ontology providers. (020)
John (021)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
Community Portal: http://ontolog.cim3.net/ (022)
|