> == Summary: Stewardship/ Quality Control==
> The community agrees that it is not sufficient for the OOR just to
> store
> ontologies, but that it needs to provide the possibility to evaluate
> the
> ontologies within it. There is no agreement on how to evaluate
> ontologies; the main strategies suggested are: (i) A market driven
> approach where ontologies are reviewed by users and ranked like
> items on
> Amazon.com; and (ii) an editorial process where ontologies are
> reviewed
> by experts in a similar way as papers which are submitted to
> scientific
> journals. This conflict is not surprising, since the different members
> of this community are using ontologies for different purposes and thus
> have different perspectives on what ontologies are and how they are
> supposed to be evaluated. (01)
Fabian, (02)
thanks for summarizing the discussion. I do not want to reopen the
whole discussion on quality control, I just want to stress again that
the open review approach can provide the same behavior as the
editorial process, if the user desires that.
You can easily make the reviewing process of the editors transparent
(which I think it should be anyway) and ask them to write their
reviews into the system. So the basic information contained in the
open system would be at least as valuable as the editorial reviews
alone. If the user now desires to also trust other reviewers, she can
profit from additional feedback. I have a feeling that some people
think of the solutions as "one XOR the other". I see most value in an
approach where the experts review the ontologies alongside regular
users. Depending on each user's use case the ranking can then be
configured to put more emphasis on the experts or simply reviews
deemed helpful. (03)
Best,
Holger (04)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
Community Portal: http://ontolog.cim3.net/ (05)
|