[Top] [All Lists]

Re: [ontology-summit] [Quality] ReFocusing

To: Ontology Summit 2008 <ontology-summit@xxxxxxxxxxxxxxxx>
From: Holger Lewen <hle@xxxxxxxxxxxxxxxxxxxxx>
Date: Tue, 1 Apr 2008 14:50:49 +0200
Message-id: <F826082F-A88E-4A7E-BE62-B361BF5C38E3@xxxxxxxxxxxxxxxxxxxxx>

> == Summary: Stewardship/ Quality Control==
> The community agrees that it is not sufficient for the OOR just to  
> store
> ontologies, but that it needs to provide the possibility to evaluate  
> the
> ontologies within it. There is no agreement on how to evaluate
> ontologies; the main strategies suggested are: (i) A market driven
> approach where ontologies are reviewed by users and ranked like  
> items on
> Amazon.com; and (ii) an editorial process where ontologies are  
> reviewed
> by experts in a similar way as papers which are submitted to  
> scientific
> journals. This conflict is not surprising, since the different members
> of this community are using ontologies for different purposes and thus
> have different perspectives on what ontologies are and how they are
> supposed to be evaluated.    (01)

Fabian,    (02)

thanks for summarizing the discussion. I do not want to reopen the  
whole discussion on quality control, I just want to stress again that  
the open review approach can provide the same behavior as the  
editorial process, if the user desires that.
You can easily make the reviewing process of the editors transparent  
(which I think it should be anyway) and ask them to write their  
reviews into the system. So the basic information contained in the  
open system would be at least as valuable as the editorial reviews  
alone. If the user now desires to also trust other reviewers, she can  
profit from additional feedback. I have a feeling that some people  
think of the solutions as "one XOR the other". I see most value in an  
approach where the experts review the ontologies alongside regular  
users. Depending on each user's use case the ranking can then be  
configured to put more emphasis on the experts or simply reviews  
deemed helpful.    (03)

Holger    (04)

Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/ 
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008 
Community Portal: http://ontolog.cim3.net/    (05)
<Prev in Thread] Current Thread [Next in Thread>