ontology-summit
[Top] [All Lists]

Re: [ontology-summit] [Quality] Gatekeeping: Democratic Ranking vs. Peer

To: Ontology Summit 2008 <ontology-summit@xxxxxxxxxxxxxxxx>
From: Holger Lewen <hle@xxxxxxxxxxxxxxxxxxxxx>
Date: Fri, 21 Mar 2008 17:11:38 +0100
Message-id: <338C7005-267A-4A24-A16B-94360C740158@xxxxxxxxxxxxxxxxxxxxx>
> BS
>>>>> Both reviewers and developers will be further
>>>>> motivated to participate in this process because
>>>>> they can thereby directly influence the standard
>>>>> set of ontology resources which will be available
>>>>> to them, thereby also motivating the creation of:
>>>>> related ontology-based software, useful bodies of
>>>>> data annotated in terms of these resources, etc.
>>>>>
>
> HL
>>>> Again, in my opinion also possible in an open reviewing system.
>
> BS
> You can't have it both ways -- on the one hand you claim multiple
> benefits from having lots of reviewers, lots of perspectives, people
> can find precisely the ontology which will suit their purposes from
> the wisdom of crowds, etc.
> Now, you say that lone reviewers lost in these crowds will be able
> 'to directly influence the standard set of ontology resources which
> will be available to them'.
>    (01)

HL
The way the ranking algorithms work, that lone reviewer will stand out  
of the crowd if enough people think the reviews are worthy.
So yes, you can indeed have it both ways. That is the beauty of the  
proposed rating and ranking system.    (02)

>
> BS
>>> Good journals allow anyone to submit.
>>> Also the system of journal publishing which we
>>> take as our starting-point provides a rather
>>> elegant way to divide up the potentially
>>> unlimited domain that is available for ontology
>>> development into limited disciplines and subdisciplines.
>>>
>
> HL
>> Yes, they do, but this is because it still results in limited
>> submissions. If everybody would start submitting to journals, it  
>> would
>> only take short time until there would be some kind of restriction
>> (e.g. only people working in science may submit).
>> I agree that your approach is valuable if taken one discipline at a
>> time with dedicated reviewers for that discipline. This is again a
>> question of scale. How do you determine with which subcategory to
>> start. If I have the best possible ontology for a subdomain you are
>> not considering yet, what do I do? It might still be extremely
>> valuable for the community to see my ontology straight away.
>>
> BS
> Register it immediately with the OBO Foundry!
> This will help people to find it.
> You will benefit from expert review.
> If there is already an OBO Foundry ontology available for your topic,
> you can start working on a merger, to reap the benefits listed by
> Bill (post copied below).
> What, after all, can you lose?    (03)

HL
Sorry, was meaning to say "if I had". It was supposed to be a  
hypothetical question. So your idea is to have the foundry as a  
starting place where ontologies would be submitted regardless of  
subdomain and consider ontologies from the foundry for your journal- 
like system?    (04)

I still think that both systems could have synergies and hope that not  
only one survives.    (05)

Thank you for the discussion,
Holger    (06)




_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/ 
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008 
Community Portal: http://ontolog.cim3.net/    (07)
<Prev in Thread] Current Thread [Next in Thread>