ontolog-forum
[Top] [All Lists]

Re: [ontolog-forum] Disagreements among reviewers

To: "[ontolog-forum] " <ontolog-forum@xxxxxxxxxxxxxxxx>
From: kenb <kenb@xxxxxxxxxxx>
Date: Sun, 19 Apr 2015 23:51:34 -0400 (EDT)
Message-id: <alpine.DEB.2.02.1504192305420.4061@xxxxxxxxxx>
Indeed, I know about such papers.  However, I wonder whether they would 
affect the statistics very much.  Such papers (from both the great ones 
and the crackpots) may be too rare to have a significant effect.  Has this 
ever been studied?    (01)

-- Ken    (02)

On Sun, 19 Apr 2015, John F Sowa wrote:    (03)

> On 4/19/2015 4:26 PM, kenb wrote:
>> Given that reviewers are not compensated and that there is no
>> assessment of the quality of the reviews, I am not at all surprised
>> by the lack of agreement.  Given the lack of agreement and assessment
>> of quality, it is also not surprising that "best paper" awards are
>> largely meaningless.
>
> The issues are more complex.  People with novel or unorthodox ideas
> in any field -- science, engineering, art, etc -- are often hard to
> distinguish from crackpots and con artists.
>
> There are countless stories of publishers that rejected great books,
> movie producers that rejected great stories, and business executives
> that rejected great inventions (xerography, for example).
>
> In an interdisciplinary field, it's hard for any reviewer to be
> able to evaluate novelties in every branch.  Sometimes an author
> who has a great idea on the boundary between fields A and B will
> be rejected or given a mediocre evaluation by reviewers from both.
>
> The methods for evaluating the quality of journals and conferences
> put pressure on the organizers to attract lots of submissions so
> that they can have a high rejection rate.  As a result, the bar
> for acceptance means a high average score from all reviewers.
>
> Some organizations recognize those issues.  One solution is
> to *accept* any paper that receives both very strong acceptance
> scores from one or more reviewers and very strong rejection scores
> from others.  That is usually a sign of a controversial topic.
>
> John
>
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/
> Community Wiki: http://ontolog.cim3.net/wiki/
> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>
>    (04)

_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (05)

<Prev in Thread] Current Thread [Next in Thread>