On 4/19/2015 4:26 PM, kenb wrote:
> Given that reviewers are not compensated and that there is no
> assessment of the quality of the reviews, I am not at all surprised
> by the lack of agreement. Given the lack of agreement and assessment
> of quality, it is also not surprising that "best paper" awards are
> largely meaningless. (01)
The issues are more complex. People with novel or unorthodox ideas
in any field -- science, engineering, art, etc -- are often hard to
distinguish from crackpots and con artists. (02)
There are countless stories of publishers that rejected great books,
movie producers that rejected great stories, and business executives
that rejected great inventions (xerography, for example). (03)
In an interdisciplinary field, it's hard for any reviewer to be
able to evaluate novelties in every branch. Sometimes an author
who has a great idea on the boundary between fields A and B will
be rejected or given a mediocre evaluation by reviewers from both. (04)
The methods for evaluating the quality of journals and conferences
put pressure on the organizers to attract lots of submissions so
that they can have a high rejection rate. As a result, the bar
for acceptance means a high average score from all reviewers. (05)
Some organizations recognize those issues. One solution is
to *accept* any paper that receives both very strong acceptance
scores from one or more reviewers and very strong rejection scores
from others. That is usually a sign of a controversial topic. (06)
John (07)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J (08)
|