ontolog-forum
[Top] [All Lists]

Re: [ontolog-forum] Disagreements among reviewers

To: ontolog-forum@xxxxxxxxxxxxxxxx
From: John F Sowa <sowa@xxxxxxxxxxx>
Date: Mon, 20 Apr 2015 09:02:50 -0400
Message-id: <5534F8FA.8010000@xxxxxxxxxxx>
Ken and John B,    (01)

KB
> Such papers (from both the great ones and the crackpots) may be
> too rare to have a significant effect.    (02)

Three points:    (03)

  1. We all know many intermediate cases that are much more common.
     But losing just one great idea can be a very significant loss.    (04)

  2. In 1905, there was a clerk in a Swiss patent office who wrote
     four revolutionary articles that challenged the ideas of the
     famous scientists of the day.  Fortunately, Albert Einstein was
     lucky to get a few good reviews -- but there are many others
     who didn't.    (05)

  3. Chester Carlson was an engineer who patented a great idea in 1938.
     But he couldn't get funding for years.  When he finally got some
     support, the result was Xerox.    (06)

JB
> What both of these (scientific and industrial) systems have in
> common is a set of agents working toward a goal of reducing errors.    (07)

Yes, but the methods you cited are the way science and engineering
converge on good methods over a period of years -- sometimes decades.    (08)

As Kuhn and others point out, a well established paradigm may survive
for a very long time -- until the people (or institutions) that use it
die, retire, or go out of business.  One of my favorite quotes:    (09)

Alfred North Whitehead, _Adventures of Ideas_
> Systems, scientific and philosophic, come and go.  Each method
> of limited understanding is at length exhausted.  In its prime
> each system is a triumphant success:  in its decay it is an
> obstructive nuisance.    (010)

Besides the paradigms that outlive their usefulness are the fads
that have a few early successes, are widely adopted, crowd out
the established methods, and fail because they really aren't
as good as they seemed on the surface.    (011)

Computer science in general and AI in particular have gone through
major swings (boom & bust periods) where certain methods become
popular, crowd out the older methods, and collapse -- only to be
replaced by another fad (which is often a revival of an earlier fad).    (012)

Finally, I'd like to comment on the article that started this thread.
It was about Neural Information Processing Systems (NIPS), which have
had 28 annual conferences:  https://nips.cc/Conferences/current    (013)

As the web site says, they bring together "machine learning and
neuroscience".  Those fields have many competing paradigms that
clash more often than they agree.    (014)

See below for an excerpt from the article.  To get some data
about the problem, they sent out a representative sample of the
accepted papers for a second review by different reviewers.  And
about 60% of the previously accepted papers were rejected.    (015)

The term "reviewer roulette" has been around for a long time,
and that study shows that the odds are definitely bad.    (016)

John
______________________________________________________________________    (017)

http://cacm.acm.org/blogs/blog-cacm/181996-the-nips-experiment/fulltext    (018)

The 26% disagreement rate presented at the NIPS conference understates 
the meaning in my opinion, given the 22% acceptance rate. The immediate 
implication is that between half and two-thirds of papers accepted at 
NIPS would have been rejected if reviewed a second time. For analysis 
details and discussion about that, see here...    (019)

_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (020)

<Prev in Thread] Current Thread [Next in Thread>