english & general Franchu on 26 Dec 2008 10:17 am
On peer reviewed papers
The other day I found on Slashdot an article that described how a computer generated paper had been accepted to a major conference.
After having been a reviewer myself for a couple of conferences, I can say that the paper reviewing process when done properly it can be very effective, but there are some fatal flaws that even with my short experience I can point out:
- Only full final papers should be reviewed. In one of the conferences that was the case, and I can say that I got a quite good feeling of which were worth to be accepted and which ones were just a rehashing of known results without any significant or outstanding contribution to the conference. Nevertheless, in the other, extended abstracts up to two pages were accepted and I got to review one of these. The experience was quite disappointing as you got to read two pages full of promises and written in future tense, vague details on how things are to be obtained and promising awesome results. While it might be true that the final version of the paper would have come to live by the expectations created by the extended abstract, the paper cannot be accepted and be called as peer reviewed when only a set of promises has been shown.
- Interactive communication should be possible with the authors. In the papers I had to review, I had the opportunity to rate from 1 to 5 several aspects of the paper and add personal comments to each of them. Nevertheless, I don’t know if the paper once accepted will include any of these suggestions or changes. And the rating can vary wildly depending on whether the changes get implemented or not! I know that I can always contact them by mail, but still there is no way for the reviewer to ensure that this communication that takes place during the reviewing process finally makes it too the final version of the papers.
With all that being said, I can understand that it gets hard to maintain quality standards when no matter what the topic of the conference is, authors submit papers as they need to publish to keep their jobs, or maintain a status in the community, and conferences need attendants to keep funding coming in order to cover their costs. Nevertheless, if we do not handle the peer reviewing process with care, its value will end up degraded, and any paper, no matter who has reviewed it and which body or organisation sanctioned it will have the same value: null.
Related posts:
- Work ethics in troubled times If you have been following the news, you have probably...
Related posts brought to you by Yet Another Related Posts Plugin.
