Skip to content

Thoughts on Proposal Peer Reviewing System

August 2, 2014

Peer review

On Feb 7, an article entitled “Peering Into Peer Review” by Jeffrey Mervis was published on Science. (Vol. 343 no. 6171 pp. 596-598) This article discussed the current status of proposal peer reviewing system. According to the statistics from the tracking study for the funded proposals, the initially highly rated projects have not obtained significantly higher achievements than those didn’t get a good score at the beginning of the funding.

I have never reviewed for a research proposal. But my reviewing experience for a broad range of journal articles tells me that there is no easy way to judge a work’s importance using a fit-for-all criterion. Although this phenomenon can be explained from different points of view, there are four issues that need to be considered before making conclusions about the fairness of the funding peer-review system.

First, scientific research has its own cycle, which may vary depending on the topic of the study. Even for the same topic, one cannot predict when a promising result can be obtained. Science is largely unpredictable. Good ideas may be proven wrong at times.

Second, it is very likely that a proposed research got a low score during the peer-review process merely because it is an “outdated” or “old-fashioned” topic, which makes the reviewers thought that it has a lower impact than those popular ones. However, those not-so-hot research areas have no reason to publish less influential papers in terms of scientific contribution.

Third, the proposals that successfully stood out with high scores could be very impactful themselves. However, the measuring standards – the number, time, and impact of the publications, are not necessary to reflect the true value of that specific research project.

Last but not least, writing a good proposal is only the very first step of carrying out good science. A highly ranked proposal needs fulfillment – a continuous efforts made by researchers, post-docs, and students. Besides, with the progress of the project, the PIs need to timely adjust the direction according to the updated situation. All of the above factors are not able to be evaluated during the peer-reviewing process.

Anyhow, there’s nothing to blame the peer-reviewing system. Perfect systems do not exist. The current process may be the best we can do so far to evaluate proposals. Scientists themselves are the ones who know better than anyone else whether a project is important enough to be granted or not. So, let us give science a little time and patience to prove itself by the outcomes.

Advertisements
Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: