Modernizing peer review : discussing Ferreira et al. 2015 Biological Reviews

Posted on

BRV_leftIn the journal Biological Reviews, a group of researchers from the Department of Biology at Trent University have published a paper looking at peer review and the need for a quasi-revolution in terms of how peer review works. Given my academic experience, I was curious to see the validity of their points and how the peer review system has (or has not) changed over the last while.

In presenting the state of affairs, the authors (led by Catarina Ferreira) point out that the system of peer review seems not well-adapted to the modern research publishing world. Over the last 50 years, the number of journals has increased exponentially (as well as evil predatory journals), publication numbers by researchers are now the yardstick used for judging success at the university level and for grant applications, and, worryingly, rates of retraction of published papers have been steadily increasing over the last decade. On top of these factors, given that good reviews require time and effort from researchers and that most researchers wish to spend more time publishing their own work, rather than reviewing (for no compensation) work by other researchers (altrusim vs fitness as the biologists correctly point out), the peer review system is weakening under the strain.

The authors lay out the current state of affairs and then attempt to offer potential solutions (both short-term and long-term) to address issues facing peer review. They point out a number of attempts that have been made, or that currently exist, in order to address the gap between growing publication numbers and the reduced pool of willing reviewers. These include incentives (such as stipends or honoraria) or development of a publishing plan, such where membership permits the right to publish in a given suite of journals with the responsibility of performing reviews for the journals. Another aspect is looking at alternatives to the impact factors of specific journals and their papers in order to take into consideration other contributions and works (including reviews) of researchers who are being evaluated or even adding a weighting for social media of published material (more “likes” equals more impact). Finally the authors point out that peer review is opening up somewhat (through Peerage of Science, for example) where reviews are posted for their own reviews, or where reviews are shared between reviewers in order increase the quality of peer revisions and assessments.

 

The final, and key, section of the paper discusses the longer-term needs of the system. The authors call for putting a greater value on reviews so that reviewers receive credit for their work. This could/should occur at the institutional level as well as the publication level where the number of publications of a researcher/year should ideally equal the number of reviews by that researcher over the same time period. Ferreira et al. also argue for a standardization of peer review across publications. For now, guidelines for reviewers are generally independent across publications and, as such, a clear view of what is required by a reviewer remains elusive. The inclusion of data, for example, should be standard for all submissions. Their final idea is the establishment of a cross-discipline, independent regulatory entity (they name this a Global Peer Review Platform) for defining of the rules of peer review, establishing criteria for review within disciplines and in doing so, streamlining the peer review process. The entity would take control of peer review from the publishers and hand it back (maybe) to the researchers, ensuring a higher, independent standard of review.

My thoughts? First off, for someone who is looking for an overview of peer review in its current state, Ferreira et al. offer a clear and straight-forward presentation of ideas. From the point of view of a (former) academic researcher, I agree with a number of points they make. First off, contributions to peer review MUST be made an explicit contribution to the science community. Currently, peer reviews are given little (or a much reduced) standing in terms of evaluation within many universities and within grant requests. In Canada, NSERC grant requests place a focus on the number of publications with little request for peer review contributions (this can be included in the attached summary form, but reviews are rarely detailed). I guess it is assumed that everyone does reviews, although as Ferreira et al. point out, the number of researchers refusing to conduct reviews has been steadily increasing and the publish-or-perish system within academia favours those who review fewer manuscripts or who provide minimal reviews. In a similar way to the value of teaching within the university milieu, peer review is touted as being important, but in the concrete reality of research, spending valued time reviewing manuscript (like investing in teaching) holds little power. Clearly a cultural shift is required here.

The authors also point out that standardization is required with a more open process allowing reviews of the reviewers. This would be a helpful first step in that the quality of reviews should increase given a more open nature (of course, the value of reviews must be also in the forefront). I absolutely HATE receiving reviews that total a few lines, saying, “Yup, OK. Change this portion of the discussion. Add my paper as a reference. Spelling error line 56.” Thanks for your lack of effort or care. Unfortunately this is a common occurrence that helps no one. I do appreciate long reviews that improve a manuscript and I do understand that the researcher must have taken much time to dissect my ideas. Thanks! (Of course, my personal appreciation gets little traction with that researcher’s funding requests).

I don’t see a cross-disciplinary platform being established soon (à la Ferreira et al.). However, their ideas hold value for a more narrow subset of the sciences – say the discipline-level. Too many aspects of peer review need changing in a near simultaneous manner (culture of placing value on reviews and reviewers, standardized peer review models, listing of researchers with their publication/review ratios). It is quite possible, and maybe ideal, if disciplines such as ecology and evolutionary biology for instance, establish a platform and standardize peer review across the discipline. As such, researchers within the domain are given value (when their peers evaluate their funding requests or research papers) and maybe show other disciplines that is it possible to strengthen peer review. Given that science rests upon peer review as a basic foundation, it is crucial that more care and value be placed upon this element.

Top