Lessons learned from open peer review: a publisher’s perspective

R
Research Publishing
By: Rachel Burley, Sat Dec 23 2017
Rachel Burley

Author: Rachel Burley

Vice President, Publishing

On August 29th ASAPbio published an open letter encouraging more openness in peer review.  As a publisher with a long track-record of open peer review, BMC was delighted to be one of the signatories. 

nature_04 © Photo by Dominik Scythe Unsplash

In 2001, BioMed Central was the first publisher to openly post named peer reviewer reports alongside published articles as part of a ‘pre-publication history’ for all medical journals in the BMC series.

With growing interest in openness and transparency, and as more journals consider open peer review, I wanted to share some of our experiences over the last 17 years.

The BMC Group publishes over 500 journals and of those, 70 operate fully open peer review. Forty-four of the BMC series medical journals use only open peer review, with no opt out for authors or reviewers. These journals publish over 8,000 papers a year, demonstrating that at least in the medical disciplines, there is clearly a community willingness to undertake open peer review. 

Forty-four of the BMC series medical journals use only open peer review, with no opt out for authors or reviewers. These journals publish over 8,000 papers a year, demonstrating that at least in the medical disciplines, there is clearly a community willingness to undertake open peer review. 

So what have we learned? 

There have been relatively few large-scale studies looking at the impact of open peer review, but it seems to me that the key issues for journals and publishers considering introducing open peer review fall into the following categories:

  1. Review uptake
  2. Quality of the review
  3. The editorial decision
  4. Value to readers

ASAPbio’s FAQs page tells part of the story, where publishers already operating some form of open peer review have added their observations or studies.

Here, we share the lessons learned from open peer review at BMC.

  1. Reviewer uptake

    For the BMC series, there is some evidence to suggest that reviewer uptake is lower due to the open peer review policy of the journals. A very small number of reviewers actually cite open peer review as the reason for declining requests, but editors for some journals tell us that it can take longer to recruit reviewers for this reason. A study conducted by the BMJ found a similar trend, where asking reviewers to consent to being identified increased the likelihood of reviewers declining to review.

    It is likely that the BMC open peer review (OPR) policy, which includes reviewers’ identities, is an added factor in lower reviewer uptake, whereas anonymous OPR may have a lower proportion of declined reviewer invitations. We haven’t yet done any research to conclusively prove this.  However, we do have data comparing fully OPR with double and single blind peer review.

    An analysis of data from 500 BMC and SpringerOpen journals between June 2001 to July 2015 found that on average, about half of invited reviewers agreed to review.  Grouped by peer review model, it was 60% for double-blind peer review, 53% for single blind and 42% for open peer review. In other words, more reviewers needed to be invited for OPR journals than for either of the other two models (6 invitations for OPR journals compared to 4 for double blind). 

    There are doubtlessly a number of reasons why more reviewers decline invitations to OPR journals, but the top two are likely to be:

    1.1.      It takes more time to conduct an open peer review: 182 minutes compared to 157 minutes according to this randomized controlled trial.  It is well known that the burden on the peer review community is increasing as research output increases and so anything that adds time to the task is likely to be a disincentive, even if reviewers recognize the value of openness.

    1.2.      Fear of reprisals: for example, reviewers may be reluctant to express their true opinions on a poor quality paper coming from the laboratory of a senior researcher.

    Some reviewers strongly support open peer review and tell us that it is in the interests of the research community to strive for openness and transparency. I believe this is the ultimate goal for all journals, but that it will be a journey rather than an event as the practice becomes more common and researchers are more comfortable with openly critiquing their peers.  By promoting the benefits of OPR we will encourage reviewers to support the journals that use it and create an environment in which more journals are willing to ‘make the leap’.

    Some reviewers strongly support open peer review and tell us that it is in the interests of the research community to strive for openness and transparency. I believe this is the ultimate goal for all journals, but that it will be a journey rather than an event as the practice becomes more common and researchers are more comfortable with openly critiquing their peers. 

    The benefits for open reviewers include recognition for the time and effort they have spent assessing the manuscript.  Credit can be given through systems such as Publons, a free service for reviewers that tracks, verifies and showcases peer review activity across all journals.   Reviewers can use their Publons profile as evidence of their peer review contributions and expertise.

  2. How open peer review affects the quality of reports

    There is a school of thought that publishing reviewer reports will encourage better-quality, more constructive comments. But is that actually true?

    A study in BMJ Open in 2015 found that the quality of peer review reports was slightly higher in BMC Infectious Diseases (OPR) compared with BMC Microbiology (single-blind). However, no effect was found in the Journal of Inflammation when it operated open vs single-blind review. Anecdotally, reviewers have told us that they take longer to write a review that they know it will be published and where they will be identified.  This might not always lead to a higher quality report and may be a factor of something else, such as reviewers who do not have English as their first language taking more time to write the reports knowing they will not be copy-edited.

  3. The editorial decision

    The study in BMJ Open in 2015 showed that reviewer recommendations were similar in BMC Infectious Diseases and BMC Microbiology, suggesting no difference between open and single blind peer review.  However, in the Journal of Inflammation, it was found that reviewers were more likely to recommend acceptance under open peer review compared with single blind.

  4. Reader perception

    For the BMC journals, the reviewer reports receive an average 1.5% of the page views for the journals (including all content types and not just research articles). 

    Since January of last year BMC journals had an average of 25,000 page views per month to open peer review reports.  The numbers are growing though - in July/August this year, average page views were more than 50,000 per month. 

Our long experience of open review has shown us that a pragmatic balance has to be struck between providing an efficient and timely service to authors and the ideal of full transparency, where the mandatory requirement of publishing reviewers’ details may have unintended consequences in discouraging and deterring some researchers from becoming reviewers.

#PeerReviewWeek18 gives us the opportunity to share our views and experiences of peer review, and to recognise the vital role of peer reviewers in building the scientific record. We want to thank the many peer reviewers who have dedicated their time to our journals, and welcome new ones into the reviewer community.

Rachel Burley

Author: Rachel Burley

Vice President, Publishing

Rachel Burley has been the Publishing Director of BioMed Central and SpringerOpen since 1st February 2016. Her responsibilities include all of BioMed Central and SpringerOpen’s editorial and commercial activity.

Burley was previously with John Wiley & Sons, where she was Vice President and Director of Open Access and Business Development. There, she led the strategic planning and development of Wiley’s open access initiatives and was instrumental in identifying and implementing strategic partnerships. Prior to that, Burley was Vice President and Publisher of Life Sciences at Wiley, and spent seven years at Nature Publishing Group.

Related Tags:

Previous posts

Transformative deals substantially aid the transition to OA. Let’s give them the time they need to do so.

Last month, Springer Nature agreed a new transformative ‘read and publish’ deal with the Norwegian library consortium, Unit.  This, coupled with renewals of existing deals in the UK (Jisc), Sweden (Bibsam) and Qatar, takes the number of such deals we have into double figures, the most of any publisher.  These deals enable researchers in these ten countries to publish open access in over 1900 Springer journals as well as read the research of others across this portfolio.

O
Openaccess
02 Aug 2019
The future of open access books

Open access book publishing has been growing in recent years. The Directory of Open Access Books (DOAB) lists 2,099 open access books published in 2018, an increase of 38% from 2017. Funders are also starting to engage with open access for books: a small number of European funders now mandate open access for books and provide financial support for open access book publication. COAlition S has also indicated that they support transitioning monographs to open access, and will provide guidance by the end of 2021.

O
Openaccess
18 Jun 2019
Congratulations to Karen Keskulla Uhlenbeck

It’s a great pleasure for us at Springer Nature to congratulate Karen Keskulla Uhlenbeck for the award of this year’s Abel Prize.

Springer Nature Group
Businesses must play a role in sustainable development

Today Springer Nature launches its Responsible Business report for 2018.

Springer Nature Group
Setting targets at Springer Nature to help create a better balance

The theme of this year’s IWD couldn’t be more relevant to us. Last month, Springer Nature’s management board announced to our 13,000 colleagues that we will set a target to improve gender representation in our leadership group.

Springer Nature Group
Without you, we are nothing: why community engagement is at the core of everything we do

Sometimes there is great value in stating the obvious; the scholarly publishing industry would not exist without a broad international community of researchers, students, librarians, funders, and commercial partners.

R
Research Publishing
Need to focus on demand as well as supply: making Plan S work for all

Steven Inchcoombe discusses Springer Nature's submission to the cOAlition of participants to make Plan S work for all.

R
Research Publishing
A postcard from Davos: what role for research at this gathering of global opinion leaders?

My first impressions of Davos? Dazzling, overwhelming, unexpected, thought-provoking, incredibly busy and much, much more…

R
Research Publishing
How Can We Accelerate Data Sharing?

What steps can publishers, funders and institutions take to make data sharing worth a researcher’s time and energy, and accelerate progress? 

R
Research Publishing
Connecting curious minds to advance learning and discovery

Science

Springer Nature will stay curious, connect ideas, question how things are done, explore new ways to deliver for our communities and then make the changes that matter.

R
Research Publishing
20 Feb 2019
The quest for more value – challenges of the scientific ecosystem in the absence of coordination: A Long Read

Research News

Daniel Ropers shares some of his early observations about the academic publishing industry.

R
Research Publishing
01 Nov 2018
Publication misconduct: changing the conversation

Suzanne Farley discusses how to increase prevention and detection of research and publication misconduct. 

R
Research Publishing
Lessons learned from open peer review: a publisher’s perspective

Research News

Rachel Burley discusses BMC's history in openness and transparency and what has been learnt.

R
Research Publishing
23 Dec 2017
Accelerating open access: what is missing from Plan S

Research News

The transition to open access needs to be sustainable for the whole research community.

R
Research Publishing
23 Dec 2017
Finding the science in social media

Research News

At Springer Nature we believe that social media is a vital tool for everyone involved in the academic research and publishing process. 

R
Research Publishing
23 Dec 2017
Hybrid journals can advance the move towards full open access

Blogs

Hybrid journals play an important role in the transition to open access.

R
Research Publishing
Science benefits from diversity – and as publishers we have a role to play

As a progressive publisher, we have a responsibility to support diversity and inclusion.

R
Research Publishing
What next for Impact Factors?

Why we need to work together to create metrics that better meet the needs of the research community.

R
Research Publishing
Can research change the world?

Springer Nature believes that great multidisciplinary research needs to be informing global discussions.  

R
Research Publishing
We need to do more to celebrate and support women in engineering

Women’s achievements in STEM fields need to be harnessed to raise awareness, create role models and foster a more inclusive culture. 

R
Research Publishing
Future of the book all down to definition

Advancing discovery

The speed of recent change in the publishing industry in the past 10 years poses the question, what will book publishing look like in the future?

R
Research Publishing
Springer Nature is standing firm behind the experts

Advancing discovery

Like the experts we publish we too need to be better at communicating what we do. 

R
Research Publishing
Welcome to Springboard

Advancing discovery

Our blog brings together views and commentary on the latest innovations, ideas and issues facing the world of academic publishing and the wider research community as a whole.

R
Research Publishing