Written on Jun 27, 2016.

OpenAIRE workshop on OPR at Elpub

Göttingen, 7 June 2016

3 elpubform 200 Few would deny that peer review, as currently practiced, has its drawbacks. It is slow, unaccountable, wasteful of resources, and lacking in incentives yet it is an essential part of the scientific process. A variety of initiatives have set up experiments with different forms of Open Peer Review (OPR) making the process faster and less opaque. But what does it entail and how can it provide better scientific publications? To explore the possibilities of OPR a workshop on “Open Peer Review: Models, Benefits and Limitations”, was held by OpenAIRE in conjunction with The International Conference on Electronic Publishing (Elpub) in Göttingen, June 2016.

Slides and recordings to come soon...

OpenAIRE its aim has become wider than facilitating open access policies. It is positioning itself as an important partner in the wider discussion around facilitating open science, this also includes the peer reviewing process. OpenAIRE encourages experimentation and wants to actively support a shift to more openness hence the call for tenders for OPR prototypes in 2015. The OPR workshop is the next step in exploring the possibilities of OPR.
In an introduction to the OPR workshop Tony Ross-Hellauer sketched the wider problems with traditional peer review: it is time consuming, lacks accountability, does not rule out bias, there are little incentives and a lot of effort gets wasted. A thorough analysis and consideration of alternatives is necessary to safeguard the reliability of scientific publications. Open peer review can be seen as an umbrella term for a variety of ways in which the traditionally closed processes of peer review can potentially be reshaped by networked digital technologies. Open peer review tries to achieve transparency in the process on at least one of the following aspects: open identity of the reviewer and/or the author, open participation: the selection process is open and open access: public processes and reviews.

Multi-stage open peer review

Ulrich Poeschl (ACP) presented us with a practical example of what an OPR model can look like. He explained how multi-stage open peer review can integrate the strengths of traditional peer review with the virtues of transparency and self-regulation.
The interactive open access journal APC is practicing a multi-stage process of publication and peer review combined with interactive public discussion. ACP examines the question how to make peer review both quick and of high standards.

There answer is the rapid publication of a discussion paper, pre-selected by editors but fully citable and permanently archived.
This paper is available for interactive discussion and public commenting by the referees, authors, and other members of the scientific community. Public discussion not only provides direct feedback, but due to the flexibility and compatibility with traditional structures of scientific publishing and peer review, the multi-stage open peer review concept enables efficient evolution in scientific communication and quality assurance.
After final revisions are completed, the final article is published with a link to the discussion paper. This method of peer review effectively resolves the dilemma between rapid scientific exchange and thorough quality assurance.
ACP demonstrates that interactive open peer review can be realized at a moderate cost and that transparency enhances self-regulation.

Grey Literature as Open Review

Joshua Nicholson (The Winnower) began with pointing out that there has traditionally been a separation between public venues and scientific ones where the public is excluded from the discussion. This is changing with the help of social media. Discussions outside of traditional publishing provides a sort of grey literature peer review. This does not only happen on blogs and specific platforms aimed at scientists but discussion on reddit, facebook and twitter provides valuable feedback for researchers. All this formats equip in a way a platform for post-publication reviewing. But the challenge is to find a way to preserve and validate these comments.

The Winnower tries to include the social, participatory and open elements of social media with the traditional publishing type tools like DOI and archiving. This way the Winnower makes grey literature citable, valuable and accessible for the future. Joshua Nicholson concluded by making a plea to use the infrastructures that are already available. Providing repositories with a more attractive design and some of the social media features for commenting could make them into platforms for scientific discussion.

Panel Discussion: The Challenges of Open Peer Review

A discussion panel led by Pierre Mounier (Open Edition) on the challenges of Open Peer Review was held with input from panellists Stephanie Harriman (BMC), Alexander Grossmann (Science Open) and Pandelis Parakakis (Open Scholar).
The panel brought up the need for a standardized vocabulary when talking about OPR. A common terminology would make discussing and evaluation of the spectrum of OPR methods a lot easier.
On the question what infrastructure is needed to facilitate OPR the emphasis was placed on using and improving the infrastructure that already exist. Repositories for example could provide services beyond being an archiving for articles. Converting open access repositories into functional evaluation platforms, could potentially make them the spill of the scientific community. The key is to implement the tools and engage researchers. A gradually change or more incentives might be necessary to successfully implement OPR-models.

Results of the breakout sessions

Varieties of OPR

The discussion showed the need for common standards. For the sake of clear communication and to assess OPR models, a controlled terminology needs to be developed. This will also help to make it clear what level of openness is necessary and provide an overview of the models in place. A standardization of the current models can provide further insights into how effective an open peer review model is. Finally it seems noteworthy that OPR should not be looked at isolated, as it is only one part of the open science set up.

Engaging stakeholders

No one solution fits all and disciplinary differences in terms of methods and openness must be taken into account if OPR aims to become a successful alternative. The wide variety of modes and experimental models presented in this workshop show potential to meet different challenges. Despite the differences, OPR is possible for every discipline; it may just need different implementation.

Ethical challenges in OPR

Because OPR emerged as an alternative to tackle shortcomings of the traditional reviewing process, it is essential to look at the possible ethical issues that might present itself in OPR. There are several issues brought up focusing especially on young researchers and reputation. Although the general consensus was that more transparency is the best way to answer to integrity issues, there was a strong recommendation to implement a clear code of conduct so both authors and reviewers know the scope and consequences of the peer review model implemented.

To sum it up:

OPR processes need to be made transparent and comparable.
A common terminology is needed as a basis of comparing (effectiveness of) OPR models.
Different modes of mediation need to be tested.
Ways to acknowledge and reward reviewers appropriately need to be found.
Development of mandates/policies needs to be supported.
Differences and particularities of disciplines should be taken serious.
Disciplinary differences in open science should be taken into account and OPR models should be adapted to these specific differences.
A common code of conduct needs to be created, considering the special needs of young researchers.
A legal agreements to decide on condition for author, reviewer and publication should be in place.