|Home News Conferences Commodities Publications Business Directory Resources Help|
:: Elsevier Responds to the Peer Review Debate
There has been much debate recently on the MEI Blog on the merits and limitations of the peer-review process (Peer-review. Is it outdated? and Are journal papers containing advanced mathematics treated fairly?)
In response to these posting, Dean Eastbury, Executive Publisher of Minerals Engineering, International Journal of Mineral Processing, and Hydrometallurgy has submitted this comment on peer-review.
Peer review is fundamental to academia and research, providing informed and objective assessments to maintain the quality of the scientific record and to ensure that science develops independently of commercial, ideological and political interests. Peer review panels are deployed to make decisions about career advancement; the award of research grants, funds and prizes; the appointment of members to professional societies; and the acceptance, improvement of, or rejection of articles for publication in peer-reviewed journals. Articles published in peer-reviewed journals mark developments in science over time. Each peer reviewed article is submitted, assessed, disseminated, and preserved and so becomes the definitive ‘version of record’. Peer reviewers assess, amongst other matters, the originality of the research, the validity of the results, the soundness of methods described, whether the interpretation and conclusions are supported by the facts presented, and any major omissions of prior work that should be acknowledged.
Since the founding of the first peer reviewed journal by the Royal Society in the mid-seventeenth century, publishers have evolved to become stewards of the peer review process on behalf of research communities. Publishers manage publication processes such as peer review on behalf of academic communities, taking on the financial risks of founding and operating journals. Publishers maintain international networks of millions of highly specialised reviewers, and these networks extend beyond those of individual academics, institutions or societies. Publishers identify and appoint editors and editorial board members who in turn appoint expert reviewers that are qualified to provide objective, informed assessments of whether a specific submission is appropriate for publication, and whether it fits with the editorial scope and mission of the journal to which it has been submitted. There are various peer review models. Typically a journal editor will solicit anonymous peer reviews from two to five experts that s/he appoints in the field, or fields, to which the paper would add knowledge. In some cases the peer review process is “double-blind”, meaning that the identities of both author and reviewer are hidden to further limit any possibility of bias. However, many employ a single-blind approach where the identity of the reviewer is not revealed to the author, unless the reviewer agrees otherwise. Academics contribute time to provide reviews. The Publishing Research Consortium estimated that researchers spend on average 40 hours per year performing reviews, and an average researcher reviews eight papers each year with an average review time of five hours. Reviewers contribute this time because it is regarded as part of being a scientist - 91% of respondents indicated they review to play their part as a member of the academic community. At the same time, researchers also benefit from having their own work peer reviewed, as the publication of their research in peer reviewed publications is valued in decisions for promotion, tenure, association memberships and grants. Researchers also gain prestige if they review for prestigious journals, and researchers often indicate on their Curriculum Vitae those journals for which they have reviewed. Publishers remain largely independent of the decision to publish or reject individual articles, as these decisions are made on a case-by-case basis by the reviewers and the editorial team that are appointed by the publisher. Around 3 million manuscripts are submitted to journal publishers for peer review each year. Around 50% of manuscripts are rejected, either because they are deemed not to be scientifically sound, or because they do not fit the editorial scope and mission of the journal. The overall level of 50% is not an artificial or arbitrary construct but one that has evolved organically as a result of peer review globally. The resultant filter is one that is neither too high to bar publication of valid research nor too low to lead to too much questionable research getting publicity. Recent research shows that academics all over the world play a role in the peer review process. In the UK there is a balance between the effort spent reviewing and publishing articles. However this is not always true: researchers in the US do more reviews in comparison to their output, whereas the opposite is true in China. Elsevier monitors these patterns and actively works in countries showing high article growth to educate researchers on the importance of effective peer review. In addition, Elsevier works with agencies that assist researchers to ensure their articles are linguistically accurate before submission. This helps to reduce burden on reviewers in navigating poor language, enabling them to focus on the research content of the article. Since the late 1990s STM publishers have invested over £2 billion in technology, including systems to support peer review processes and to increase their efficiency. Investments include submission systems that enable authors to upload their manuscripts online, and track the progress of their manuscripts. So while the functions that publishers have performed have remained stable for over 350 years, the ways in which we perform these functions have been dramatically modernised. These systems facilitate the review of around 3 million submissions by 125,000 editors, 350,000 editorial board members, and hundreds of thousands of peer reviewers. 3.8 million peer review reports and 30 million author/publisher communications are generated each year. Around 1.5 million peer reviewed articles are then published and disseminated to 14 million people globally, resulting in over 2 billion article downloads and over 40 million article citations per year. Given the massive scale of publishing - which continues to grow by 3-4% annually driven by equivalent growth in R&D funding - STM publishers have established best practices and standards to protect the scientific record, nurture public trust in science, and to build the reputations of journals. For example, COPE (Committee on Publication Ethics) provides case studies to assist editors in resolving cases of ethical infringements (e.g. plagiarism, fraud, etc), and an online forum to share best practice in protecting the integrity of the scientific record. Publishers also train and support editors and reviewers to have the confidence, integrity and skills to adhere to these high standards. Elsevier provides support in handling ethical issues to editors directly and through an online Publishing Ethics Resource Kit. Publishers have also invested in systems such as CrossCheck to help detect plagiarism. Publishers have robust procedures to take action and to correct the scientific record when errors or fraud are detected. Like other societal systems scientific research and its communication is not immune to abuse including the conscious misrepresentation or misinterpretation of facts. However, such cases are the exception, not the rule. Elsevier publishes over 260,000 articles per year, of which we typically retract 70 articles per year due to information that surfaces post-publication. A further 200 are detected post acceptance but before final publication. The peer review system has formal mechanisms to correct and record abuse, and there are serious consequences for those responsible to discourage such behaviour.
Overall, academics value peer review extremely highly. For example, 90% of researchers think that peer review improves the quality of published research and 84% of researchers indicate that without peer review there would be no control in scientific communication. As Richard Horton, Editor of The Lancet, has commented, “Science journals create the norms and rules that determine the ethics and integrity of science in society, and as such are crucial in building public trust in science. Without journals, there would be a cacophony of claims and voices with no means of judging quality or authenticity. Journals shape an ethics of knowledge, which is critical to the effective use of that knowledge in public affairs”. In the UK, the significance of peer review is reflected in the Higher Education Funding Council for England application criteria for sub chairs for the 2014 Research Excellence Framework. The second criterion for appointment as a review panel sub chair is “experience and understanding of peer review and research quality standards
The peer review process is dynamic, and continues to evolve and improve further. Despite the embedded role of peer review and the high levels of satisfaction with it by scientists, we do not claim that peer review is perfect. We strive for continual improvement in three key areas: speed, time commitment for reviewers, and impartiality. While peer review has existed for hundreds of years, it is a dynamic system that continues to evolve to further improve effectiveness, efficiency, and transparency. This is also a by-product of the intensely competitive nature of publishing: thousands of journals compete to publish the articles of millions of authors. Examples of peer review innovations currently in development at Elsevier include: (a) Peer Choice enables reviewers to use advanced analytics software to select articles that match his/her academic competency and current interest. Early results suggest this model can decrease the time to publication decision by 9 days. (b) Scientific Screening professional screening helps editors manage the large number of out of scope and substandard papers that would otherwise require peer review. (c) Review Sharing if a paper is rejected from one journal and is considered to be more appropriate for publication in another journal, the article and reviews can be automatically forwarded to the editor of the other journal if the author agrees. We are experimenting with such a system within Elsevier. We are also part of the NeuroScience Peer Review Consortium which cascades submissions and reviews between journals published by different publishers. 129 papers were successfully cascaded through the consortium during 2010. (d) ReviewerFinder a new tool to help editors expand their reviewer network to improve quality and also to decrease the workload of long serving reviewers. (e) Reviewer Mentor Programme experienced editors employed at two universities mentor postdoctoral researchers who have authored papers but not yet served as peer reviewers. Each mentor runs training workshops for the postdocs and then the postdocs review real articles under supervision. Each postdoc is marked, and upon successful completion receives a certificate. We are exploring ways to provide formal certification and a reviewer kite mark to scale up this successful pilot. (f) Author Feedback pilots to improve transparency for authors, so that they understand where their article is in the review process and understand how and when their article will be published or the reason for rejection. (g) Open Peer Commentary published review articles are accompanied by five one-page comments from other scientists along with the author’s statement/rebuttal of these comments. While successful in attracting attention to a journal, it is very time intensive. How scalable this is remains to be seen. Some have suggested that the process of review by experts could be replaced and potentially bettered by social networking approaches, leveraging the “wisdom of crowds”. Publishers have experimented with open peer review models. So far the outcomes of none suggest that review by selected experts can be replaced to sustain the production and dissemination of high-quality science over the long term: (a) Atmospheric Chemistry & Physics journal operates a two-stage open review process. Following initial review by an editor to assess alignment with the title’s coverage the manuscript is published online (usually 2-8 weeks after submission). Comments and discussion by members of the public and select reviewers then take place for an 8-week period. The author responds to comments within 4 weeks, and then prepares a final revised article. The editor then decides whether to accept the paper. The original paper, comments, and final paper are all permanently archived and remain accessible. Other than comments from invited reviewers, spontaneous comments from members of the scientific community have been relatively low. (b) A European-funded project, Liquid Publications, envisages an online platform on which scientists can post research outputs including papers, datasets, slides, and other materials. The platform enables other scientists to search, read, comment on, link to and from, and collate materials together into ‘personalized online journals’. The reputation of individuals active on the platform is used to assess quality, assign credit, and measure impact. This project is at an early stage of development and outcomes are unknown. (c) PLoS ONE provides post-publication tools that allow readers to rate the quality and impact of a paper, or to leave comments. All papers are reviewed by invited experts. However, the take up of post-publication commentary or ratings has been very low. (d) Nature tried an open review model in 2006. Willing authors had their submissions posted online for reader comment while in parallel a traditional blind peer review process was conducted. The trial was cancelled as public comments were rare, and editors found that these were less helpful than the comments of the conventional peer reviewers. Elsevier will continue to innovate in these areas. When developments have potential to improve the peer-review process without compromising current high levels of quality, accuracy, objectivity and efficiency then we will actively invest in those innovations. Occasional suggestions are made to replace peer review entirely with post-publication metrics such as citation and/or usage data or to substitute publisher-managed peer review with review by internal university panels. None of these alternatives attract major support from the academic community and most academics continue to see publisher-managed peer review as the best option . Whatever approach is taken it is important that the review system caters for differences between disciplines by continuing to accommodate anonymous and/or identified reviewers, register new science; lead to rapid high-quality publications; be used for static or dynamic publications; facilitate search and retrieval of underpinning data, operate effectively with any business model, and create a permanent, citable, cross-referenced record of science.
© 1998-2017, Minerals Engineering International