THURSDAY, 7 JANUARY 2021
On my first day of remote working, I set up a journal alert for research relating to the COVID-19 pandemic. As an infectious disease researcher, I had a surreal feeling that I was a living dot on a scatterplot I would later be charged with analysing, and to prepare, I wanted to keep up with important SARS-CoV-2 findings. The next morning, I woke up to 20 new publications in my inbox. By the end of the week, it was over 300.
The rate at which research on the novel coronavirus is being churned out is unprecedented. The Chief Editor of Nature Medicine tweeted that submissions to the journal had increased by 150-200% since mid-March. According to a report by research technology company Digital Science, over 42,000 articles relating to the COVID-19 pandemic were released in just four months from January to April. To put that in perspective, the entire field of ‘deep learning’ in artificial intelligence, one of the fastest growing research sectors according to Digital Science, contains around 150,000 publications in total. Use of preprint servers — online repositories hosting unreviewed scientific findings and often heralded as the fastest route to disseminate results — has also skyrocketed. The website Rxivist, which collects statistics on the life science pre-print server bioRxiv, recorded a 10-fold increase in the monthly rate of new submissions to the server between January and May 2020, compared to the previous 9 months. It quickly became clear to publishers that access to these results is essential in shaping a pandemic response. The Wellcome Trust stated that by mid-March over 30 leading publishers had removed paywalls on any content related to COVID-19, granting access to those without subscriptions to their journals.
At the time of writing in June, though the initial peak of the publication boom appears to have plateaued, the waterfall of information is far from petering out. In fact, the flood of research relating to the pandemic continues to serve as a fascinating test of strength for the levees employed by scientific publishers against rising tides of unsupported or misleading information.
Journal editors subject any new finding to rigorous review by experts in the field to critically assess both the study’s design and reported conclusions. Depending on a number of factors, this process takes anywhere from weeks to months. Amidst a global pandemic however, dissemination of valid results harbours a new sense of urgency. A recent report in Nature Human Behaviour found while the median time to publication before the pandemic sat at 93 days, for COVID-19-related submissions, this figure plummeted to just six days. A similar ‘rush’ was observed during the first 12 weeks of the 2014 Ebola outbreak in West Africa, with median time to publish reduced to 15 days. Even outside of a public health emergency one can find peer-reviewed research released less than a week following initial submission, but what makes this pandemic different is largely the novelty of the SARS-CoV-2 virus, coupled with its unprecedented global reach and level of societal disruption. As many researchers are busy adapting to lab shutdowns and remote working setups, expert reviewers can be difficult to find. ‘Many experts in the relevant areas are writing their own papers and have less time to review the work of others’, said Anna Akhmanova, a Deputy Editor at eLife, ‘Also, the fact that many SARS-CoV-2 virus-related studies are put together rather quickly and often not very profound makes it difficult to find reviewers’.
A dwindling pool of available reviewers and accelerated turnaround times, however, do not mean that editorial standards have been loosened. A group of publishers including eLife, PLOS, and The Royal Society released an open letter in late April stating their commitment to ensuring rapid but thorough assessment of new coronavirus research. In the statement, they address the ‘expertise drought’ by calling for the names of reviewers willing to commit to fair but rapid review. According to Magdelena Skipper, Editor-in-Chief of Nature, ‘the pandemic is putting immense pressure on everyone — including authors and reviewers — and we’re grateful to each and every one of them for working with such dedication and prioritising this work’. It is the combined efforts of researchers, reviewers, and editorial staff, she says, that keeps the vast majority of poorly supported information at bay.
Skipper also believes the pandemic has helped to highlight the important alliance between peer-reviewed journals and preprints. Both Nature and eLife advocate for authors to submit their findings on preprint servers while the peer review process takes place. ‘We view the relationship between preprints and journals as symbiotic’, says Skipper, ‘preprints surface research in its earliest form, facilitating early access to research for the wider scientific community, whilst journals provide curation and deliver the rigorous peer review process, and scrutiny, alongside other functions’. Detlef Weigel, also a Deputy Editor at eLife, explains how the publisher is also trying to bridge the gap between the two platforms, ‘one of our initiatives, Preprint Review, aims to fast-track reviews for preprints, so that research in preprints is vetted more quickly — independently of formal acceptance for publication in a traditional journal’.
Despite these efforts, the unusually invested interest of the global lay audience in research output from the field occasionally leads to the misinterpretation and spread of poorly supported claims in both preprints and published articles by researchers, media outlets, and even world leaders. In the early days of the pandemic, preliminary results touting the use of antimalarial drug hydroxychloroquine as a treatment for COVID-19 gained international traction, leading to shortages of the drug in the United States and at least one lethal administration. The study has since been retracted, and further research has found the drug potentially ineffective in treating the disease. Additional retractions of research published in the Lancet and the New England Journal of Medicine, this time attempting to track the spread of viral cases, also made international headlines. These incidents, while widely reported, are exceedingly rare. In July 2020, of the tens of thousands of available papers relating to the pandemic, the website Retraction Watch has recorded at least 22 (including preprints) as officially retracted.
The responsibility remains on researchers to continue to independently critically assess any new findings, peer-reviewed or not. ‘There is no way around judging for yourself the merit of work published as a preprint or in a journal’, says Weigel, ‘yes, reviews increase the chance that the work shows what the authors claim, but we all know that reviewers can make mistakes’. Akhmanova adds, ‘it is also the responsibility of the media to provide a clear background about their sources of information (e.g. peer-reviewed papers vs. non-peer-reviewed preprints) when they further disseminate this information to the public’. This is a sentiment shared by Skipper at Nature. She advocates for better dataset availability as a means of helping individual researchers locate shortcomings in a study that might have been missed in the peer-review process, ‘assessing data quality and appropriateness of a dataset to address a given question can also be key. This is why we encourage (and in some cases mandate) data sharing — it facilitates a more thorough peer review and, post-publication, helps others to further verify and build on a given set of findings’.
The pandemic continues to provide unique challenges for researchers, policymakers, publishers, and news outlets to adapt to. Personally, I’ve had to turn off my journal alert for the sake of my overwhelmed email client, but have discovered my own new ways of accessing and critically appraising research that I will continue to employ long after the pandemic is over. As is the case for so many in this pandemic, we are all still learning to keep our heads above the surging stream of information and stay afloat.
Juliana Cudini is fourth year PhD student in Infection Genomics at Gonville and Caius College. Artwork by Rianna Man.
