Now Reading
In COVID-19 Research, Not All That Glitters Is Gold

In COVID-19 Research, Not All That Glitters Is Gold

Featured image: pixabay/Pexels.

As the novel coronavirus pandemic continues in the physical world, a parallel pandemic is underway in the scientific literature. Researchers around the world had already published more than 100,000 articles and papers on COVID-19 and related issues by May. They include findings about the nature of the virus, epidemiological models and predictions about how the pandemic could evolve.

Scientific studies usually don’t receive much public attention but as one of the greatest humanitarian disasters of the 21st century continues to unfold, articles about research and public health have seldom left the front-pages of India’s, and the world’s, major newspapers and news websites. It seems scientists everywhere are helping quell the pandemic and its associated crises.

However, this may not be the full picture.

In mid-May, Jeffrey Brainard wrote in Science that scientists today are inundated with COVID-19-related papers. There has been a veritable explosion in the number of peer-reviewed and preprint research articles in the last few months, as researchers race to publish their findings.

It’s impossible to keep up with the flood of papers about the disease and the virus. Many research groups have in fact turned to advanced computational tools and analytical software to even classify research articles. At the same time, it can also be said that not all publications are equally valuable. They simply can’t be.

Recently, researchers at the Johns Hopkins University created the Novel Coronavirus Research Compendium (NCRC), a publicly available resource to assess emerging research on COVID-19 in eight areas: diagnostics, modelling, epidemiology, pharmaceutical interventions, non-pharmaceutical interventions, clinical presentation and prognostic risk factors, vaccines, and ecology and spillover. Until June 28, the NCRC team had selected 238 research papers on these eight topics combined, implying that these papers were the only ones they had found to be credible and robust.

Now, 238 out of 137,000 is only 0.17%; the remaining 99.83% had been left out because it contained only commentaries and protocols, presented low-quality models or simply contained no original findings. Of course, it is possible the NCRC team missed many good articles and/or misjudged others. At the same time, it seems likely that the final total could be off by a few hundred articles, and certainly not by the tens of thousands.

This realisation at least rings true because academia already has a well-known problem called ‘publish or perish’. Academicians often race to publish their results because they are motivated by a desperation to secure lucrative but limited academic positions and awards.

However, even if only during this pandemic, even scientific journals reputed for their strong gatekeeping of the scientific literature have tripped up. The Lancet and the New England Journal of Medicine have both had to retract published papers about using hydroxychloroquine to treat COVID-19. Forty-five epidemiologists also wrote a letter demanding that the Proceedings of the National Academy of Sciences retract a published paper for claiming airborne transmission is the “dominant [mode] for the spread of COVID-19” despite evidence to the contrary.

And this is just what we have noticed, largely because these are prominent journals read by hundreds of thousands of researchers around the world. There is likely to be a glut of papers of little or no value tucked away in less popular journals, and more so in the pages of titles published by low-grade publishers.

So we must consider how much of the ‘explosion’ of research articles is the result of peer pressure to publish more and more. Everything from appointments and promotions to salary hikes and grants is directly related to one’s research output, which is often just the number of papers published. As a result, many studies are simply studies to boost one’s chances of a promotion, and don’t necessarily contain findings that advance the field.

In addition, many universities as well as research-funding organisations pay attention to the ‘prestige’ of the journals in which their researchers publish as well. These journals in turn have their own preferences about what they will and won’t publish, further limiting the scope of ideas that eventually enter the literature.

In this setting, the pandemic resembles an opportunity: epidemiological data is widely available, public demand for scientists’ opinions is high, and journals need to stay relevant. This mix of circumstances effectively accelerates the publication of papers related to COVID-19. As one Nature News & Views report noted, many scientists, including physicists and mathematicians, have even switched their research interests to COVID-19.

Ultimately, we perpetuate a self-fulfilling prophecy if we push the ‘publish or perish’ policy on the one hand while we assess the scientific community’s response based on the number of papers alone on the other – a prophecy that holds little of actual value.

Atanu Biswas is a professor of statistics at the Indian Statistical Institute, Kolkata.

Scroll To Top