The Publish-or-Perish Paradox: Unraveling the Reproducibility Dilemma

Thank you. Listen to the article through the player above. ✖

The field of science is currently facing a reproducibility crisis, and a recent investigation published in PLoS Biology indicates that many biomedical researchers attribute this issue to the notorious “publish or perish” research culture. 

Irreproducibility in biomedical investigations

Reproducibility fosters trust in scientific endeavors; it promotes progress in science and ensures that research can make a significant impact globally.

Unfortunately, there is substantial evidence suggesting that the scientific community is grappling with a reproducibility crisis. A 2016 Nature survey involving 1,500 researchers revealed that over 70% could not replicate another scientist’s experiments. Additionally, more than 50% were unable to reproduce their own findings.

The same survey indicated that 83% of participants concur that a reproducibility crisis exists in science, with 52% expressing that they perceive this crisis as “significant”.

Building upon this survey (but crucially, not attempting to replicate it), the recent study in PLoS Biology sought to delve into this intricate matter, particularly within the biomedical research community.

“It is critically important to ensure that biomedical research is reproducible: findings that were later found to be non-reproducible have resulted in patient harm,” the authors stated. “By capturing a varied and global representation of biomedical researchers’ views on reproducibility in the field, we aim to gain a better understanding of how to guarantee reproducibility in biomedicine.”

Over 1600 biomedical researchers share their insights on reproducibility

The investigation, led by Dr. Kelly Cobey, an associate professor in the School of Epidemiology and Public Health at the University of Ottawa Heart Institute, randomly selected 400 journals listed in MEDLINE from October 1, 2020, to October 1, 2021.

Authors of these articles were contacted and invited to participate in an anonymous online survey that gathered demographic information, views on the reproducibility crisis, perceived causes of irreproducibility in research, experiences with conducting reproducibility studies, and their knowledge regarding funding and education for research in reproducibility. 

“We characterize reproducibility as repeating a study utilizing similar methods and achieving results consistent with the original study, and as irreproducible when the results do not align with the original study. This definition accommodates variations in methodology (e.g., conceptual and direct replications) between the original study and the reproducibility study and also incorporates different interpretations of how ‘consistent results’ are defined (i.e., using p-values, observing results in the same direction, comparing effect sizes),” explained Cobey and colleagues described.

More than 1600 researchers from over 80 nations participated. The majority of respondents were faculty members or primary investigators, with more than 50% identifying as male. Approximately half of the participants were engaged in clinical research.

Emphasis on quantity over quality fuels irreproducibility in science

Cobey and collaborators discovered that 72% of survey participants concur that a reproducibility crisis is present in biomedicine. About 27% of these respondents consider this crisis to be “significant.”

“The concern seems to extend to biomedicine in general, but notably to clinical research, in vivo studies, and in vitro studies (11% or fewer participants indicated that they believe more than 80% of publications in each category are reproducible),” stated Cobey and colleagues said.

More than 62% of respondents attributed the irreproducibility in science to the “publish or perish” mentality.

The “publish or perish” phenomenon reflects the unfortunate truth that, frequently, researchers must continually publish in esteemed journals to ensure their long-term career advancement. This generates an ongoing cycle of anxiety and pressure that is not conducive to a thriving research landscape. 

“Concerns regarding the current academic reward system prioritizing quantity over quality have been raised for decades – a notion supported by the data from this study, which suggests that the performance of researchers is adversely affected, in terms of generating reproducible research, by what the academic system rewards,” emphasized Cobey and colleagues emphasized.

Respondents pointed out various other perceived factors contributing to irreproducibility, including but not limited to poor experimental design, fraud, and minimal…quality peer assessment and deficiency of training in reproducibility. Merely 16% of respondents believed that their respective institutions had implemented protocols to promote reproducibility in biomedical research. Sixty-seven percent of respondents indicated a perception that their institution prioritized novel research over replication studies.

Assisting researchers with reproducible scientific initiatives

Cobey and associates are of the opinion that their findings could serve to inform training and initiatives aimed at enhancing reproducibility. They further assert that repeating the same survey over time could assist in examining how perceptions and behaviors are transforming.

“This global survey presents a modern snapshot of the biomedical community. While our survey method and the direction of results are aligned with the earlier Nature study, continuous monitoring of the community’s views on reproducibility is essential to assess changes over time. Indeed, administering this same survey again in the future would enable a temporal evaluation of how perceptions and behaviors evolve over time.”

Reference: Cobey KD, Ebrahimzadeh S, Page MJ, et al. Perspectives of biomedical researchers on the reproducibility of research. PLOS Biol. 2024;22(11):e3002870. doi: 10.1371/journal.pbio.3002870