The state of null results: a new survey of researchers

Author: Deidre Hudson Reuss

Senior Marketing Manager

Globally, the scale of research published is roughly three times larger than it was 30 years ago. Yet an estimated €26 billion in Europe is wasted in any one year by research that is conducted but not shared by publication. Springer Nature’s new white paper, based on responses from 11,069 researchers across 166 countries, is the first large-scale look at why ‘null results’ are not getting published, and why this needs to change.  

I spoke to Springer Nature’s Chief Scientific Officer, Ritu Dhand, about what the survey tells us. 

What is a null result, and why do they matter?

In the survey, we defined null results as ‘an outcome that does not confirm the desired hypothesis.’ It is not that there is ‘no data’, but rather that the data fail to support the expected effect. Researchers need access to all results to fully understand a research topic. One person’s ‘null’ result may mean something positive for another researcher. Yet the scientific community appears to have created a research environment where researchers either feel little point in writing up null results or struggle to publish negative findings. We’re wasting time and money re-running the same experiments because results that don’t conform to any given hypothesis are considered to be unworthy of publication and thus are not being shared.

What’s a real-world example of null results?

One example that comes to mind is from a time when impactful papers were being published around the findings of a mouse-knockout.  A researcher built an entire grant proposal around a mouse knockout experiment only to learn, privately, that the embryos had no phenotype. The null result had never been published, and that single omission would have burned through a huge grant. Scale that up and you see why duplicated studies cost funders so much every year. It’s a waste of time, waste of money, and ultimately affects people’s lives.

What other problems does not publishing these results create?

Science could advance so much faster if we publish null results. Unpublished null findings allow researchers to start one rung higher on the ladder. By knowing what doesn’t work, researchers can more quickly redirect efforts to finding a new hypothesis or method that does. If just one researcher is inspired by a null finding, and that negative finding contributes to a new scientific discovery, then that value should be recognised.

In a global research environment, it is also no longer adequate to just share your results with close colleagues. Publishing null results ensures reproducibility and equitable access.

What motivated Springer Nature to undertake this survey?

Although everyone knows null results exist, we wanted to better understand why null results are not regularly being reported. One of the main findings of our survey is that the majority of researchers acknowledge their importance. The fact that we heard from more than 11,000 researchers shows this is a system-wide problem that affects every region and discipline. It was as though researchers were saying, ‘Thank you for asking, this needs discussing.’

Key findings from the survey:

What challenges to publishing null results did the survey find?

The survey identified an intent-action gap: although most believe that sharing null results is important, there are less than a third sharing effectively via journals. There are six notable challenges we need to address:

Firstly, outdated sharing habits. The research community is global; swapping data informally worked 25 years ago but today, your community is no longer on your doorstep.

Secondly, researchers are assessed in the main by where they publish and citations continue to drive research assessment and career progression, but a traditional focus on formally only citing positive advances in a research paper also means that null papers are by default not cited and thus appear to lack merit using traditional metrics. We need to encourage new metrics that recognise and reward all rigorous research. 

Thirdly, the number one barrier researchers identified is the fear that journals won’t accept null results. 82% of researchers agreed that ‘null results are less likely to be accepted for publication at a journal’ was their greatest concern when submitting to a journal. In practice, more than half of null-result paper submissions are accepted (58%), but fears here outpace reality.

This is compounded by poor signposting: only 15% of researchers were aware of a journal that encourages publication of null results. If a journal considers null results, then they need to clearly indicate this in the author guidelines.

There is also a fear of negative consequences from publishing a null result. 20% reported career concerns. That worry often confuses two issues. We’re talking about original studies that yield null results, not papers that claim earlier work was wrong. The former should enhance a reputation for rigour, not harm it. The reality is that most authors cited benefits from sharing null results, including enhanced reputation and new opportunities for collaboration.

Lastly, 55% of respondents were unaware of any institutional or funder support for publishing nulls. Interestingly, those who were aware of support were more likely to publish their null results: 72% had shared null results and 34% had submitted null results to a journal. It’s a clear signal that we can boost publication of null results through providing more training.

What is Springer Nature doing to encourage the publication of null results?

Springer Nature is dedicated to communicating all forms of research and we publish a range of inclusive journals that aim to publish all in-scope, technically sound research that has undergone rigorous peer review and validation. These journals provide a platform for null results, foundational and fundamental advances, as well as more descriptive papers on experimental design and data studies, which support reproducibility and data sharing. 

By publishing this white paper, we also hope to stimulate more discussion to address the barriers we have identified. There are actions for all stakeholders across the research community, including publishers, institutions and funders. We need to ensure researchers see value in sharing null results, are aware of how and where they can share them, and are incentivised to do so. 

Read the white paper The state of null results for further insights.

Ritu Dhand, Chief Scientific Officer, Springer Nature

Ritu Dhand is responsible for championing our editors, focusing on promoting and driving external editorial excellence, in partnership with all the journal publishers across Springer and BMC journals.

Before taking on the role of Chief Scientific Officer in January 2022, Ritu served as VP Nature Editorial, overseeing editorial strategy and management of Nature, Nature Communications, and the Nature Research and Review Journals.

Ritu holds a PhD in cancer research from University College, London.

Don't miss the latest news and blogs, sign up to The Source Monthly Digest!