Open Science and Negative Results Sharing

One of the things open science aims to find out is whether scientific research findings can be replicated or reproduced. Can a different researcher arrive at the same key results given the same data set? We will cover the answer to this question in more detail.

If you haven’t read our article on the open science movement yet, you can find it here.

Increased transparency is valuable for those who argue in favor of open science, which, in turn, becomes necessary when taking negative results into account. The sharing of negative results for a particular area of research or finding is beneficial from many points of view, but sadly, it’s not predominantly the go-to option for most scientists or publications. We will cover both facets.

Are Negative Results really that negative?

Are negative results rendered meaningless? Not necessarily. In fact, negative results could even be considered helpful for further research in the sense that they could inspire new directions for future studies. Negative results should also not be automatically associated with unskilled scientists or poor experimental conduct.

Probably the best example of negative results sharing gone good is the work conducted by Michelson and Morley. The Michelson-Morley experiment became one of the most famous failed experiments to date. The series of experiments they conducted in the 1880s related to the substance ether contradicted the prevalent theory at the time. While other versions of the experiments were carried out, the Michelson-Morley calculations were the first with enough accuracy to challenge the existence of the ether.

Not only did Michelson become the first person to receive the Nobel Prize in Physics in 1907, but the results of their work inspired a new line of research that ultimately led to the insights provided by Einstein’s theory of special relativity in 1905. This example shows how someone else’s theory permits questioning and correcting for the same data set. Additionally, this highlights the importance of disseminating negative data further to find any discrepancies.

Quantity over quality

It is very natural for scientists to want to share their success stories rather than their failures. This can also have a downfall, though. A Psychologist at the University of Virginia, Brian Nosek, states “There is no cost to getting things wrong. The cost is not getting them published”.

There are a plethora more errors in the scientific papers that are being published, written about, or acted upon than most would think. Although academic scientists readily acknowledge that they often make errors in their papers, they also just as easily (and erroneously) believe that those errors will be corrected in time as other scientists take over. Unfortunately, there is some evidence that peer review is not always successful in identifying methodological flaws.

The aim of peer review is to assess a manuscript’s quality and provide feedback that can be used to improve it upon revision. However, peer-reviewing doesn’t always manage to achieve that. In biomedical sciences, for example, there is very little scientific evidence that supports that a peer-review process ensures the quality of manuscripts.

Additionally, the work of experienced scholars rarely receives harsh reviews. And when manuscripts are not properly reviewed and corrected, we’re left with inconclusive research or information once they’re published. As scientists prefer to accentuate the positive over the negative, far more inconsistent results are published in completed manuscripts than those that are corrected or withdrawn at a later date. Negative results, on the other hand, are much more trustworthy. Though, the fact that negative results are largely ignored or not published becomes hugely problematic for both the integrity and advancement of scientific research. This does not, however, mean that all negative results will bring about a groundbreaking revelation. But the good news is that negative results do not have to bring about said revelation to be useful or valuable.

Replication - science’s self-correction mechanism

Sometimes, through replicating other scientists’ work, we can find that those replications make a difference. Other times we, unfortunately, are hit with the fact that replication is hard and thankless.

Several things stand in the way of researchers following through with replication. Some replication impediments include:

Journal disinterest

Journals are no longer interested in results from replication but rather in novelty information. This also leads to some journals choosing quantity over quality. Not just that, but most academic researchers would rather spend time on work that is more likely to advance their career than to replicate another scientist’s work. Junior researchers, for example, see replication as an implicit challenge to authority.

Data unavailability

When considering clinical trials, lives may be on the line. The need to see the results of clinical trials, whether positive or negative, has led to the insistence that clinical trials be registered publicly. Unfortunately, while many clinical trials are eventually published, negative results (where the treatment being tested had no effect) are statistically less likely to be published. And negative trial results that are published come out about one year later than positive studies.

Software issues

Software can also be another impediment to those “would-be” replicators. The result of years of work gives the possessors of specific data, or code to analyze data, an edge in future research. Although most scientists will agree that data should be openly available, there exists genuine disagreement on software.

Conclusion

Journals sometimes fail at sufficiently scrutinizing the results that they publish. Simply moving on from errors or mistakes and not publicly announcing them could severely damage a scientific reputation. There needs to be a value system in place to make scientists and researchers more responsible when it comes to sharing negative results. Instead of further encouraging the spread of incomplete information, researchers could collaborate with other researchers or different communities to improve in this area.

Negative results sharing is an essential aspect of the information we come in contact with. Although there are many impediments to replication, negative results may help future trials, as more information will be available to be disseminated.

Start publishing your negative results with Orvium today!