Research training programs, research on research, the development of uniform guidelines and tools, and improved incentive structures are generating new opportunities to promote, demonstrate, and sustain rigorous research. However, it is not possible to completely eliminate irreproducible research.
Due to existing variables, including the presence of random error, you may not always be able to replicate your own work. Irreproducible work is not always caused by poor research practices; it may be due to a lack of necessary knowledge that has not yet been discovered or uncontrollable environmental conditions. When research results in irreproducible data, it can stimulate scientists to look deeper or redirect their work, which can lead to the generation of critical, new knowledge.
Click "Next" to review the story of Otto Loewi (1873–1961), whose irreproducible research in the 1920s culminated in the Nobel Prize in Physiology or Medicine in 1936.
Continue on to learn more about the story of Otto Loewi (1873–1961), whose irreproducible research in the 1920s culminated in the Nobel Prize in Physiology or Medicine in 1936.
One night in 1920, Otto Loewi woke up and wrote down some new ideas related to a hypothesis he had formulated years ago. Unfortunately, the next day he could not read his notes nor remember what they were about – not until the early hours of the morning.
Once he remembered his ideas from the previous night, Loewi immediately went to his lab and performed an experiment to test them.
Otto Loewi had been thinking about chemical transmission in nerves since 1903. His dreams occurred in 1920 and he conducted a series of experiments in the middle of the night using two isolated beating frog hearts in individual perfusion chambers. He was able to show that nerve impulses could be transmitted through the secretions of chemicals from the terminals of nerves. Otto Loewi successfully demonstrated the principle of chemical transmission that night. The chemical signals are now known respectively as acetylcholine and adrenaline.
For several years, the pivotal experiment that Dr. Loewi conducted in 1920 could not be reproduced. However, the quests to replicate his work led to a much deeper understanding of the original observation, demonstrating that uncertainty and irreproducible research can effectively drive scientific progress.
There were multiple variables affected by the simple experiment. Through further research it was found that the results were influenced by:
Otto Loewi first met Sir Henry Dale (1875–1968) in London in 1902 and they became lifelong friends. Dale predicted the presence of an enzyme in the body that rapidly metabolized acetylcholine. This work complemented Otto Loewi's research and the two shared the 1936 Nobel Prize for their contributions related to chemical neurotransmission.
Had Otto Loewi slept through the night, he may not have made his important discovery. He could have conducted his experiment at a different time of the day, during a different season, or using a different species of frog and his results may have been unremarkable. Otto Loewi was fortunate to succeed during his nocturnal experimentation; however, it took many years of painstaking research to clarify the conditions needed to reliably reproduce his discovery!
Not all irreproducible experiments can be reproduced as Otto Loewi's research was. Sometimes the results are simply false positives and cannot be replicated, even if you test multiple conditions. When approached correctly, however, all irreproducible research can be a learning opportunity. Whether the results cause you to challenge and improve the "failed" experiment or you use the knowledge you gained to influence how you design future experiments, there are always lessons to be learned from all research.
As you think about this story, consider your research activities and the following questions. Make a note of your response and then click the button for some feedback.
As you think about this story, consider your research activities and the following questions. Make a note of your response and then continue on for some feedback.
Have you ever been unable to replicate an experiment that you conducted?
There is a natural tendency to immediately disregard the repeat experiment that is at odds with previous work. While challenging, trying to clarify the basis for inconsistent findings can lead to a deeper and sharper understanding of your work. Indeed, this is fundamental to scientific advancement. Remember that if the unexpected results persist, it is important to find ways to share them to inform others who may be encountering similar challenges or considering similar approaches.
Were you able to identify the cause of irreproducibility?
There are a multitude of reasons as to why experiments cannot be reproduced. There may be unrecognized problems at any point in the research process, including understanding of the literature, experimental design, data interpretation, reliability of tools/materials, or false positives. Any scientist can encounter a false positive at any time, so it is important to be patient, to conduct replication studies, and to be prepared to dig deeper to understand the conditions of the experiment.
Did the investigation lead to a deeper understanding of your work?
Identifying the cause of research irreproducibility is key to rigorous and responsible research practice. Even if the irreproducible (but transparent and high-quality) work leads you in new directions, you will be advancing your understanding and contributing new knowledge to the research enterprise.
Did the investigation lead to improvements in study design, procedure, or analysis?
Careful and rigorous investigation can often lead to basic modifications in methods, materials, analyses, data interpretation, and use of technology or tools, leading to more robust research outcomes. As with the story of Otto Loewi, this can markedly deepen our understanding and advance knowledge.
There is some comfort from knowing that we will learn from reproducible work and from work that takes longer to assess. In each case, transparency, objectivity, and good documentation practices are critical.
In addition, these best practices support the expanding needs of scientific collaboration and the management and use of "big data." It is increasingly important to focus on strategies and practices that will make it easier to manage, secure, share, and curate extensive amounts of research data in a way that facilitates collaboration, data sharing and reporting, as well as research quality and reproducibility.
There are practical considerations that may influence the approach you choose to take when performing research.
There are practical considerations that may influence the approach you choose to take when performing research. The more certain we want to be about our research, the more time and energy we may need to invest in it.
Focusing on one research project can divert resources from other projects, so it is something to consider thoroughly. How will you distribute your resources? Which research projects should you focus your attention on?