Sometimes things fall apart—no one understands this better than engineers because they are usually the folks called upon to put those broken things to right. One of their tools? Scientific facts that enable makers of all kinds to choose the right course of action. In this post, Robert Stephenson probes how the facts themselves are derived, verified, challenged, and revalidated.—Editor

Author: Robert Stephenson

I was recently reminded of a quote by John Adams: “Facts are stubborn things; and whatever may be our wishes, our inclinations, or the dictates of our passions, they cannot alter the state of facts and evidence” [1]. This notion of the stubbornness of facts rings louder than ever for those of us working in the science and technology community.

Here at MITRE, most of us belong to the science and technology community, and for many of us, scientific studies are one of the foundational tools we use to discern fact from fiction. However, over the past decade, multiple new reports have shined light on the fallibility of studies across multiple scientific fields. In 2011, Life Science company Bayer analyzed 67 published drug research findings and found that more than three quarters of replicated results had data findings that did not match with original data [2]. In 2015, the Open Science Collaboration (OSC) attempted to replicate one hundred published psychology experiments and found that 65 percent of the experiments failed to show statistical significance upon replication [3]. Several other reports demonstrate that many prominent scientific journals, which should represent the shining beacon of the scientific method and scientific achievement, might not be the sources of truth we once thought they were. As it becomes more difficult to cite scientific studies as a source of fact, without feeling doubt as to whether the referenced information is factual, our way forward in the pursuit of truth can appear perilous, with no haven in sight.

Despite these grim realities, there is still hope for all of us, because facts, according the Oxford English Dictionary, are still things that are proven to be true, and scientific study is still one of the best methods we have of discerning facts. Clarity on this claim can be found in the notion expounded on by Christie Aschwanden in her 2015 article, “Science Isn’t Broken,” in which she states that “Science is hard… (and) requires lots of choices that can shape the results. This doesn’t mean that science is unreliable. It just means that it’s more challenging than we sometimes give it credit for.” [4]

For those of us at MITRE, best practices and lessons learned in systems engineering have been provided in the Systems Engineering Guide to help guide us in our fact finding in support of government sponsors. With this knowledge in hand, those of us in working in science and technology must strive to put in the hard work and due diligence to produce valuable results, while not succumbing to fears and doubts that might discourage us from working to solve the riddles of the universe when the complexity of the task seems overwhelming.

It’s easy to view Mr. Adams’ statement as an indicator that our pursuit of facts will never become a simple endeavor. However, it’s important to remember how far we have come on this earth, and that our ability to reason and build upon complex ideas with one another is our greatest asset in discovering the facts of this universe.

[1] John Adams, “Speech by John Adams at the Boston Massacre Trial”, Boston Massacre Historical Society. Accessed 2 March 2017: http://www.bostonmassacre.net/trial/acct-adams3.htm

[2] William A. Wilson, “Scientific Regress,” First Things, May 2016. Accessed 24 January 2017: https://www.firstthings.com/article/2016/05/scientific-regress

[3] Pascal-Emmanuel Gobry, “Big Science is broken,” The Week, 18 April 2016. Accessed 24 January 2017: https://www.firstthings.com/article/2016/05/scientific-regress

[4] Christie Aschwanden, “Science Isn’t Broken”, FiveThirtyEight, 19 August 2015, scientific method. Accessed 24 January 2017: https://fivethirtyeight.com/features/science-isnt-broken/

Robert Stephenson is a systems engineer, with degrees from Clemson and the University of Virginia. A maker by disposition, Robert has tested the waters from more than one hemisphere. He joined MITRE in 2016.

© 2017 The MITRE Corporation. All rights reserved. Approved for public release; Distribution unlimited. Case Number 17-3877

The MITRE Corporation is a not-for-profit organization that operates research and development centers sponsored by the federal government. Learn more about MITRE.