Why do we refer to scientific “reasoning”?
We use “reasoning” to emphasize the process or logic of how to analyze data: i.e. how to structure the process of solving data puzzles with statistical and computational tools; how to determine what kind of scaffolding is needed for solving problems with data; or how to decide what comes next while analyzing data. In other words, we refer to “reasoning” as the navigational system for solving data problems.
This differs from efforts that apply data science to science (like “AI for physics”). Or courses that are about scientific method but do not teach data science. Or classes that teach philosophy of science and data science separately, without integrating both throughout.
Philosophers of science tend to agree that there is no one definition of scientific method.
So what do we mean by “scientific” reasoning?
First, a way of reasoning that looks for being wrong in order to ultimately get it right – instead of primarily seeking confirmation for pre-existing beliefs. This means actively and systematically looking for errors and biases in our assumptions, data, models and results. And that then seeking to correct this. The expectation is that, as humans, we are prone to fool ourselves. Scientific reasoning and methods are evolving responses to counteract this tendency — at least that is the aspiration. The idea is that we care about evidence and are willing to update beliefs in the face of countervailing evidence — if we fail to do so as individuals, ideally the scientific community holds us to account or updates beliefs with the next generation of scholars.
This is a practical take on the more theoretical debates about falsification and refutation in philosophy of science.
It translates to teaching materials and resources that put pitfalls in reasoning and statistical biases front and center. The goal is to teach students not only to debug their code but also to debug the underlying logic of how they address a research problem.
Second, a systematic and enduring search for what might be wrong, not a one-off shot. It is part of a series of tests over time, where understanding is built cumulatively, not a one-time crucial experiment. The hope is that, through this process, we are able to gain knowledge that has been tested rigorously and is thus more reliable than knowledge that has not undergone this vetting process.
Third, an iterative process that goes back and forth between potential explanations and evidence, as both hypotheses and data are updated in a continuous effort to learn and improve results. It reflects an understanding of science as a methodical process instead of a collection of facts. This iterative (“abductive”) process differs from confirmatory approaches that test a theory only once. It is also distinct from approaches that “let the data speak” without much theoretical framing — or from non-empirical logical reasoning based on axioms and proofs. Finally, it varies from models that seek to predict if something happens or whether a treatment works, with less emphasis on why it happens or works.
For more detail, this is some of the literature we draw on in this context.