Understanding Scientifically-Based Research in Education

We want reading instruction for all students to be based on “scientifically based research”.  In RTI students are identified for special education services based on how they respond to interventions that are supported by scientifically based research.  Also, at our colleges and universities, we are required to make sure preservice teachers receive instruction in reading methods that are supported by scientifically based research.  It cannot be just any old kind of research; it must be scientifically based research.  So, what exactly does this term mean?  What does it take to be scientifically based research?  Can only scientists do scientifically based research?  Who decides what kind of research is scientifically based?  What is science?  What is research?

Exploring and explaining these mysteries will be the focus of this short article.

RESEARCH, SCIENCE, AND METHODOLOGY

Research in its most basic form is simply a matter of asking questions and using data to answer those questions.  We do this in varying forms every day.  For example: What happened to my tomato plants?  What is the strange noise in my car engine?  Why is Johnny not learning?  What is the most effective approach to use to teach this skill?  Was my lesson effective? How should I teach reading to students who are struggling readers?  Asking and answering these questions are all forms of research or re-search (to search again).

Scientific research is a process that uses systematic methods to collect data to answer questions.  These systematic methods come in a variety of forms.  Meaning that there really is no such thing as the scientific method; rather, there are methods of science.  In contrast, the US Department of Education has determined that only a single type of research methodology can be used to ask and answer questions in the field of education (see Figure 1.).  They have ironically labeled this, scientifically based research (SBR).  The irony is this: however, can anything be said to be truly scientific if it ignores or invalidates an abundance of data? 

In identifying scientifically based research for use in education, the problem is not with the type of research methodology described in Figure 1.   This has been and continues to be an invaluable research tool.  The problem is with the exclusive use of this type of research methodology to the exclusion of all others.  This methodological peephole creates a very narrow and unrealistic view of the very educational reality it seeks to examine.   Yet, the Federal government (not the source I would use to understand research methodology in any field), has mandated that this is the only type of research that can be used to evaluate reading instruction. 

To be clear, people who were not reading researchers, who had spent little (if any) time reading research related to reading instruction, determined that the only type of reading research that can be used to evaluate reading instruction is that which meets the narrow parameters described in Figure 1.  They decreed that the same research methods used to study the physical world in the artificial confines of a controlled laboratory setting are the only research methods that can be used to study and make causal assertions about human beings in real-world educational settings.  Hence, the only data that counts in coming to understand educational reality is data that is collected using one specific research method.  And this method is said to be “the gold standard”.

Limiting What We Might See  

As stated above, the limitations of the gold standard are not in the methodology itself; rather, in the exclusive use of that methodology to understand educational reality and make causal assumptions.  Causal relationships can be determined using a variety of methodologies. 

The Joy of Petri Dishes

If the students with whom we work were all homogeneous elements existing within the confines of a very large petri dish where all variables could be isolated and experimental treatments could be implemented with fidelity, then the exclusive use of the gold standard to establish causal factors would indeed be appropriate for evaluating reading instruction and other educational variables.  However, this is not the case.  Human beings are wonderfully complex, multidimensional entities existing in social worlds that are incredibly varied and extremely uncontrollable.  These human beings interact with other human beings, all of whom have a diverse array of abilities, motivations, dispositions, values, life situations, and other forces acting upon them.  As such, it is unlikely that causal certainty could ever be achieved with any type of scientific method (Heshusius, 1994; Iano, 1987; Poplin, 2011).  In the world of educational research, certainty is never achieved.  At best, the gold standard can be used to reduce uncertainty. 

INSTRUCTIONAL PRACTICES 

The gold standard mandate imposed by the US Department of Education reflects a simplistic view of science and how it is used to understand phenomena (Seriven, 1994).  This in turn has led to a simplistic view of “research-based” strategies.  One of the results of these simplistic views is that cause-effect relationships have been converted into stimulus-response instructional techniques in which students are manipulated into producing predetermined behaviors that represent measurable learning objectives.  Learning is said to have occurred only when students display the appropriate behavioral response to an instructional stimulus.  While this approach is appropriate for mice in a Skinner box, it is remarkably inappropriate for human beings in a classroom.

Also, the strategies designated to be “research-based” by gold standard research standards include a troubling amount of direct instruction.  While direct instruction has been shown to be effective for learning low-level skills, it is extremely ineffective for developing high-level thinking, understanding complex concepts, and acquiring sophisticated skills (Eppley & Dudley-Marling, 2018).  The subsequent problem is that students who are struggling readers are often force-fed an interminable array of mind-numbing lessons that use direct instruction to teach low-level concepts and skills.  As a result, they have few opportunities to participate in activities that would enable them to develop complex thinking and deeper understandings of the world.  You do not have to control any variables to understand that if only low-level skills are taught, only low-level learning occurs. 

EDUCATIONAL WONDERLAND.

If gold standard research were truly the answer to all problems related to teaching struggling readers (as proponents claim), why have these problems not been solved?  The Education Science Reform Act was passed in 2002.  Surely with all the gold standard researchers clamoring for tenure, promotion, grants, and book contracts all the problems related to reading instruction would have been solved by now.  To help struggling readers, we would simply need to identify the “breakdowns” in the educational machinery or find the “impaired” educational products and then insert the correct research-based widgets into the educational machinery.  Teachers would simply implement the correct research-based widget with “fidelity” and keep the products moving down the conveyor belt.  Simple and tidy, neat as that.  We would be living in an educational wonderland, a utopia where all children would be reading books at or above grade level. 

But this is not the case. Why?  Because the educational world is much more complex and multi-dimensional than can be contained in or controlled by any set of variables.  For the field of education to evolve, it must allow the research method to be determined by the question and not the other way around.  It must be able to escape from the choke hold put on it by the reductionist gold standard paradigm that restricts the flow of new information.

REFERENCES

Eppley, K. & Dudley-Marling, C. (2018). Does direct instruction work? A critical assessment of direct instruction research and its theoretical perspective.  Journal of Curriculum and Pedagogy. Doi.org/10.1080/15505`70.2018.143821

Hushusius, L. (1994). Freeing ourselves from objectivity: Managing subjectivity or turning toward a participatory mode of consciousness?  Educational Researchers, 23, 15-22.

Iano, R. (1987). Rebuttal: Neither the absolute certainty of prescriptive law nor a surrender to mysticism.  Remedial and Special Education, 8, 52-61.

Poplin, M. (2011). Valuing a plurality of research methodologies and instructional ideologies in classroom research.  Learning Disabilities Quarterly, 34, 150-152

Seriven, M. (1994). The psycho-logical foundations of modern science (pp 47-80).  In W. Harman & J. Clark (Eds). New metaphysical foundations of modern science.  Sausalito, CA: Institute of Noetic Sciences.