This blog was originally published on the Brookings Institution’s Brown Center Chalkboard.
As millions of students head back to school, families are probably wondering if those shiny new devices, apps, and even games that are becoming a typical part of the school day are good for learning. As an education researcher focused on blended learning, I am often asked if education technology “works.” The underlying question here for all of us, myself included, is: “Based on the current evidence, do I want my child’s educational experience to include ed tech?”
The answer to this question is often some variation of “it depends.” You’ve probably heard that the research base on ed tech is young, or that evidence is mixed, or perhaps you’ve read that asking if it “works” is not the right question to begin with. I recognize how frustrating and unhelpful these answers are to families and educators, and I have been noticing a trend that I think is quite stable and relevant to those of us working to provide effective learning experiences to all students.
Generally, the evidence base on ed tech supports existing evidence about in-person learning. In other words, teaching practices are the key to effective instruction and matter more than the presence or absence of specific tools (e.g., textbooks, blackboards, devices) in the learning environment. Therefore, ed tech is effective to the extent that it can support teachers in delivering, scaling, and sustaining effective teaching practices. This is unsurprising when you consider that the most effective teaching practices are those which require the most nuanced and sophisticated human expertise.
Contradictory Findings Tell the Same Story
I’d like to illustrate this trend by taking a deep dive into a study of blended learning, published in 2018, that concluded blended learning is more effective than typical instruction in similar grade-levels and content areas.
In the Tennessee Department of Education’s 2016-2017 Blended Learning Pilot Report, researchers investigated whether an Algebra-focused pilot of blended learning increased student achievement (among other things). This pilot initiative was aligned with Tennessee’s strategic goal to focus “individualized opportunities and support” on students who were furthest behind. Tennessee defined blended learning as, “…the combination of strong human teaching strategies and technology-based teaching strategies fused together to strategically personalize learning for students.” (p.1). This report focuses on the 2016-2017 school year, the pilot itself continued through the spring of 2018.
The mixed-methods study included 50, predominantly Algebra I, teachers and over 5.000 middle and high school students from 37 schools in 21 districts. Researchers used students’ standardized scores on their End of Course Algebra assessments as the outcome measure, and their 2014-2015 math scores on a state standardized measure to control for any differences in math achievement between students in the pilot and comparison students. The authors reported, “The results, taken as a whole, show a positive relationship between participating in the blended learning pilot and increased test scores.” (p. 8). I should note here that these results are encouraging, but the authors point out (p. 10) that these results should be further substantiated through more research.
At first glance, this appears to contradict other recent studies, e.g., Backes and Hansen’s 2018 examination of the Opportunity Culture blended learning model, and Miron, Shank, and Davidson’s 2018 Virtual Schools Report that found blended learning to be less effective than traditional instruction.
How can all of these studies be right? More importantly, is there a single, actionable takeaway from them? The answer lies in how the effective program used ed tech to blend instruction. In Tennessee, ed tech was used in a “deliberate, measured” (p. 1) way to increase students’ ownership of their learning, to differentiate instruction, and to allow teachers to focus on the students that needed the most support. From my perspective, these varied results demonstrate the importance of using ed tech to support what we already know to be good teaching—because there are inconsistent findings about “blended learning” as a collective framework of instructional strategies, it is important to ground implementation in strategies that are evidence-based. The Tennessee study investigated blending that was used to prioritize known-effective practices and allowed teachers to spend more time on providing more personalized instruction for more students, thus resulting in positive impacts.
Teaching > Tech
Perhaps the best kept secret of research on ed tech is the fact that teachers and teaching remain the most important influences on learning. When ed tech is used in ways that support good teaching practices, both in digital and face-to-face environments, effective learning happens. When ed tech is used in ways that propagate ineffective instructional strategies, learning is hindered.
Further support for this perspective can be found in another 2018 report on personalized learning by the Center for Reinventing Public Education. In “Personalized Learning at a Crossroads,” the authors studied two initiatives funded by the Bill & Melinda Gates Foundation to determine how educators design and implement personalized learning, and how school and district conditions support or impede the implementation and spread of innovation. This report is quite comprehensive and focuses on many broad issues, but one small nugget within their findings is particularly relevant to this discussion. What they found, among other things, was that the observed enthusiasm and even action from educators was not enough to scale and sustain innovations without an attendant focus on instructional practices—rather than classroom structures—supported by school and district administrators.
Simply adding ed tech to a learning environment doesn’t guarantee better learning. In fact, using ed tech to replace teachers or to scale ineffective practice guarantees poorer learning outcomes. When it comes to teaching and learning, it’s not what you use, but how you use it that counts.
