What Practitioners and Policymakers Want The Communication and

What Practitioners and Policymakers Want – The Communication and Miscommunication of Education Research TIMOTHY SHANAHAN UNIVERSITY OF ILLINOIS AT CHICAGO WWW. SHANAHANONLITERACY. COM

Concerns about Application of Education Research “Concern about the weak links between research and public policy is not new. Even as the volume of applied social science research has increased over the past 20 years, policymakers have decried a general lack of useful information and researchers have grown frustrated as their studies gather the proverbial dust of disuse. ” –Mc. Donnell, 1988 “Why don’t teachers use education research in teaching? ” Mac. Lellan, 2016 “many teachers view research as a search for bright shiny objects pushed by administrators without adequate attention to the needs and skills of teachers” (Schneider, 2018) “Despite the good intentions of many policymakers to solve problems with evidence-based solutions, their decisions are often influenced by ideology, political pressure, and partisan information. ” –Stone, 2018

These concerns are common The idea of research-based policy and practice is a major concern in many other fields of study including medicine, various social sciences, environmental sciences, etc. Lots of reasons for the lack of implementation of research have been advanced in scholarly treatments of the issue and the major conclusion repeatedly is that the problem is not with the quality of the research but with miscommunication and sociological mismatches between communities And those problems seem particularly basic…

The equivalent of Mel Gibson’s hair dryer accident Peter Afflerbach, University of Maryland Debbie Hunsaker, Montana Department of Education Beth Ann Bryant, Former Senior Advisor to the Secretary of Education Evan Lefsky, Public Consulting Group David Coleman, College Board P. David Pearson, University of California, Berkeley Jan Dole, University of Utah Michael Petrilli, Thomas Fordham Institute Nell Duke, University of Michigan Robert Pondiscio, Thomas Fordham Institute Doug Fisher, San Diego State University Chris Rausher, American Institute of Research John Hattie, Melbourne Education Research Institute Robert Slavin, Johns Hopkins University James Hoffman, University of Texas Mary. Ellen Vogt, California State Univ, Long Beach

Different Worlds, Different Cultures People who live in different worlds, develop different cultures We often focus on cultural divisions: men and women, rich and poor, Black and White, red states/blue states, urban and rural, etc. Many of the experts whom I consulted focused heavily on the fact that researchers/policymakers live in different worlds and therefore have trouble communicating effectively

Two Worlds, Two Cultures Practitioners/Policymakers Researchers Time-frames Schools are busy places, where Research is slower and methodical, many decisions have to be made on things take time—data collection may the fly, crises are almost constant, be the briefest part of a study things must move at a fairly quick pace Values Things need to work smoothly, results are expected quickly; desire actionable advice that will deliver results on scale or confirmation Publications, conference paper acceptance, citation counts and journal impact factors Language Stories, anecdotes focused on uncharacteristic cases Statistical significance, RCTs, effect sizes Rewards systems Effectiveness, smoothness of operations, public acceptance, P&T system: publication, grants, participation in public service

National Reading Panel (1997 -2000) Panel was formed to make “a determination of fact” as to what the research findings were concerning reading instruction The panel was to be selected by the Secretary of Education and the Director of NICHD and was to be supervised by NICHD (because of its portfolio) By law the panel had to include a teacher, a school administrator, and a parent in addition to a dozen scientists The non-scientists struggled to participate My negative reaction and what I found out

Research Training for Practitioners One of the biggest differences between medicine and education is that medical practitioners are trained in research This requirement is not usual in teacher and principal preparation standards, and such requirements are not common even in Colleges of Education within Research I universities It is difficult to get people to value research if they neither understand its methods and logic nor its potential value Wouldn’t that be a great project for the National Science Foundation or the National Academy of Education?

But “they” aren’t the only ones who need to change Much as Mel Gibson learned in “What Women Want, ” problems with communication tend to be on both sides of the divide It would be wonderful if we could develop a generation of educators (especially those who are leaders) who are sufficiently knowledgeable about research and its limitations But it would also be wonderful if we were to reform ourselves sufficiently to improve the communication of research I was taken aback by the tone and content of many of my informants who helped guide this presentation today They expressed views of policymakers and practitioners who were angry, disappointed, and disillusioned by researchers and their messages

What “they” think about “us” Researchers often don’t honor or respect educators’ knowledge and experience Researchers look down their noses, and treat teachers as less sophisticated and knowledgeable Research problems ignore the information needs of educators—researchers’ should seek input from educators with regard to what should be studied Researchers are out of touch with the “real world” of the classroom or school Researchers are arrogant and lack humility (”do not continue to tell us about your expertise, once is enough, ” “follow through on your commitments to schools”) Teachers often resent that researchers make more $$$ than they do—and, in their opinion, do much less work Researchers do not communicate research in terms that are convincing or meaningful to practitioners Researchers swoop in and give advice and leave Researchers too often try to win an argument rather than trying to inform one

Meeting the needs of educators The notion that educators should consult the research when making decisions is deeply engrained in the underlying philosophy of SREE and its membership (as it should be) But the notion that the research community should listen to practitioners and policymakers when it comes to what needs to be studied is not Researchers pursue issues that interest them, that they think can be funded, that are of theoretical interest, that will connect with other studies, and so on–but we have few mechanisms for identifying what it is that educators and policymakers seek to know

Meeting the needs of educators (cont. ) Even infrastructure like the What Works Clearinghouse fails to seek the expressed needs of educators in their Practice Guides Likewise, IES would benefit from practitioner and policymaker advisory panels to help shape the research directions they support or encourage The Regional Education Labs are a good example of what I’m talking about, though even they don’t systematically attempt to identify educators’ questions (they do sometimes respond to policymaker questions—but the labs are largely unknown to educators surveys have shown)

Research should provide the kinds of information “they” want There have been various calls for providing training to researchers in how to to report research to practitioners or for creating a corps of research translators I think the needs here are even more basic or foundational than that and have to do with the nature of what we should be finding out in our studies (despite calls for greater simplicity and coherence of research reporting, practitioners usually ask for more information)

Research should provide the kinds of information “they” want (cont. ) Greater attention to control group – if an intervention ”works” it confers a greater learning advantage than something else (and we need to describe that something more thoroughly)

Research should provide the kinds of information “they” want (cont. ) Greater attention to variation in outcomes – no matter how effective a treatment, some kids don’t benefit (marginal gains), and this is variation that educators and policymakers are interested in

Research should provide the kinds of information “they” want (cont. ) Greater attention to variations and adjustments in inputs – our notions of fidelity of treatment tend to leave out any of the natural treatment variation and responsiveness to student needs that practitioners need to know

Research should provide the kinds of information “they” want (cont. ) Greater attention to dosage – how much treatment was provided and how much instruction of a particular type is then to be prescribed?

Research should provide the kinds of information “they” want (cont. ) Greater attention to relevant anecdote – teachers and policymakers often make decisions on the basis of resonant anecdote (humanity, real world): and our research should produce not only statistically sound results, but a collection of relevant practical anecdotes that are consistent with these results

Need for public deliberations over major issues Practitioners and policymakers are not interested in research itself – they want to solve problems and seek information that could help them to do so (it is rare that a single study will meet their needs) We don’t need to be advocates for a particular position, approach or program, until the accumulation of evidence is sufficient that there is substantial agreement in the research community Our rhetoric needs to acknowledge that when we do support a particular approach or intervention we are doing so because it is “marginally” better – not because it works and the alternative does not

Need for public deliberations over major issues (cont. ) Our rhetoric needs to acknowledge conflicting information and to treat it equivalently – when we don’t do this, practitioners conclude that you can “prove anything with research” Our rhetoric needs to be as candid about what we don’t yet know as it is about what we do know (if we don’t know if this approach works for African American kids, dyslexics, ELLs, or kids in the upper elementary grades, we should be forthright about that) Our rhetoric needs to be a model of willingness to be moved by data, we need to demonstrate our own belief in data and its ability to solve problems

”Knowledge is proud that he has learn’d so much; Wisdom is humble that he knows no more. ” --William Cowper “Winter Walk at Noon”