Principal Investigator and Program Director Views of Successful

Principal Investigator and Program Director Views of Successful Propagation Raina Khatri*, Charles Henderson*, Renee Cole†, Jeff Froyd‡ Abstract We are beginning a project to help developers of educational innovations promote adoption of their work at other institutions through deliberate propagation strategies. To gain a better understanding of the current situation with regard to the spread of educational innovations, we analyzed a webbased survey of 1284 Principal Investigators (PIs) in the NSF Transforming Undergraduate Education in STEM (TUES) program and held focus groups with NSF TUES program directors (PDs). Overall, PIs tend to think of spreading their innovations through one-way transmission methods, such as publishing a paper. On the other hand, PDs think that interactive methods, such as multi-day workshops, are more effective. We recommend use of the term “propagation”, rather than “dissemination”, to describe the process from awareness to adoption of educational innovations by other faculty. What is Increasing the Impact? • Goal: To help educational researchers increase the impact of their projects. • We are studying TUES projects and other programs to map the type of educational innovation and the propagation strategies used to adoption outcomes. • By matching a specific type of propagation strategy to the type of innovation, PIs will see greater adoption rates of their projects. • We're figuring out the strategies that work best for various innovations. • Finally, we’re putting what we learn together into a "How-To" guide that will make it easy for PIs to plan their approach to propagation. *Western Michigan University, †University of Iowa, ‡Texas A&M University Principal Investigators (PIs) Methods: National Web Survey of TUES PIs Program Directors (PDs) In January 2012 we visited the NSF in Arlington, VA , where we held focus groups with program directors in each discipline of the TUES program (biology, chemistry, computer science, engineering, geoscience, interdisciplinary, physics, and mathematics). We asked their for their views on successful propagation and what they’d like to see PIs do to spread their innovation to other institutions. Meetings were not recorded, but detailed notes were taken. Survey Questions From those notes, major themes relating to propagation were developed for each disciplinary group. These themes were then incorporated into a one-paragraph summary of the discussion that used the language of the PDs as much as possible. Finally, similar themes expressed by at least half of the PD groups were extracted and written in language that best represented the ideas expressed by each group. We sent these themes to the PDs as a member check and they agreed they represent their views. Question 7. “Briefly describe what successful dissemination of your educational innovation means to you? (Please be as specific as possible, e. g. , instead of ‘lots of users’, tell us how many users, i. e. , 1 user or 1, 000 users. )” Survey Questions 14 and 15: 14. Please rate the importance of the following possible barriers in disseminating your educational innovation (Likert scale from ‘not at all important’ to ‘extremely important’). • Dissemination of my innovation is not a priority to me • Lack of financial resources to support dissemination activities • Don’t know where or how to disseminate • Lack of peer reviewed journals focusing on educational innovation in my field • Technology changed so rapidly that educational innovation became out of date • Lack of peer reviewed conferences focusing on educational innovation in my field • Dissemination activities are not valued by my institution • Other (please specify) 15. Please make any comments you have about this survey or about dissemination of your educational innovation. Categories for what Successful Dissemination Means to PIs (themes from Question 7) Category Example Response 1. Dissemination activity (20%) “Presentations at regional and national conference, publications in widely distributed journals (focused both at practitioners and researchers), online or hardcopy publication of curriculum materials” 2. Result of using activity (55%) “ 25 other college faculty would use one of the instructional modules developed for his course. ” 3. Both an activity and the results (11%) “Publication of products – commercialization. Use by more than one class in one institution. Obviously this depends on the project” 4. Other/vague (12%) “The innovation is shared with a significant percentage of those educators who are interest in the particular field or area. ” 5. They did not know or haven’t disseminated (2%) “We have not disseminated our material yet, we are only at the developmental stage. ” Categories for barriers PIs encounter (themes from Question 14/15) Barrier category 1. Lack of support (51%) 2. Lack of NSF help (11%) 3. Other faculty barriers (16%) Example Response “Finding time to develop materials for dissemination has proved to be difficult as the institution is more concerned with pursuing additional funding opportunities. ” “I have always wondered why NSF does not have any tools and venues for dissemination of project results. ” “Funding for others to adopt the technology. A basic plate reader costs $5, 000, and without outside support, this is a significant barrier for other institutions to adopt its use. ” 4. No barriers encountered(8%) “The book is doing well. . . above the publisher's expectations. ” 5. Dissemination comment (14%) “Targeted dissemination is best instead of general. ” Based on the categories developed above, we present a visual model of how PIs think of propagation from the development of the innovation to its adoption by other faculty. The PI model is simple, linear, and as noted by many PIs, does not always lead to success. Independent Development Mass-Market Dissemination • PIs and PDs share a common goal, that faculty at other institutions adopt the innovation. Methods: Focus Groups with TUES Program Directors In fall 2011, Tront et al. carried out an online survey of 1285 TUES PIs. Tront et al. shared the survey data with us and we analyzed the open-ended question s using an emergent coding scheme. Questions 14/15 were treated as the same question, as PIs used both spaces to discuss barriers. Adoption Conclusions Themes from Program Directors Theme Description 1 PDs don’t think traditional methods (publishing a paper, presenting at conferences) are effective propagation tools. 2 PDs want to see more collaborations between PIs at different institutions, so more faculty have a stake in the innovation and it has more initial adopters. 3 PDs consider workshops to be the most effective propagation tool. Immersive multi-day workshops with follow up interaction with the PI lead to others adopting the innovation. 4 PDs noted that many well-propagated projects had been funded over a long period of time (10 years or more) through a series of related grants. 5 PDs observed that in some disciplines professional societies played key roles in propagation, and they feel there are more possibilities for PIs and professional societies to work together. • PIs have a simple linear idea of how to propagate their materials, using traditional “mass-market” channels to share their work with others. • PDs believe these methods don’t lead to sustained changes at multiple institutions. • PDs want to see PIs using interactive methods, enlisting help from professional societies and running immersive, multi-day workshops. • PDs also want to see propagation built into the project from the earliest stages by having collaborators at other institutions. Implications • There is a disconnect between PI and PD expectations. • PIs don’t feel prepared to take on a more active role in propagation, and the NSF offers little guidance. Based on themes found above, we present how PDs perceive successful propagation in a visual model easily comparable to the PI model. This model is more complex, although it shares the general phases of “development” followed by a “dissemination” phase, although both these phases involve more communication with other parties. Finally, both models conclude with “adoption. ” Interactive Development Interactive Dissemination • Both groups would benefit from thinking about propagation as occurring throughout the project. Check us out! Adoption www. drupalgardens. increasingtheimpact. org Acknowledgements This is based upon work supported by the National Science Foundation under Grant No. 1122446. We wish to thank Joseph Tront, Flora Mc. Martin, and Brandon Muramatsu for making their data available. We also wish to thank the NSF TUES program directors for their time and insights.