The scientific premise of implementation science What generalizes

The scientific premise of implementation science: What generalizes? David A. Dzewaltowski, Ph. D. Professor and Community Chair for Activity, Nutrition, and Obesity UNMC and Buffett Early Childhood Institute

Agenda • Scientific premise as a contribution to generalizable science • Implementation science scientific premise • Generalizable product and/or generalizable principles • Linking scientific premise to standardized implementation protocols • Generalizable product • Generalizable principle

A few thoughts on innovation before we start • Henry Ford • “If I had asked people what they wanted, they would have said faster horses. ” • (I think this may be fake) • Akio Morita, Sony Founder • On focus group data not supporting making the Sony Walkman • ''The public does not know what is possible, but we do. '' • “We don’t ask consumers what they want. They don’t know. Instead we apply our brain power to what they need, and will want, and make sure we’re there, ready”

i. Phone Innovation quotes • Jon Rubinstein, Palm CEO and former Apple Vice President, i. Pod Division, September 27, 2005 • “Is there a toaster that also knows how to brew coffee? There is no such combined device, because it would not make anything better than an individual toaster or coffee machine. It works the same way with the i. Pod, the digital camera or mobile phone: it is important to have specialized devices. ” • Ed Colligan, Ex-Palm CEO, 16 Nov 2006 • “We've learned and struggled for a few years here figuring out how to make a decent phone. PC guys are not going to just figure this out. They're not going to just walk in. ” • Steve Ballmer, Microsoft CEO, 30 April 2007 • “There's no chance that the i. Phone is going to get any significant market share. No chance. It's a $500 subsidized item. They make a lot of money. But if you actually take a look at the 1. 3 billion phones that get sold, I'd prefer to have our software in 60% or 70% or 80% of them, than I would to have 2% or 3%, which is what Apple might get. ” • Ed Zander, Motorola CEO/Chairman, 10 May 2007 • “How do they deal with us? ”

The Innovation Problem • Grant writing as intervention for reviews • Convincing reviewers and funders to support your innovation • Imagine if the i. Phone or Sony Walkman had to go through the peer review process.

Scientific Premise • Significance • The scientific premise for an application is the research that is used to form the basis for the proposed research question(s). NIH expects applicants to describe the general strengths and weaknesses of the prior research being cited by the applicant as crucial to support the application. It is expected that this consideration of general strengths and weaknesses could include attention to the rigor of the previous experimental designs, as well as the incorporation of relevant biological variables and authentication of key resources. • Approach • Scientific Rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. This includes full transparency in reporting experimental details so that others may reproduce and extend the findings. NOT-OD-16 -011 Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications Release Date: October 9, 2015 (https: //grants. nih. gov/reproducibility/index. htm)

Scientific Premise • ” The scientific premise will be reviewed as part of the Significance criterion for research grant applications. Instructions for significance already include consideration of the importance of the problem, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change if the aims are achieved. Scientific premise includes a retrospective consideration of the foundation for the application, rather than a prospective analysis should the aims be achieved. ” July 20, 2017 - https: //grants. nih. gov/reproducibility/faqs. htm#4825

Scientific Premise • My interpretation of scientific premise is demonstrating that your study is based on and extends the generalizable body of knowledge. • Research is defined as the • "systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. Activities which meet this definition constitute research for purposes of this policy [45 CFR 46], whether or not they are conducted or supported under a program which is considered research for other purposes. For example, some demonstration and service programs may include research activities. " • From CITI IRB Training • Objective is to frame how your study is building on the existing body of evidence

What is generalizable implementation science knowledge? • The “ 7 -Ps” ? • Dissemination and implementation research objective is to understand how the following seven types of interventions can be delivered in and function effectively in varying contexts. • • programs (e. g. , cognitive behavioral therapy) practices [e. g. , “catch-em being good” principles (e. g. , prevention before treatment) procedures (e. g. , screen for depression) products (e. g. , m. Health app for exercise) pills (e. g. , Pr. EP to prevent HIV infection) policies (e. g. , limit prescriptions for narcotics). • Evidence-based products and/or principles? Brown, C. H. , Curran, G. , Palinkas, L. A. , Aarons, G. A. , Wells, K. B. , Jones, L. , . . . & Tabak, R. G. (2017). An overview of research and evaluation designs for dissemination and implementation. Annual Review of Public Health, 38, 1 -22.

What is generalizable implementation science knowledge? • I am not sure that ”local knowledge” leading to “generalizable knowledge” is an accurate frame for the problem. • Pipeline myth?

What is Generalizable Implementation Science Knowledge? • My hypothesis • The use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings (Glasgow et al, 2012). • Evaluate the product in the implementation context • evidence-based generalizable product with implementation practices • Contributing to theory • evidence-based generalizable implementation science principle

• Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Health. 2012; 102(7): 1274 -1281. • Khoury MJ, Gwinn M, Ioannidis JP. The emergence of translational epidemiology: from scientific discovery to population health impact. Am J Epidemiol. 2010; 172 (5): 517 -524. • Khoury MJ, Gwinn M, Yoon PW, Dowling N, Moore CA, Bradley L. The continuum of translation research in genomic medicine: how can we accelerate the appro- priate integration of human genome discoveries into health care and disease prevention? Genet Med. 2007; 9 (10): 665 --674.

Providing the scientific premise for an implementation science proposal Significance Section To meet concerns of reviewers who believe in the pipeline • What is the evidence-base for T-0 - No relevant gap (Paragraph on each) • What is the evidence-base for T-1 - No relevant gap (Paragraph on each) • What is the evidence-base for T-2 - No relevant gap (Paragraph on each) • From practice guidelines to health practice • What is the evidence-base for T-3 • Gap in knowledge (what is known and what is missing) • How will your project contribute to a generalizable product (solution) in the implementation context to fill a gap? . • How will your project contribute to generalizable implementation science principles? • From health practice to population health impact • What is the evidence-based for T-4 • Gap in knowledge (what is known and what is missing) • How will your project contribute to a generalizable product (solution) in the population health impact context to fill a gap? • How will your project contribute to generalizable implementation science principles?

Aims that Generalize (Scientific Premise) • Example Aims • Determine the impact of strategy on implementation behavior (practice patterns). • • Does the implementation intervention work? • Builds on existing literature and fills gap on need for strategy. Determine the processes that influence implementation behavior. • Why does the implementation intervention work? • Builds on existing literature and fill gaps on implementation science theory

Contributing Generable Knowledge to the Key Ingredients 1. The Quality Gap or Care Gap 2. The Evidence-Based intervention or policy to be implemented or disseminated 3. Conceptual model/theoretical framework 4. Stakeholder priorities and engagement in change 5. Setting’s readiness to adopt a new intervention 6. Implementation strategy/process 7. Team experience with the setting, treatment, implementation process, and review environment. • 8. Feasibility of Proposed Research Design and Methods • 9. Measurement and Analysis • 10. Policy environment will leverage, support, and sustain the change • •

Application of mediator/moderator vision to implementation science Bauman, A. E. , Sallis, J. F. , Dzewaltowski, D. A. , & Owen, N. (2002). Toward a better understanding of the influences on physical activity: the role of determinants, correlates, causal variables, mediators, moderators, and confounders. American journal of preventive medicine, 23(2), 5 -14.

What type of parent involvement was most effective in achieving desired outcomes?

Scientific Premise • Population Health Outcome • No Gap • Population Treatment (Behavior) Outcome • Gap – Did it work? • Testable hypothesis • Generalizable knowledge • Implementation Outcome • Gap – Why did it work? • Testable hypothesis • Generalizable knowledge • Product * implementation interaction Schlechter, C. R. , Rosenkranz, R. R. , Guagliano, J. M. , & Dzewaltowski, D. A. (2016). A systematic review of children's dietary interventions with parents as change agents: application of the RE-AIM framework. Preventive medicine, 91, 233 -243.

Roger’s Diffusion of Innovations theory • Perceived Attributes of Innovation • Nature of social system • Communication Channels • Type of Innovation-Decision • Extent of Change Agent's Promotion

Summary of my bias for scientific premise • More than program evaluation • Contributes to generalizable implementation science knowledge • Determine the impact of implementation strategy on behavior (practice patterns). • Determine the processes that influence implementation behavior.

Linking scientific premise to standardized implementation protocols • What is Standardized?

Generalizable form or function?

What generalizes? • Product • -i. Phone • Phone • Text • E-Mail • Kodak • Film company or image company? • Principle • i. Phone • Need for peer-to-peer communication • Kodak • Need for image retention and sharing

Linking scientific premise to generalizable implementation protocols • Did it work? • Product Evaluation • Why did work? • Form • Function

Generalizable protocol by form or function Hawe, P. , Shiell, A. , & Riley, T. (2004). Complex interventions: how “out of control” can a randomised controlled trial be? . BMJ: British Medical Journal, 328(7455), 1561.

Scientific Premise Aim 1 - Did the implementation strategy work? Aim 2 - Why did it work? • Community • Whole-of-Community Systems Intervention for Youth Population Physical Activity (Dzewaltowski – Next 5 Years) R 01 NIH-NCI • Schools • Evaluating a Systems-Based Health Intervention for Middle School Wellness (Chen et al, Next 2 years) R 21 NICHD • El Paso CATCH (Coleman et al. , Dzewaltowski, 2005) – American Heart Association • Healthy Youth Places (Dzewaltowski, et al. , 2009) – NIH • Healthy ONES (Coleman et al. , 2012) - USDA • Iowa SWITCH (Welk et al. , Dzewaltowski In Progress) – USDA • Kansas School Health (Dzewaltowski et al, In Progress) – Kansas Department of Health/CDC • Libraries • Russell, Dzewaltowski & Ryan (1999) • After School Programs • HOP’N After School (Dzewaltowski, et al. , 2010) – USDA • Project PLANT (Shoemaker et al. , Under Review) – USDA

Scientific Premise – Did it work? Why did it work? • Youth Clubs • Girl Scouts (Rosenkranz, et al. , Dzewaltowski 2010) – Sunflower Foundation/Kansas Health Foundation • Youth Sport • Guagliano, et al. Dzewaltowski (In Progress) • Summer Camps • Foster et al. Dzewaltowski (2010) • Hastmann et al. Dzewaltowski (2012) • Guagliano, et al (under review) • Child Care • Move and learn (Trost, et al. , Dzewaltowski, 2007) – United Methodist Health Ministry Fund • HOP’N Home Child Care (Hastmann, et al. , Dzewaltowski, In review) – United Methodist Health Fund

Dzewaltowski DA, Estabrooks PA, Welk G, Hill J, Milliken G, Karteroliotis K, Johnston JA. Healthy youth places: a randomized controlled trial to determine the effectiveness of facilitating adult and youth leaders to promote physical activity and fruit and vegetable consumption in middle schools. Health Educ Behav. 2009 Jun; 36(3): 583 -600.

1. Coleman, K. J. , Shordon, M. , Caparosa, S. L. , Pomichowski, M. E. , & Dzewaltowski, D. A. (2012). The Healthy Options for Nutrition Environments in Schools (Healthy ONES) group randomized trial: Using implementation models to change nutrition policy and environments in low income schools. International Journal of Behavioral Nutrition and Physical Activity, 9, 80.

Summary • Scientific premise as a contribution to generalizable science • Implementation science scientific premise • Generalizable product and/or generalizable principles • Linking scientific premise to standardized implementation protocols • Generalizable product/Generalizable principle Integration • Standardizing by function and allows for adaptation of form

Discussion • 5 to 7 in a Group Pick one project to discuss in detail • At least one person in the group should be a social and behavioral scientist. • Make a case for how this will fill a product gap • Program evaluation is necessarily not implementation science • Make a case for how this will fill a generalizable scientific impact gap. • 5 -7 in a group • 1. In a sentence or two at the most share with your group an evidence-based intervention that you would like to examine from a dissemination or implementation perspective. Pick one project to discuss in detail • 2. Make a case how their proposed project will fill a product gap and a generalizable scientific impact gap. • 3. Relate this back to how you would sell a review panel on your scientific premise.