The National Standards Project Phase 2 Diana Askings

The National Standards Project– Phase 2 Diana Askings Mc. Carty, B. S. University of Utah School Psychology Program February 24, 2016 Superheroes social skills training, Rethink Autism internet intervention, parent training, evidencebased practices classroom training, functional behavior assessment: An autism spectrum disorder, evidence-based practices training track for school psychologists US Office of Education Personnel Preparation Grant H 325 K 12306 William Jenson, Ph. D. , Elaine Clark, Ph. D. , Julia Hood Ph. D. , & John Davis, Ph. D.

Overview Introduction National Autism Center & National Standards Project Evidence-Based Practice Methodology Scientific Merit Rating Scale Intervention Effects Rating Scale Strength of Evidence Classification System Subclassification System Results NSP-Phase 1 NSP-Phase 2 Other Systematic Reviews—NPDC

Introduction Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social interactions/communication and by restricted, repetitive patterns of behavior ASD occurs in approximately 1 in every 68 births (CDC, 2014) Societal costs for each individual with ASD across the lifespan is estimated at $3. 2 million (Ganz, 2007) Intervention selection is complicated A large number of interventions are available Different levels of intervention support The research is often confusing to parents, educators, and service providers (National Standards Project-Phase 2, 2015)

The National Autism Center May Institute’s Center for the Promotion of Evidence-based Practice “…a nonprofit organization dedicated to disseminating evidence-based information about the treatment of autism spectrum disorder (ASD), promoting best practices, and offering comprehensive and reliable resources for families, practitioners, and communities. ” Training Public Policy Research (National Standards Project-Phase 2, 2015; http: //www. nationalautismcenter. org/)

What is the National Standards Project? The National Standards Project (NSP) is a comprehensive analysis of the interventions available for individuals with ASD Strength of evidence supporting interventions Age, diagnosis, and skills/behaviors targeted Limitations of research Evidence-based practice recommendations If you use effective intervention, the lifetime cost can be reduced by 65% (Jarbrink & Knapp, 2001) (National Standards Project-Phase 2, 2015)

Evidence-Based Practice (EBP) “…the integration of the best research evidence, professional judgment, and values and preferences of clients. ” (Sackett et al. , 2000) EBP has become the standard in the fields of medicine, psychology, education, allied health, and ASD Key Elements of EBP Professional Judgment Values and Preferences Capacity Research Findings (National Standards Project-Phase 2, 2015)

Scientific Merit Rating Scale (SMRS) Developed to objectively evaluate the strength of each study’s methodology 1) Research Design Experimental control; number of participants/groups; attrition; type of research design 2) Measurement of the DV Accurate and reliable data that represent the most direct/comprehensive target sample 3) Measurement of the IV Intervention fidelity; implementation accuracy; percentage/type of sessions for data collection 4) Participant Ascertainment Well-established diagnostic tools/procedures were used to determine participant inclusion; use of the DSM or ICD 5) Generalization & Maintenance Effects Objectively demonstrating intervention effects across time, settings, stimuli, responses, or persons Ratings from 0 (poor) 5 (strong) (National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015)

SMRS Scores Composite score Research Design (. 30) + Dependent Variable (. 25) + Participant Ascertainment (. 20) + Procedural Integrity (. 15) + Generalization (. 10) Score of 3, 4, or 5 Sufficient scientific rigor Score of 2 Initial evidence about intervention effects, but more rigorous research is needed Score of 0 or 1 Insufficient scientific rigor/evidence to suggest whether the intervention was or was not beneficial, ineffective, or harmful (National Standards Project-Phase 2, 2015)

Intervention Effects Rating Scale Beneficial Sufficient evidence to support favorable outcomes resulted from the intervention Ineffective Sufficient evidence to support favorable outcomes did not result from the intervention Unknown Not enough information to confidently determine the intervention effects Adverse (NSP-1) Sufficient evidence that the intervention was associated with harmful effects (National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015)

Strength of Evidence Classification System • Little/no evidence to draw firm conclusions about intervention effectiveness • Additional high quality studies must consistently show favorable outcomes to draw firm conclusions • Sufficient evidence to determine that an intervention produces favorable outcomes Unestablished Emerging Established (National Standards Project-Phase 2, 2015)

Strength of Evidence Classification System (National Standards Project-Phase 2, 2015)

Intervention Subclassification Phase 1 was broken down into Treatment Targets, Age Groups, and Diagnostic Groups Intervention Targets (14 targets in 2 categories) Skills Increased Academic, Communication, Higher Cognitive Functions, Interpersonal, Learning Readiness, Motor Skills, Personal Responsibility, Placement, Play, Self-Regulation Behaviors Decreased General Symptoms, Problem Behaviors, Restricted/Repetitive/Nonfunctional Patterns of Behaviors/Interests/Activity (RRN), Sensory or Emotional Regulation (SER) Age Groups 0 -21 years 22+ years (National Standards Project-Phase 2, 2015)

Subclassification Process Identify all studies associated with a given intervention Identify relevant variables in each of the studies Identify the SMRS Score and the Intervention Effects Ratings for each relevant variable Determine if there is evidence suggesting favorable outcomes for relevant variables Identify the quality, quantity, and consistency of research findings for relevant variables (National Standards Project-Phase 2, 2015)

Phase 1 Released in 2009 Research published between 1957 to 2007 7, 038 abstracts initially identified 775 studies after inclusion/exclusion criteria were applied Unrelated to ASD Unrelated to the treatment of ASD Not empirical articles 38 treatment categories identified by an expert panel Reliability IOA. 92 (National Standards Report, 2009)

Inclusionary & Exclusionary Criteria Inclusionary Published in peer-reviewed journals Diagnosed with ASD Autistic Disorder, Asperger’s Syndrome, and Pervasive Developmental Disorder-NOS The treatments could be implemented in school systems, early intervention, home, hospital, and/or community-based programs Exclusionary Rhett’s Disorder, Childhood Disintegrative Disorder, “At Risk” for ASD, & uncommon co-morbid conditions Only educational and behavioral treatments that target the core characteristics of ASD Non-empirical research Primary purpose to identify mediating or moderating variables Over the age of 22 Non-English publications (National Standards Report, 2009)

Results- “Established” 11 treatments had sufficient evidence to confidently determine that the treatment produces beneficial effects Antecedent Package (99) Behavioral Package (231) Comprehensive Behavioral Treatment for Young Children (22) Joint Attention Intervention (6) Modeling (50) Naturalistic Teaching Strategies (32) Peer Training Package (33) Pivotal Response Treatment (14) Schedules (12) Self-management (21) Story-based Intervention Package (21) (National Standards Report, 2009)

Results- “Emerging” 22 treatments had 1+ studies to suggest beneficial intervention effects, but additional high quality studies are needed to draw firm conclusions Augmentative and Alternative Communication Device (14) Cognitive Behavioral Intervention Package (3) Music Therapy (6) Peer-mediated Instructional Arrangement (11) Developmental Relationship-based Treatment (7) Exercise (4) Exposure Package (4) Imitation-based Interaction (6) Initiation Training (7) Language Training—Production (13) Language Training—Production & Understanding (7) Massage/Touch Therapy (2) Multi-component Package (10) Picture Exchange Communication System (13) Reductive Package (33) Scripting (6) Sign Instruction (11) Social Communication Intervention (5) Social Skills Package (16) Structured Teaching (4) Technology-based Treatment (19) Theory of Mind Training (4) (National Standards Report, 2009)

Results- “Unestablished” &“Ineffective/Harmful” 5 treatments had little to no evidence to draw firm conclusions about treatment effectiveness Academic Interventions (10) Auditory Integration Training (3) Facilitated Communication (5) Gluten- and Casein-Free Diet (2) Sensory Integrative Package (7) 0 identified studies indicated sufficient evidence that a treatment is ineffective or harmful Ethical reasons Not published (National Standards Report, 2009)

Limitations & Discussion Since Phase 1, the DSM criteria for ASD has changed Did not include individuals with ASD 22+ years old “At risk” or comorbid not included Treatment categories Only quantitative studies All articles were in English Did not evaluate “real world” vs. laboratory settings Treatment intensity not evaluated Literature review ended in September 2007 (National Standards Report, 2009)

Phase 2 Released April 2, 2015 To provide updated information on intervention effectiveness Reviewed studies between 2007 and February 2012 Included studies with adults 22+ years old Reviewed studies since 1987 27 articles identified Category Revision “Established” interventions are presented in more detail (National Standards Project-Phase 2, 2015)

Phase 2—Goals Primary goals 1) Identify peer-reviewed intervention outcome studies for individuals with ASD since the publication of NSP-1 2) Review interventions for individuals across the lifespan 3) Incorporate feedback received regarding NSP-1 categorization 4) To help parents, caregivers, educators and service providers understand how to integrate evidence-based interventions into a well-rounded, individualized educational/behavioral program (National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015)

Expert Panel Hanna C. Rue, Ph. D. , BCBA-D (Chair) Gail Mc. Gee, Ph. D. Grace Baranek, Ph. D. , OTR/L Samuel L. Odom, Ph. D. Jane I. Carlson, Ph. D. , BCBA-D Cathy L. Pratt, Ph. D. Alice Carter Ph. D. Patricia A. Prelock, Ph. D. , CCC-SLP Marjorie H. Charlop, Ph. D. , BCBA Robert F. Putnam, Ph. D. , BCBA-D Ronnie Detrich, Ph. D. Sally J. Rogers, Ph. D. Melanie Du. Bard, Ph. D. , BCBA-D Carol M. Schall, Ph. D. Glen Dunlap, Ph. D. Ilene S. Schwartz, Ph. D. , BCBA-D Peter Gerhardt, Ed. D. Mark D. Shriver, Ph. D. Lynne Gregory, Ph. D. Tristram H. Smith, Ph. D. Robert H. Horner, Ph. D. Brenda Smith Myles, Ph. D. Kara Anne Hume, Ph. D. Aubyn C. Stahmer, Ph. D. , BCBA-D James K. Luiselli, Ed. D. , ABPP, BCBA-D Pamela J. Wolfberg, Ph. D. Daniel Martin, Ph. D. , BCBA John G. Youngbauer, Ph. D. (National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015)

Inclusionary & Exclusionary Criteria Inclusionary Published in peer-reviewed journals Diagnosed with ASD Autistic Disorder, Asperger’s Syndrome, and Pervasive Developmental Disorder-NOS The interventions could be implemented in school systems, early intervention, home, hospital, vocational and/or community-based programs or clinic settings Exclusionary “autistic characteristics” or “suspicion of ASD”, & uncommon co-morbid conditions (except for individuals 22+) Only educational and behavioral treatments that target the core characteristics of ASD Non-empirical research Primary purpose to identify mediating or moderating variables Non-English publications (National Standards Project-Phase 2, 2015)

Presentation (National Standards Project-Phase 2, 2015)

(National Standards Project-Phase 2, 2015) Results- “Established” Under 22 years old 14 interventions had sufficient evidence to confidently determine that the intervention produces beneficial effects (**: NSP-1 “emerging”; underlined: Different from NSP-1) Behavioral Interventions (NSP-1=298 NSP-2=155) Peer Training Package (NSP-1=43 NSP-2=3) Cognitive Behavioral Intervention Package ** (NSP-1=3 NSP-2=10) Pivotal Response Treatment ® (NSP-1=11 NSP-2=6) Comprehensive Behavioral Treatment for Young Children (NSP-1=21 NSP-2=20) Schedules (NSP-1=11 NSP-2=2) Language Training—Production ** (NSP-1=10 NSP-2=2) Scripting ** (NSP-1=6 NSP-2=5) Modeling Self-management (NSP-1=51 NSP-2=28) (NSP-1=21 NSP-2=10) Natural Teaching Strategies Social Skills Package ** (NSP-1=27 NSP-2=3) (NSP-1=14 NSP-2=21) Parent Training (NSP-1=37* NSP-2=11) Story-based Intervention (NSP-1=21 NSP-2=15)

Results- “Emerging” Under 22 years old 18 interventions had 1+ studies to suggest beneficial intervention effects, but additional high quality studies are needed to draw firm conclusions Augmentative and Alternative Communication Devices Developmental Relationship-based Treatment Exercise Exposure Package Functional Communication Training Imitation-based Interaction Initiation Training Language Training—Production & Understanding Massage Therapy Multi-component Package Music Therapy Picture Exchange Communication System Reductive Package Sign Instruction Social Communication Intervention Structured Teaching Technology-based Intervention Theory of Mind Training (National Standards Project-Phase 2, 2015)

Results- “Unestablished” Under 22 years old 13 interventions had little to no evidence to draw firm conclusions about intervention effectiveness Animal-assisted Therapy SENSE Theatre Intervention Auditory Integration Training Sensory Intervention Package Concept Mapping Shock Therapy DIR/Floor Time ** Social Behavioral Learning Strategy Facilitated Communication Social Cognition Intervention Gluten-free/Casein-free Diet Social Thinking Intervention Movement-based Intervention (National Standards Project-Phase 2, 2015)

Results- 22+ years old Earliest study published in 1987 1 “established” intervention Behavioral Interventions (17) 1 “emerging” intervention Vocational Training Package 4 “unestablished” interventions Cognitive Behavioral Intervention Package Modeling Music Therapy Sensory Integration Package More research is needed (National Standards Project-Phase 2, 2015)

Intervention Selection A team of individuals should consider the unique needs of the individual as well as the environment that they live in (e. g. , family situation, community, cultural and ethnic background) Consider implementing “established” interventions Do not begin with “emerging” interventions but consider them if an “established” intervention is inappropriate or unsuccessful at producing positive outcomes. Only consider “unestablished” interventions if additional research produces supportive results (National Standards Project-Phase 2, 2015)

Limitations & Future Directions Only published, peer-reviewed research through January 2012 Lack of information on race/ethnicity, gender, or SES Intervention categories Remember–even “established” interventions will not be effective for all individuals with ASD More research is needed to identify variables that predict who will likely respond to intervention Interventions for “high-risk siblings” Level of functioning, intervention intensity, & social validity Qualitative studies Non-English studies Impact of cultural diversity (National Standards Project-Phase 2, 2015)

Other Systematic Reviews Centers for Medicare and Medicaid Services (CMS) http: //www. impaqint. com/work/project-reports/autism-spectrum-disordersservices-asds-final-report-environmental-scan Agency for Healthcare Research and Quality (AHRQ) http: //www. effectivehealthcare. ahrq. gov/ehc/products/544/1974/autismupdate-140923. pdf The National Professional Development Center on Autism Spectrum Disorder (NPDC) http: //autismpdc. fpg. unc. edu/evidence-based-practices (National Standards Project-Phase 2, 2015)

Evidence-Based Practices for Children, Youth, and Young Adults with ASD The goal of the NPDC was to promote the use of evidence-based practices (EBPs) for children and youth with ASD, birth to 22 years of age Comprehensive Treatment Models vs Focused Intervention Practices TEACCH, ESDM, UCLA Young Autism Program DTT, Pivotal Response Training, Prompting, Video Modeling 1 st review released in 2010 (1997 -2007) 24 interventions identified as EBP Very similar results to NSP-1 Current review 2014 (1990 -2011) (Wong et al. , 2014)

Methodology National set of external reviewers (N=159) Inclusion Individuals with ASD under 22 years old Co-morbid conditions Behavioral, developmental, or educational interventions and outcomes implemented at home, school, or the community Compared to no intervention or alternative intervention Experimental group, quasi-experimental group, or single-case design Literature search 29, 106 articles (ASD/Interventions) 456 articles Articles categorized EBP determined (Wong et al. , 2014)

Evidence-Based Practice Criteria At least 2 high quality experimental or quasiexperimental group design articles • Conducted by at least 2 different researchers or research groups At least 5 high quality single case design articles • Conducted by at least 3 different researchers or research groups • Having a total of at least 20 participants across studies A combination of at least 1 high quality experimental or quasi-experimental group design article and at least 3 high quality single case design articles • Conducted by at least 2 different research groups

(Wong et al. , 2014) Results 27 interventions identified as evidence-based practices Antecedent-based Intervention (0; 32) Pivotal Response Training (1; 7) Cognitive Behavioral Intervention (3; 1) Prompting (1; 32) Differential Reinforcement of Alternative, Incompatible, or Other Behavior (0; 26) Discrete Trial Teaching (0; 13) Exercise ** (3; 3) Extinction (0; 11) Functional Behavior Assessment (0; 10) Functional Communication Training ** (0; 12) Modeling (1; 4) Naturalistic Intervention (0; 10) Parent-implemented Intervention (8; 12) Peer-mediated Instruction and Intervention (0; 15) Picture Exchange Communication System ** (2; 4) Reinforcement (0; 43) Response Interruption/Redirection (0; 10) Scripting (1; 8) Self-management (0; 10) Social Narratives (0; 17) Social Skills Training (7; 8) Structured Play Group (2; 2) Task Analysis (0; 8) Technology-aided Instruction and Intervention ** (9; 11) Time Delay (0; 12) Video Modeling (1; 31) Visual Supports (0; 18)

Discussion 24 interventions identified were not classified as EBP, but had some support; more research is needed Foundational applied behavior analysis techniques have the most support Idiosyncratic Behavioral Intervention Package Many similarities to NSP Developing online modules (AFIRM) for each of the 27 identified practices Currently modules are available for 24 practices previously identified (Wong et al. , 2014)

References Centers for Disease Control and Prevention. (March 28, 2014). Morbidity and mortality weekly report. Retrieved February 16, 2016, from http: //www. cdc. gov/mmwr/preview/mmwrhtml/ss 6302 a 1. htm? s_cid=ss 6302 a 1_w. Ganz, M. L. (2007). The lifetime distribution of the incremental societal costs of autism. Archives of Pediatrics Adolescent Medicine, 161(4), 343 -349. Jarbrink, K. and Knapp, M. (2001). The economic impact of autism in Britain. Autism: the international journal of research and practice, 5(1), 7 -22. doi: 10. 1177/1362361301005001002 National Autism Center. (2009). National standards report. Retrieved January 31, 2016, from http: //www. nationalautismcenter. org/. National Autism Center. (2015). Findings and conclusions: National standards project, Phase 2. Retrieved January 31, 2016, from http: //www. nationalautismcenter. org/. Odom, S. L. , Collet-Klingenberg, L. , Rogers, S. J. , & Hatton, D. D. (2010). Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure, 54(4), 275 -282. Sackett, D. L. , Straus, S. E. , Richardson, W. S. , Rosenberg, W. , & Haynes, R. B. (2000). Evidence-based medicine: How to practice and teach EBM. London, UK: BMJ Books. Warren, Z. , Veenstra-Vander. Weele, J. , Stone, W. , Bruzek, J. L. , Nahmias, A. S. , Foss-Feig, J. H. , & Mc. Pheeters, M. (2011). Therapies for children with autism spectrum disorders. Comparative effectiveness review, Number 26. AHRQ publication No. 11 -EHC 029 -EF. Rockville, MD: Agency for Healthcare Research and Quality. Wong, C. , Odom, S. L. , Hume, K. , Cox, A. W. , Fettig, A. , Kucharczyk, S. , Brock, M. E. , Plavnick, J. B. , Fleury, V. P. , Schultz, T. R. (2014). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. Journal of Autism and Developmental Disorders. doi: 10. 1007/s 10803 -014 -2351 -z. Young, J. , Corea, C. , Kimani, J. , & Mandell, D. (2010). Autism spectrum disorders (ASDs) services: Final report on environmental scan (pp. 1 -59). Columbia, MD: IMPAQ International.

Thank you! NSP-1: http: //www. nationalautismcenter. org/reports/ NSP-2: http: //www. nationalautismcenter. org/national-standards-project/results-reports/