Building on the Measurement of CFIR Constructs in

Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here? Maria Fernandez, Ph. D on behalf of the CPCRN FQHC WG Investigators CPCRN Steering Committee - Virtual Spring Webinar April 16, 2014

National Association of Community Health Centers (NACHC) Primary Care Associations Partnership Committee CPCRN CHC Survey Align with CHCs’ missions CHC Survey Subgroup Health Care Reform Meaningful Use of EHR Patient-Centered Medical Home FQHC Workgroup Data Subgroup Community Health Centers (CHCs) Guided by real world health policy Qualitative Inquiry & Subgroup health care delivery landscapes

Developing Measures of Constructs Associated with EBI Implementation Purposes • Describe the survey development process and rationale • Present initial psychometric validation of CFIR Inner Setting measures • Discuss findings and what is next in our work assessing CFIR constructs (and PAR) related to EBI implementation Value • One of the first studies to measure several constructs from CFIR • Developed and tested measures that describe factors influencing implementation of EBAs in FQHCs • Sets the foundation for establishing causal pathways and developing interventions that target these factors

Consolidated Framework for Implementation Research (CFIR) Source: Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 2009; 4: 50. Note: Authors from the VA and University of Michigan, SPH, Department of Health Management and Policy

CFIR Constructs Patient Needs & Resources, External Policies & Incentives Outer Setting Knowledge & Beliefs about EBAs Characteristics of Individuals Available Resources, Leadership Engagement, Culture, Implementation Climate, Learning Climate Inner Setting Relative Advantage, Complexity Characteristics of Intervention Executing, Reflecting & Evaluating Process

Multiple Recruitment Strategies • Sites partnered with their state’s PCAs; PCAs emailed their CHCs • Sites recruited CHCs via email, telephone calls, or inperson meetings • Introductory email with online survey link; 4 reminder emails; in-person meeting (one site) • January - May 2013 • IRB approval at each site and coordinating center

Inner Setting Constructs • • Available Resources Culture Overall Culture Effort Culture Stress Implementation Climate Learning Climate Readiness for Change

Analysis plan • • • We conducted separate CFAs for each of the seven inner setting constructs. Multiple fit indices were used to evaluate CFA model fit (CFI >. 90, TLI >. 90, SRMR <. 08, and RMSEA <. 08). Cronbach’s alpha to evaluate internal consistency reliability. Correlation coefficients of each pair of scales to examine discriminant validity. Inter-rater reliability (ICC(1) & ICC(2)) and inter-rater agreement (r. WG(J)) statistics were computed to assess the reliability and validity of computing clinic- and systems-level means from the individual-level data.

A 35 A . 419 A 35 B . 546. 580 A 35 C C 20 A C 20 B C 20 C C 20 D . 708. 766 . 824 A 12 . 896 A 14 . 847 C 11 . 546. 830. 759. 462 C 05 Readiness for change 0. 968 . 872 C 13 0. 627 . 626. 741 A 13 Available Resources A 01 Implementation climate . 694. 646. 768. 727. 735 A 02 A 06 A 15 A 19 Learning climate 0. 514 0. 980 A 03 A 05 A 16 Rev A 07 A 22 . 663. 723. 556. 642. 742 A 08 . 674 A 09 . 719 A 10 . 731. 775 A 21 A 36 A 37 A 38 A 39 A 40 A 21 A 42 Rev A 43 A 44 Rev . 332. 438. 387. 594. 480. 612. 440. 581. 387 0. 945 Culture Inner Setting

Combined CFA results and fit statistics 1. Available Resources CFA Standardized Factor Loadingsa 0. 627 (0. 068) 2. Culture 0. 945 (0. 022) 3. Implementation Climate 0. 514 (0. 068) 4. Learning Climate 0. 980 (0. 024) 5. Readiness for Change 0. 968 (0. 017) Robust Std. Errors in parenthesis CFI = 0. 848 TLI = 0. 835; SRMR = 0. 079; RMSEA = 0. 065

Discriminant validity of the five dimensions of inner setting Scale Culture Available Learning Implementation Readiness Culture resources climate for change 1. 00 Available 0. 3641 1. 00 resources Learning climate 0. 6757 0. 4555 1. 00 Implementation 0. 2464 0. 6254 climate Readiness for 0. 6978 0. 5203 change 0. 3201 1. 00 0. 8114 0. 3758 1. 00 Correlations using average score for each scale *p <. 05

Clinic Level Inter-rater reliability and inter-rater agreement statistics Scale ICC(1) ICC(2) r. WG(J) Culture . 182* . 479 . 929 Available resources . 116* . 321 . 821 Learning climate . 178* . 476 . 785 Implementation climate Readiness for change . 219* . 504 . 828 . 169* . 462 . 634 Using average score for each scale *p <. 05

System level Inter-rater reliability and inter-rater agreement statistics Scale ICC(1) ICC(2) r. WG(J) Culture . 181* . 725 . 925 Available resources . 114* . 542 . 867 Learning climate . 150* . 679 . 774 Implementation climate Readiness for change . 292* . 792 . 845 . 106* . 558 . 592 Using average score for each scale *p <. 05

Conclusions Our measures have structural validity Our measures have reliability Our measures have discriminant validity Our measures can be aggregated to the clinic level • Clinics within systems have more similar values for inner-setting domain constructs than do clinics in different systems • We fit a second order “inner setting” factor • •

PRACTICE ADAPTIVE RESERVE AND PATIENT CENTERED MEDICAL HOME BEST PRACTICES AT COMMUNITY HEALTH CENTER CLINICS IN SEVEN STATES

Adjusted Logistic Regression Frequency of PCMH Best Practices and PAR Scores PCMH CRC Screening Best Practices Dichotomized Score (6 -8 vs. 0 -5) Respondent reported performing PCMH best practices “usually” or “always” PAR Frequency of PCMH Best Practices (6 -8 vs. 0 -5) OR 95% CI 0. 08 – 1. 00 5. 49 2. 31, 13. 06 0. 60 - <0. 80 2. 23 1. 11, 4. 47 0. 00 - <0. 60 Referent Adjusted for state, age, job type, years worked at the clinic, hours worked each week National Demonstration Project - Highly-motivated practices w/ significant capability for change • Mean baseline PAR score 0. 69 (s. d. 0. 35) • Post intervention PAR score increased to 0. 74

Discussion • What have we learned about some of the measurement challenges in FQHCs? • Where do we go from here in terms of measurement? • Which constructs need more conceptual and measurement work? • What additional work is needed in terms of reliability and validity assessment?

Discussion Continued • Where do we go from here in EBI implementation in FHQCs? • What opportunities exist? • How can the measurement work be used in EBI implementation in FQHCs?

How FQHC WG can move forward? • How could this group move forward on what’s next? • What resources, partners, and preliminary work is needed to pursue what’s next?