Using the CANS for Evaluation Abram Rosenblatt Ph

Using the CANS for Evaluation Abram Rosenblatt, Ph. D.

Setting a Level Playing Field Practice Program System

Evaluating What? Effectiveness Efficiency Equity

Model for Assessing Child and Family Outcomes: The Dependent Variable Systematic Domains of Measurement Perspectives on Measurement Social Contexts of Measurement

Setting the Stage: A Framework System Goals Practice Level Program Level System Level Effectiveness The effect of a clinical intervention on outcomes The effect of program philosophy/culture on outcomes The effect of standards of care, service provision, and/or funding on outcomes Efficiency The effect of provider productivity on costs The effects of staffing choices, provider mix, and/or work hours on costs The effect of fiscal incentives, service system integration, and/or service mix on costs Equity The effect of provider choice and decision making on disparities The effect of program location and accessibility on disparities The effect of program mix and/or fiscal incentives on disparities

Effectiveness: Outcomes Clinical Status Functional Status Life Satisfaction Safety and Welfare

Model for Assessing System Reform Efforts: Linking the Independent and Dependent Variables Oindep = f(S, P, C) O = Outcomes for the individual at time “t”, domain “d”, context “c”, and perspective “p” S = System Level P = Program Level C = Clinical Level

The Measure and Perspective Matter 8

9 Perspective Matters

Some Perspectives and Measures Really Matter 10

Setting the Stage: Trends Systems of Care System Focus? Efficiency and Equity Impacts Effectiveness unclear Wraparound Mostly Program, also practice and system Fidelity Effectiveness…maybe Evidence Based Practice, also program Effectiveness

Assessment and Outcomes: Child and Adolescent Needs and Strengths (CANS) Clinician Based Rating Scale Parent and Child Perspectives? Mental Health Assessment Symptoms Diagnosis Real World Referents In Home In School Out of Trouble

System: CANS Overall Reliable Change Index (RCI)

System Toward Program: CANS Overall RCI

CANS Life Domain Functioning Domain RCI by Program

CANS Child Strengths Domain RCI by Program

CANS Caregiver Strengths and Needs Domain RCI by Program

CANS Child Behavior and Emotional Needs Domain RCI by Program

CANS Child Risk Behaviors Domain RCI by Program

20

21

Individual Level Practice Feedback

Individual Level (Practice) Feedback

WFI & CANS Outcomes 324 clients with two CANS Outcomes at least 6 months apart and a WFI at 6 months of service

WFI & Life Domain Functioning CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)

WFI & Child Strengths CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)

WFI & Caregiver Strengths/Needs CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)

WFI & Child Behav. & Emo. Needs CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)

WFI & Child Risk Behaviors CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)

Subjective and Objective = Quality of Life Objective: Living Environment (At Home) Subjective: Living Situation, Family on CANS Objective: School Attendance, expulsions, Suspensions, graduation, grade (In School) Subjective: School Behavior, School Achievement, School Attendance on CANS Objective: Juvenile Probation Involvement Subjective: Legal, Delinquency on CANS

Back to the bigger picture…

Remembering Why: Collecting Data and Perceived Value

CANS and Perceived Value Service Refinement Item Level Highly Reasonable and Understood Highly Relevant Convincing of Value Single Indicator (Not multiple Results) Understandable (Action or Not) Science Good Psychometrics Highly Dependent on Administration and Training Complimentary Measures and Indicators

OK, Finally, What about the Evaluation Design? Design is linked to Purpose Strength, Integrity and Social Ecology Program Evaluation Campbell, Stanley, Cook and the classics Public Health Approaches and Health Services Research Consistent Data (Indicator Approach, CABG) Risk Adjustment Catching Up WHERE DOES THE CANS FIT?

CANS and Research Design The Role of the CANS in Strength, Integrity and Social Ecology CANS as Outcome for Program Evaluation Using the CANS for Public Health Data Sets Risk Adjustment Outcomes Quality of Life Indicators

Using the CANS Across Levels and Goals System Goals Practice Level Program Level System Level Effectiveness CANS Item Level CANS Subscales CANS Overall CANS for Risk Adjust CANS with QOL Efficiency Provider choice of efficient interventions using CANS (Matching Intervention to need) Disparities based on Strengths and Needs All Needs Addressed CANS for Risk Adjust Appropriate/Most Efficient Level of Care CANS for Risk Adjust Program Matches Needs Program Addresses Access CANS to match program Mix to Needs at the System Access by Need Equity CANS for Level of Care Quality = Efficient

And So…the Pitfalls Not Using the CANS in Practice and CQI Ignoring Data Quality Pretending Training Does Not Matter “Just a form” Functional Fixedness Not understanding the purpose and goal Forgetting Why