Using the CANS for Evaluation Abram Rosenblatt Ph
Using the CANS for Evaluation Abram Rosenblatt, Ph. D.
Setting a Level Playing Field Practice Program System
Evaluating What? Effectiveness Efficiency Equity
Model for Assessing Child and Family Outcomes: The Dependent Variable Systematic Domains of Measurement Perspectives on Measurement Social Contexts of Measurement
Setting the Stage: A Framework System Goals Practice Level Program Level System Level Effectiveness The effect of a clinical intervention on outcomes The effect of program philosophy/culture on outcomes The effect of standards of care, service provision, and/or funding on outcomes Efficiency The effect of provider productivity on costs The effects of staffing choices, provider mix, and/or work hours on costs The effect of fiscal incentives, service system integration, and/or service mix on costs Equity The effect of provider choice and decision making on disparities The effect of program location and accessibility on disparities The effect of program mix and/or fiscal incentives on disparities
Effectiveness: Outcomes Clinical Status Functional Status Life Satisfaction Safety and Welfare
Model for Assessing System Reform Efforts: Linking the Independent and Dependent Variables Oindep = f(S, P, C) O = Outcomes for the individual at time “t”, domain “d”, context “c”, and perspective “p” S = System Level P = Program Level C = Clinical Level
The Measure and Perspective Matter 8
9 Perspective Matters
Some Perspectives and Measures Really Matter 10
Setting the Stage: Trends Systems of Care System Focus? Efficiency and Equity Impacts Effectiveness unclear Wraparound Mostly Program, also practice and system Fidelity Effectiveness…maybe Evidence Based Practice, also program Effectiveness
Assessment and Outcomes: Child and Adolescent Needs and Strengths (CANS) Clinician Based Rating Scale Parent and Child Perspectives? Mental Health Assessment Symptoms Diagnosis Real World Referents In Home In School Out of Trouble
System: CANS Overall Reliable Change Index (RCI)
System Toward Program: CANS Overall RCI
CANS Life Domain Functioning Domain RCI by Program
CANS Child Strengths Domain RCI by Program
CANS Caregiver Strengths and Needs Domain RCI by Program
CANS Child Behavior and Emotional Needs Domain RCI by Program
CANS Child Risk Behaviors Domain RCI by Program
20
21
Individual Level Practice Feedback
Individual Level (Practice) Feedback
WFI & CANS Outcomes 324 clients with two CANS Outcomes at least 6 months apart and a WFI at 6 months of service
WFI & Life Domain Functioning CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)
WFI & Child Strengths CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)
WFI & Caregiver Strengths/Needs CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)
WFI & Child Behav. & Emo. Needs CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)
WFI & Child Risk Behaviors CANS Improved or Declined CANS groups determined by Reliable Change Index (RCI)
Subjective and Objective = Quality of Life Objective: Living Environment (At Home) Subjective: Living Situation, Family on CANS Objective: School Attendance, expulsions, Suspensions, graduation, grade (In School) Subjective: School Behavior, School Achievement, School Attendance on CANS Objective: Juvenile Probation Involvement Subjective: Legal, Delinquency on CANS
Back to the bigger picture…
Remembering Why: Collecting Data and Perceived Value
CANS and Perceived Value Service Refinement Item Level Highly Reasonable and Understood Highly Relevant Convincing of Value Single Indicator (Not multiple Results) Understandable (Action or Not) Science Good Psychometrics Highly Dependent on Administration and Training Complimentary Measures and Indicators
OK, Finally, What about the Evaluation Design? Design is linked to Purpose Strength, Integrity and Social Ecology Program Evaluation Campbell, Stanley, Cook and the classics Public Health Approaches and Health Services Research Consistent Data (Indicator Approach, CABG) Risk Adjustment Catching Up WHERE DOES THE CANS FIT?
CANS and Research Design The Role of the CANS in Strength, Integrity and Social Ecology CANS as Outcome for Program Evaluation Using the CANS for Public Health Data Sets Risk Adjustment Outcomes Quality of Life Indicators
Using the CANS Across Levels and Goals System Goals Practice Level Program Level System Level Effectiveness CANS Item Level CANS Subscales CANS Overall CANS for Risk Adjust CANS with QOL Efficiency Provider choice of efficient interventions using CANS (Matching Intervention to need) Disparities based on Strengths and Needs All Needs Addressed CANS for Risk Adjust Appropriate/Most Efficient Level of Care CANS for Risk Adjust Program Matches Needs Program Addresses Access CANS to match program Mix to Needs at the System Access by Need Equity CANS for Level of Care Quality = Efficient
And So…the Pitfalls Not Using the CANS in Practice and CQI Ignoring Data Quality Pretending Training Does Not Matter “Just a form” Functional Fixedness Not understanding the purpose and goal Forgetting Why
- Slides: 37