Ci MH Palette of Measures Evaluation Training Eyberg

Ci. MH Palette of Measures Evaluation Training: Eyberg Child Behavior Inventory™ (ECBI) Cricket Mitchell, Ph. D CIMH Evaluation Consultant

Palette of Measures Evaluation: What You Will Need (slide 1 of 2) • Palette of Measures Data Entry Shell v 2 (Excel file) – Developed by Ci. MH and customized for each participating agency – Holds all data for clients served thru the Palette of Measures project – Demographics, service delivery information, pre- and post- outcome measure data • Palette of Measures Data Dictionary v 2 (Word document) – A guide for using the associated data entry shell – Defines each column in the excel file 2

Palette of Measures Evaluation: What You Will Need (slide 2 of 2) • Outcome measures from the two-pronged approach – General Outcome Measure – Target-Specific Outcome Measure(s) – For example… • Eyberg Child Behavior Inventory™ (ECBI) – Trade-marked and published by Psychological Assessment Resources, Inc. (PAR) • Must be purchased by interested Palette of Measures project participating agencies 3

Overview of Training • Brief Overview of Palette of Measures evaluation protocol • Eyberg Child Behavior Inventory (ECBI): Target-specific measure when the focus of treatment is disruptive behaviors + Sutter-Eyberg Student Behavior Inventory (SESBI) – Administration – Scoring – Clinical Utility • Instructions for Palette of Measures data entry and data submissions – Data entry: ECBI & SESBI – Data entry: Demographics & Services – Data submissions to Ci. MH 4

Brief Overview of Palette of Measures Evaluation Protocol 5

Outcome Assessment • Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning • Pre- and Post- a “dose” of treatment / an intervention interval – General measure of youth mental health functioning (e. g. , YOQ/YOQ-SR, CANS, Ohio Scales) – Target-specific measure linked to focus of treatment/intervention (e. g. , AQ, ECBI, PHQ-9, PTSDRI, RCADS) • Providers may choose to administer mid-course assessments as well – e. g. , at 3 -month intervals in usual care 6

A note about the use of standardized assessment measures… (slide 1 of 2) • Assessment is the beginning of developing a relationship with the child and family – Demonstrates a desire to know what the child and family are experiencing – By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

A note about the use of standardized assessment measures… (slide 2 of 2) • Using standardized assessment measures of functioning… – Assists in initial clinical impressions – Provides valuable information to guide treatment/interventions – Assesses sufficiency of treatment delivered – Demonstrates treatment-related improvements in child functioning 8

Eyberg Child Behavior Inventory. TM (ECBI™) and Sutter-Eyberg Student Behavior Inventory. TM (SESBI-RTM) 9

Ci. MH ECBI Training • Information on the administration, scoring, and clinical utility of the ECBI & SESBI was obtained from the measures’ Professional Manual, written by Sheila Eyberg and Donna Pincus, published by Psychological Assessment Resources, Inc. (PAR) 10

ECBI™ and SESBI-R™ Description ECBI™ and SESBI-R™ Descripti • Target-specific measure when the treatment focus is disruptive behavior • Measures the current frequency and severity of child disruptive behavior problems – Parent/caregiver report for children ages 2 -16 – Teacher report for children ages 2 -16 • Available in English and Spanish 11

ECBI™ and SESBI-R™ Description ECBI™ and SESBI-R™ Descripti • • Written at a 6 th grade reading level 5 -10 minutes to complete Valid and reliable Trade-marked and published by Psychological Assessment Resources, Inc. (PAR) – Must be purchased by interested Palette of Measures project participating agencies – After purchase of the manual ($52), the cost per use ~$1. 60 – http: //www 4. parinc. com/Products/Product. aspx? Produ ct. ID=ECBI 12

ECBI™ and SESBI-R™ Description and SESBI-R™ Descripti • ECBI: 36 items • SESBI-R: 38 items • 7 -point Likert scale response options – – – Never (1) Seldom (2 -3) Sometimes (4) Often (5 -6) Always (7) • Two Scales – Intensity • Frequency with which the child exhibits the behaviors – Problem • Extent to which the parent or teacher considers the behaviors to be a problem for him/herself 13

Example: Items from the ECBI™ • • Refuses to do chores when asked Argues with parents about rules Has temper tantrums Hits parents Lies Physically fights with friends own age Is overactive or restless 14

Example: Items from the SESBI-R™ • • Teases or provokes other students Does not obey school rules on his/her own Demands teacher attention Acts bossy with other students Sasses teacher Physically fights with other students Has difficulty staying seated 15

ECBI™ and SESBI-R™ Administration • Administer pre- and post- a dose of treatment, or an intervention interval – ECBI completed by mother/mother figure or father/father figure (when available) • Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes – Select parent/caregiver who spends most time with client – SESBI can be completed by a teacher or child care worker • Residential treatment settings can have staff complete this measure 16

Let’s take a look at the ECBI. . . (not distributed due to copyright laws) 17

ECBI™ and SESBI-R™ Administration • Approximately 5 minutes to complete • Parents and/or teachers complete independently – Written at a 6 th grade reading level – Can be read aloud if necessary/desired • There are no “right” or “wrong” answers; encourage honest responses – Informants can easily be influenced by the attitude of the person administering the scale – Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall • Ask parents (and teachers) to complete all items 18

ECBI™ and SESBI-R™ Scoring • Calculate the Intensity scale Page 1 subtotal by summing the marked scores and recording this sum in the space provided • Tally the number of Problem responses for which a “Yes” rating was provided, and record this total in the Page 1 subtotal for the Problem scale • Transfer Page 1 subtotals to appropriate row on Page 2 • Calculate Page 2 subtotals in the same manner as Page 1 • Each scale total raw score is obtained by summing Page 1 and Page 2 subtotals 19

ECBI™ and SESBI-R™ Scoring: Missing Data • If more than four items missing from Intensity scale, it is invalid and cannot be scored – With three or fewer items missing, circle 1 (Never) and calculate raw score • If more than four items missing from Problem scale, it is invalid and cannot be scored – With three or fewer items missing, circle the “No” response and calculate raw score 20

ECBI™ and SESBI-R™ Scoring • ECBI Raw Scores (36 items): – Intensity Scale Raw Scores Range from 36 -252 – Problem Scale Raw Scores Range from 0 -36 • SESBI-R Raw Scores (38 items): – Intensity Scale Raw Scores Range from 38 -266 – Problem Scale Raw Scores Range from 0 -38 21

ECBI™ and SESBI-R™ Scoring • Convert Raw scores to T scores – T scores are standardized scores with a mean of 50 and a standard deviation of 10 – Intensity scale T score conversions are in Appendix C (ECBI) and Appendix E (SESBI-R) • Pg 53 and pg 57 of the ECBI/SESBI-R Professional Manual – Problem scale T score conversions are in Appendix D (ECBI) and Appendix F (SEBI-R) • Pg 55 and pg 59 of the ECBI/SESBI-R Professional Manual 22

Clinical Utility of the ECBI™ and SESBI-R™ • Higher scores indicate a greater level, or greater frequency, of conduct-disordered behavior (Intensity) and a greater impact on the parent or teacher (Problem) 23

Clinical Utility of the ECBI™ and SESBI-R™ • T scores > 60 are clinically significant – ECBI Intensity Raw Score > 131 – ECBI Problem Raw Score > 15 – SESBI-R Intensity Raw Score > 151 – SESBI-R Problem Raw Score > 19 • T scores < 60 are within the normal range 24

Clinical Utility of the ECBI™ • Parenting education workshops and parent training interventions can be helpful even to those who do not score in the clinical range – Behavior problems of “normal” children are often annoying to parents and others – ECBI is sensitive to change even when initial administration is not in clinical range 25

Clinical Utility of the ECBI™ • Discrepancies between scores on the ECBI Intensity and Problem scales can occur; however, it happens relatively infrequently… 26

Clinical Utility of the ECBI™ • …When the Intensity score is low and the Problem score is high, interpret cautiously – Parent could have a low tolerance for normal misbehaviors of child – Parent may have unrealistically high expectations for child conduct and an authoritarian parenting style – Parent may have limited understanding of child behavior and the kinds of behavior problems that are developmentally appropriate – Parent may be attempting to cope with chaotic and difficult circumstances and be overwhelmed by the stressors of child rearing 27

Clinical Utility of the ECBI™ • …When the Intensity score is high and the Problem score is low – Parent may have a high tolerance for misbehavior – Parent may hold well-intentioned beliefs about permissive parenting that are mismatched to the temperament of his/her child – Parent may be detached from the child either because of the child’s problems or because of characteristics in the parent’s own personality 28

Clinical Utility of the SESBI-R™ • The SESBI-R is a newer measure with much less empirical study than the ECBI – While cutoff scores are provided to estimate the severity of child behavior problems at school, there is a wide variability both within and between geographic locations – Use caution in interpretation, and consider prevailing local customs and norms as needed 29

Clinical Utility of the ECBI™ and SESBI-R™ • Comparisons of pre/post scores reveal specific areas of treatment-related improvement in child disruptive behavior – e. g. , • Does the frequency of disruptive child behavior decrease substantially? • Does the extent to which disruptive child behaviors are problematic to parents and/or teachers improve? 30

Palette of Measures Data Entry and Data Submissions 31

Palette of Measures Data Entry Shell • There is a separate spreadsheet in the excel workbook (aka database) for each type of data: – Demographics & Services – Pre- General outcome measure – Post- General outcome measure – Pre- Target-specific measure(s) – Post- Target-specific outcome measure(s) • Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 32

Palette of Measures Data Entry Shell • In addition to the spreadsheets that hold data. . . – There is an Instructions spreadsheet • Basic data entry instructions • Contact information for T. A. (Cricket Mitchell) – There is a Data Lists spreadsheet at the end of the workbook that you will not use • Data Lists populate the pull-down menus in other spreadsheets 33

Palette of Measures Data Entry: ECBI Pre-ECBI TEACHER REPORT (SESBI): IF PARENT REPORT (ECBI): AGES 2 -16 AVAILABLE Intensity Scale Problem Scale EEEESSSSClient ID# Assessment Int. Raw 1 Int. T 1 Prob. Raw 1 Prob. T 1 Date of • There is a separate spreadsheet for Pre-ECBI&SESBI data, Post-ECBI&SESBI data, and Mid-ECBI&SESBI data • In each spreadsheet, there is a separate field for Intensity and Problem Scales, Raw and T Scores, for each measure • In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. 34

Palette of Measures Data Entry: ECBI Mid-ECBI Date of Client ID# Assessment PARENT REPORT (ECBI): AGES 2 - TEACHER REPORT (SESBI): IF 16 AVAILABLE Assessment Intensity Scale Problem Scale Interval EEEESInt. Raw Int. T Prob. Raw Prob. T Int. Raw M M M SSSInt. T Prob. Raw Prob. T M M M • For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-ECBI&SESBI spreadsheet by selecting from the available pull-down menu • e. g. , 1 st mid-treatment assessment, 2 nd mid-treatment assessment 35

But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information. . . 36

Palette of Measures Data Entry: Demographics & Services (1 of 2) Client Information: Client ID# DOB Primary (DSM-IV code) Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID • Use a unique identifier for Client ID# • Categorical variables will have pull-down menus from which you’ll select an option (e. g. , gender, ethnicity, language) • Dates should be entered as xx/xx/xxxx • Axis I diagnoses s/b the numeric DSM-IV code • Therapist ID is optional 37

Palette of Measures Data Entry: Demographics & Services (2 of 2) Focus 1 of Treatment Date of First Session Focus 1 Date of Total # Completed Last of Sessions Focus 1? (if Focus 1 not completed) Completed (if Services not completed) Reason Services? Reason • Select Focus from pull-down menu (e. g. , anxiety, depression) • The shell will hold data for up to 4 foci, or treatment targets • Enter Date of First Session • The remaining fields are to be completed at the end of treatment targeting this particular focus (e. g. , Date of Last, # Sessions) • 2 levels of “Completed? ” • Treatment targeting this particular focus • Overall service delivery 38

Palette of Measures Data Submissions (slide 1 of 3) • Data submissions to Ci. MH will occur twice a year throughout the duration of the project – The end of each May (reflecting all clients served from the initiation of the project through the end of that April) – The end of each December (reflecting all clients served from the initiation of the project through the end of that November) – Note that this is the anticipated schedule; actual data submission dates may vary slightly • An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 39

Data Submissions (slide 2 of 3) • Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to Ci. MH: – Use You. Send. It, or another secure web-based transfer site, to submit data electronically • You. Send. It (www. You. Send. It. com) is a vendor that supports the secure transfer of electronic data (encrypted and passwordprotected) – Mail a password-protected CD to Ci. MH and submit the password separately (via email or phone) – Email an encrypted, password-protected file(s) to Ci. MH and submit the password separately (via email or phone) 40

Data Submissions (slide 3 of 3) • After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of clients served through the Palette of Measures project • Every effort is made to distribute reports within two months of each data submission – Aggregate and site/agency-specific reports 41

Questions 42

The End Contact Information • Cricket Mitchell, Ph. D • Email: [email protected] org • Cell phone: 858 -220 -6355 43