Ci MH Palette of Measures Evaluation Training Youth

Ci. MH Palette of Measures Evaluation Training: Youth Outcome Questionnaires (YOQs) Cricket Mitchell, Ph. D CIMH Evaluation Consultant

Palette of Measures Evaluation: What You Will Need (slide 1 of 2) • Palette of Measures Data Entry Shell v 2 (Excel file) – Developed by Ci. MH and customized for each participating agency – Holds all data for clients served thru the Palette of Measures project – Demographics, service delivery information, pre- and post- outcome measure data • Palette of Measures Data Dictionary v 2 (Word document) – A guide for using the associated data entry shell – Defines each column in the excel file 2

Palette of Measures Evaluation: What You Will Need (slide 2 of 2) • Outcome measures from the two-pronged approach – General Outcome Measure – Target-Specific Outcome Measure(s) – For example… • Youth Outcome Questionnaire © (YOQ) and Youth Outcome Questionnaire – Self Report © (YOQ-SR) – Provided to agencies participating in the Palette of Measures project under Ci. MH’s statewide license 3

Overview of Training • Brief Overview of Palette of Measures evaluation protocol • Youth Outcome Questionnaires© (YOQ & YOQ-SR): General outcome measures of youth mental health functioning – Administration – Scoring – Clinical Utility • Instructions for Palette of Measures data entry and data submissions – Data entry: YOQ/YOQ-SR – Data entry: Demographics & Services – Data submissions to Ci. MH 4

Brief Overview of Palette of Measures Evaluation Protocol 5

Outcome Assessment • Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning • Pre- and Post- a “dose” of treatment / an intervention interval – General measure of youth mental health functioning (e. g. , YOQ/YOQ-SR, CANS, Ohio Scales) – Target-specific measure linked to focus of treatment/intervention (e. g. , AQ, ECBI, PHQ-9, PTSDRI, RCADS) • Providers may choose to administer mid-course assessments as well – e. g. , at 3 -month intervals in usual care 6

A note about the use of standardized assessment measures… (slide 1 of 2) • Assessment is the beginning of developing a relationship with the child and family – Demonstrates a desire to know what the child and family are experiencing – By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

A note about the use of standardized assessment measures… (slide 2 of 2) • Using standardized assessment measures of functioning… – Assists in initial clinical impressions – Provides valuable information to guide treatment/interventions – Assesses sufficiency of treatment delivered – Demonstrates treatment-related improvements in child functioning 8

Youth Outcome Questionnaires© (YOQ & YOQ-SR) 9

Ci. MH YOQ & YOQ-SR Training • Information on the administration, scoring, and clinical utility of the YOQ & YOQ-SR was obtained from each measure’s respective Administration and Scoring Manual published by OQ Measures, LLC 10

YOQ & YOQ-SR Description • General measure of functioning • Assesses the global mental health functioning of children – Parent/caregiver report for children ages 4 -18 – Self-report for adolescents 12 -18 – Adult self-report is also available for 19+ • Available in English and Spanish 11

YOQ & YOQ-SR Description • 5 -15 minutes to complete • Sensitive to clinical change in short periods of time – Regardless of treatment modality, diagnosis, or discipline of the treating professional • Valid and reliable • Provided to partner agencies under Ci. MH’s statewide license – Partnering agencies sign a User’s Agreement and return it to CIMH 12

YOQ & YOQ-SR Description • Parallel versions of the same measure • 64 items • 5 -point Likert scale response options* – – – • Six Scale Scores – – – Intrapersonal Distress (ID) Somatic (S) Interpersonal Relations (IR) Social Problems (SP) Behavioral Dysfunction (BD) Critical Items (CI) Never or Almost Never Rarely Sometimes • Total Score Frequently Almost Always or Always *response values vary by item 13

YOQ & YOQ-SR Administration • Administer pre- and post- a dose of treatment, or an intervention interval – YOQ-SR completed by adolescent client – YOQ completed by mother/mother figure or father/father figure (when available) • Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes – Select parent/caregiver who spends most time with client 14

Let’s take a look at the YOQ-SR. . . 15

YOQ & YOQ-SR Administration • “… during the past 7 days. ” • Ask parents and youth to fill out the questionnaires as honestly as possible – Informants can easily be influenced by the attitude of the person administering the scale – Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall • Ask parents and youth to complete all items 16

YOQ & YOQ-SR Scoring • For each item, transfer the value corresponding to the selected response into the box at the right-hand side of the page titled, ‘For Office Use Only’ – Each item loads onto one of the six scales (e. g. , ID, SP, IR) – Note that some items have negative response option values • Sum the items in each scale on Side 1 – Add the numbers in all boxes under the heading ID and enter that subtotal into the ID box at the bottom of the page – Repeat for each scale • • Sum the items in each scale on Side 2 Transfer the subtotals from Side 1 to Side 2 Sum the subtotals to determine Scale Scores Sum the Scale Scores to determine Total Score – Note that it is possible to have negative values for scores 17

YOQ & YOQ-SR Scoring • Missing Data (items that are left blank) – If 5 or more items are missing, consider the questionnaire invalid – Substitute a mean item response for the missing item • Determine in which scale the missing item belongs • Add up the other items in that scale, and determine their average • Substitute the average score for the missing response 18

Clinical Utility of the YOQ & YOQ-SR • Total Score – Possible scores range from -16 to 240 – Clinical cutpoints • 46 or higher on the YOQ • 47 or higher on the YOQ-SR – Lower scores indicate more normative, nonclinical, aspects of general mental health functioning • Elevations on certain scales indicate areas of specific distress for the child/youth 19

Clinical Utility of the YOQ & YOQ-SR • Intrapersonal Distress (ID) Scale – Possible scores range from -4 to 68 – Clinical cutpoints • 16 or higher on the YOQ • 17 or higher on the YOQ-SR – Assesses the amount of emotional distress in the child/youth, including anxiety, depression, fearfulness, hopelessness, and self-harm – High scores indicate a considerable degree of intrapersonal distress in the child/youth 20

Clinical Utility of the YOQ & YOQ-SR • Somatic (S) Scale – Possible scores range from 0 to 32 – Clinical cutpoints • 5 or higher on the YOQ • 6 or higher on the YOQ-SR – Indicates change in somatic distress or physical complaints – High scores indicate the parent/caregiver is aware of, or the youth is experiencing, a high number of somatic symptoms; while low scores indicate either absence or unawareness of them 21

Clinical Utility of the YOQ & YOQ-SR • Interpersonal Relations (IR) Scale – Possible scores range from -6 to 34 – Clinical cutpoints • 4 or higher on the YOQ • 3 or higher on the YOQ-SR – Assesses issues relevant to the child/youth’s relationship with parents, other adults, and peers – High scores indicate significant interpersonal difficulty; while low scores reflect a cooperative, pleasant interpersonal demeanor 22

Clinical Utility of the YOQ & YOQ-SR • Social Problems (SP) Scale – Possible scores range from -2 to 68 – Clinical cutpoints • 3 or higher on the YOQ & YOQ-SR – Assesses problems that are socially related including aggression and delinquency – A feature of these items is that they are slow to change; whereas, content tapped by many of the other scales often changes over a period of time as a result of treatment intervention 23

Clinical Utility of the YOQ & YOQ-SR • Behavioral Dysfunction (BD) Scale – Possible scores range from -4 to 40 – Clinical cutpoints • 12 or higher on the YOQ • 11 or higher on the YOQ-SR – Assesses inattention, hyperactivity, impulsivity, concentration, ability to organize tasks, and ability to handle frustration 24

Clinical Utility of the YOQ & YOQ-SR • Critical Items (CI) Scale – Possible scores range from 0 to 36 – Clinical cutpoints • 5 or higher on the YOQ • 6 or higher on the YOQ-SR – Assesses areas such as paranoia, obsessive -compulsive behaviors, hallucination, delusions, suicide, mania, and eating disorders 25

Clinical Utility of the YOQ & YOQ-SR • Assesses a variety of specific areas of difficulty in child/youth mental health functioning • Assists in initial clinical impressions • Provides valuable information to guide treatment/interventions • Comparisons of pre/post scores reveal areas of clinical improvement as well as areas of potential unmet need 26

Summary of YOQ/YOQ-SR Score Ranges and Clinical Cutpoints YOQ/YOQ-SR Scale Range of possible scores Clinical Cutpoint for YOQ-SR Intrapersonal Distress (ID) -4 to 68 16 17 Somatic (S) 0 to 32 5 6 Interpersonal Relations (IR) -6 to 34 4 3 Social Problems (SP) -2 to 30 3 3 Behavioral Dysfunction (BD) -4 to 40 12 11 0 to 36 5 6 -16 to 240 46 47 Critical Items (CI) Total Score 27

Palette of Measures Data Entry and Data Submissions 28

Palette of Measures Data Entry Shell • There is a separate spreadsheet in the excel workbook (aka database) for each type of data: – Demographics & Services – Pre- General outcome measure – Post- General outcome measure – Pre- Target-specific measure(s) – Post- Target-specific outcome measure(s) • Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 29

Palette of Measures Data Entry Shell • In addition to the spreadsheets that hold data. . . – There is an Instructions spreadsheet • Basic data entry instructions • Contact information for T. A. (Cricket Mitchell) – There is a Data Lists spreadsheet at the end of the workbook that you will not use • Data Lists populate the pull-down menus in other spreadsheets 30

Palette of Measures Data Entry: YOQ/YOQ-SR Pre-YOQs PARENT/CAREGIVER REPORT: (ages 4 -18) Date of Client ID# Assessment YOQ (Scale Scores and Total Score) YOQ 1 -ID YOQ 1 -S YOQ 1 -IR YOQ 1 -SP YOQ 1 -BD YOQ 1 -CI YOQ 1 -Total … • There is a separate spreadsheet for Pre-YOQ data, Post-YOQ data, and Mid-YOQ data • In each spreadsheet, there is a separate field for each of the YOQ and YOQ-SR Scale Scores and Total Scores • In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields. 31

Palette of Measures Data Entry: YOQ/YOQ-SR Mid-YOQs Date of Assessment YOQ (Scale Scores and Total Score) YOQM- YOQM- YOQMClient ID# Assessment Interval ID S IR SP BD CI Total • For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-YOQs spreadsheet by selecting from the available pull-down menu • e. g. , 1 st mid-treatment assessment, 2 nd mid-treatment assessment 32

But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information. . . 33

Palette of Measures Data Entry: Demographics & Services (1 of 2) Client Information: Primary (DSM-IV code) Client ID# DOB Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID • Use a unique identifier for Client ID# • Categorical variables will have pull-down menus from which you’ll select an option (e. g. , gender, ethnicity, language) • Dates should be entered as xx/xx/xxxx • Axis I diagnoses s/b the numeric DSM-IV code • Therapist ID is optional 34

Palette of Measures Data Entry: Demographics & Services (2 of 2) Focus 1 of Treatment Focus 1 Date of First Session Date of Total # Completed Last of Sessions Focus 1? (if Focus 1 not completed) Completed (if Services not completed) Reason Services? Reason • Select Focus from pull-down menu (e. g. , anxiety, depression) • The shell will hold data for up to 4 foci, or treatment targets • Enter Date of First Session • The remaining fields are to be completed at the end of treatment targeting this particular focus (e. g. , Date of Last, # Sessions) • 2 levels of “Completed? ” • Treatment targeting this particular focus • Overall service delivery 35

Palette of Measures Data Submissions (slide 1 of 3) • Data submissions to Ci. MH will occur twice a year throughout the duration of the project – The end of each May (reflecting all clients served from the initiation of the project through the end of that April) – The end of each December (reflecting all clients served from the initiation of the project through the end of that November) – Note that this is the anticipated schedule; actual data submission dates may vary slightly • An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 36

Data Submissions (slide 2 of 3) • Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to Ci. MH: – Use You. Send. It, or another secure web-based transfer site, to submit data electronically • You. Send. It (www. You. Send. It. com) is a vendor that supports the secure transfer of electronic data (encrypted and passwordprotected) – Mail a password-protected CD to Ci. MH and submit the password separately (via email or phone) – Email an encrypted, password-protected file(s) to Ci. MH and submit the password separately (via email or phone) 37

Data Submissions (slide 3 of 3) • After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of clients served through the Palette of Measures project • Every effort is made to distribute reports within two months of each data submission – Aggregate and site/agency-specific reports 38

Questions 39

The End Contact Information • Cricket Mitchell, Ph. D • Email: cmitchell@cimh. org • Cell phone: 858 -220 -6355 40