Standardized Scales Standardization n Use of identical procedures

Standardized Scales

Standardization n Use of identical procedures to collect, score, interpret, and report results of a measure n Assures that differences over time or among different people are due to the variable being measured and not to different measurement procedures

What are Standardized Scales? n Set of uniform procedures to collect, score, interpret, and report numerical results n Usually have norms and empirical evidence of reliability and validity n Typically include multiple items aggregated into one or more composite scores n Frequently used to measure constructs

Construct n Complex concept (e. g. , intelligence, well- being, depression) n Inferred or derived from a set of interrelated attributes (e. g. , behaviors, experiences, subjective states, attitudes) of people, objects, or events n Typically embedded in a theory n Oftentimes not directly observable but measured using multiple indicators

Evaluating and Selecting Standardized Scales n Purpose n Reference populations and normative groups n Reliability n Validity n Practical considerations

Purpose n Identify whether or not a client has a significant problem n Measure and monitor your client’s outcomes to determine if your client is making satisfactory progress

Reference Population of people for which a measure is intended and from which a normative group is sampled and norms are created

Normative Group n Representative sample of a reference population, used to estimate norms for that population and, more generally, used to develop and test standardized measures n Also known as a “standardization group” or “standardization sample” Population Sample

Reliability n Internal consistency reliability (coefficient alpha) (most important) n Interrater reliability (sometimes) n Test-retest reliability

Validity n Face n Content n Criterion n Construct n Sensitivity to change especially important

Practical Considerations n Time n Effort n Training n Cost n Availability n Acceptability (e. g. , clients, practitioners, etc. )

Decisions, Decisions… n Who n Where n When n How often to collect outcome data

Who n Client n Practitioner n Relevant others n Independent evaluators

Where and When n Private, quiet, physically comfortable location n Complete at about the same time and under the same conditions on a regular basis

How Often n Regular, frequent, pre-designated intervals n Often enough to detect significant changes in the problem, but not so often that it becomes problematic n In general about once per week

Engage and Prepare Clients n Be certain the client understands and accepts the value and purpose of monitoring progress n Discuss confidentiality n Present measures with confidence n Don’t ask for info the client can’t provide

Engage and Prepare Clients (cont’d) n Be sure the client is prepared n Be careful how you respond to information n Use the information that is collected

Administering, Scoring, and Interpreting Standardized Scales n Score, scoring formula, composite score n Unidimensional and multidimensional scales n Cut scores n Reverse-worded items n Reliable change, reliable improvement, reliable deterioration n Clinically significant improvement n Expected treatment response

Score n Generic term for a number derived from a measure that represents the quantity or amount of an attribute or observation (e. g. , number of times a behavior is observed, value obtained from a standardized scale) n Interpret in context of all available quantitative and qualitative information

Scoring Procedure by which data from a measure are used to produce a score (e. g. , number of times a behavior occurs or value on a standardized scale) or category (e. g. , diagnostic category)

Scoring Formula A mathematical rule by which data from a measure are used to produce a score (e. g. , sum or average of responses to items on a multi-item standardized scale) Item 1 Item 2 Item 3 Score

Composite Score that combines results from two or more related items or other measures using a specified formula (e. g. percentage of items answered correctly on a statistics test) Item 2 Item 1 Item 3 Score

Unidimensional Scale that measures a single attribute or construct (e. g. , depression). (Contrast with multidimensional scale. )

Multidimensional Scale that measures two or more distinct but related attributes or constructs, and measures of the different attributes or constructs are referred to as “subscales” Problems & Symptoms Subjective Well-Being Social Functioning Global Distress

Cut Scores n Specific predetermined numerical values along a continuum of scores n n Used to separate people into categories with distinct substantive interpretations (e. g. , clinically depressed or not) Used to make decisions (provide treatment for depression or not) n Only as good as the normative sample(s) on which it is derived n Interpret in context of all available quantitative and qualitative information

Reverse-Worded Item for which smaller numbers indicate a higher score on the measured variable because the item is worded to mean the opposite of the measured variable

Reliable Change in a score from one time to another that is more than expected just from random measurement error n Clinical significance. xls

Reliable Improvement in a score from one time to another that is more than expected just from random measurement error

Reliable Deterioration in a score from one time to another that is more than expected just from random measurement error

Clinically Significant Improvement Change that occurs when a client’s measured functioning on a standardized scale is: n n n In the dysfunctional range before intervention (e. g. , greater than 5 on the QIDS-SR) In the functional range after intervention (e. g. , 5 or below on the QIDS-SR) Change is reliable

Clinically Significant Improvement (cont’d) n Interpret in context of all available quantitative and qualitative information n Does not guarantee a meaningful change in a client’s real-world functioning or quality of life n Only as good as the normative sample(s) on which it is derived n Does not speak to the question of whether it was your intervention or something else that caused the change

Expected Treatment Response n Session-by-session progress is determined in comparison to normative data from ongoing responses to treatment of thousands of clients n Feedback used in real time to monitor client progress and modify services as needed to reduce treatment failures and increase overall effectiveness

Global Rating Single rating based on a rater’s integration of information about numerous factors (e. g. , global rating of change, improvement, or social functioning)

Single-Item Global Standardized Scales n Global Assessment of Functioning (GAF) n Children’s Global Assessment Schedule (CGAS) n Social and Occupational Functioning Assessment Scale (SOFAS) n Global Assessment of Relational Functioning (GARF)

Potential Advantages of Standardized Scales n Pretested for reliability and validity n Structured, so information less likely to be missed n Can be used to compare individual functioning to normative group functioning n Can be efficient and simple to use

Cautions in the Use of Standardized Scales n May not measure concept suggested by scale name n Different measures of the same concept may not be equivalent n Sometimes limited information about reliability and validity n Concepts as measured may not be completely relevant to individual clients

Resources n Compendiums of measures n See Appendix B n Web measurement resources n See Appendix B