Single Case Design An Option for Evaluating Programs

Single Case Design: An Option for Evaluating Programs in Communities with Small Sample Sizes Dennis Embry, Ph. D Paxis Institute Kate Lyon, MA James Bell Associates, Inc. Aleta Meyer, Ph. D Office of Planning, Research, and Evaluation Administration for Children and Families April 2012

Our Ancestors were everyday scientists • They were scientists of their physical, social, and spiritual environments… 2

Humans are born as Everyday scientists… 3

What makes a good everyday scientist of behavior? Holds labels and past explanations lightly. 4

What makes a good everyday scientist of behavior? Ability to notice variation and sources of variation in. Physical environment Social environment 5

What makes a good everyday scientist of behavior? Ability to count behavior reliably. 6

What makes a good everyday scientist of behavior? Experiments by hypothesis testing. 7

What makes a good everyday scientist of behavior? Notices cause and effect in time. 8

What makes a good everyday scientist of behavior? Notices how behavior is adaptive to context. 9

Single Case Design: • Rigorous • Tests impact on a single case • Determines general impact using small sample size • Visual analysis of data • Each case serves as its own control 10

Single Case Design: • Requires replication of effect with multiple cases to demonstrate causal evidence that intervention caused change in behavior 11

12

13

Dependent Variable = observable target behavior that should be changed by the intervention • Precisely define target behavior to allow for measurement of the relative occurrence of the behavior before and after an intervention 14

Your definition of a dependent variable should be: ü Objective ü Clear ü Complete Two Examples: 1. Increase positive parenting practices OR 2. Increase parents’ rates of verbal praise for children and decrease rates of verbal criticism 15

Dennis’ definitions as a prevention scientist… Behavior = Reliably measurable actions (frequency, duration and/or intensity) of a person (adult or child) in a specific environment. 16

Frequency = counting how many behaviors in a certain time (e. g. , per minute, per hour) • What behaviors on your list might you count by frequency when at a home visit? 17

Duration = how long a specific behavior lasts (e. g. , seconds, minutes, hours) • What behaviors on your list might you count by duration at a home visit? 18

Intensity = positive or negative strength (i. e. , voice, gestures, expressions) • What behaviors on your list might you count by intensity at a home visit? 19

Frequency means? A) How loud or soft a behavior is. B) Number of occurrences per unit of time. C) How long a behavior lasts. D) None of the above Vote 20

Duration means? A) How long a behavior lasts after learning. B) Percentage of behavior decay. C) Total time from start to stop of a behavior. D) A and B Vote 21

Intensity means? A) Qualitative rating of force, threat, or loudness of a behavior. B) Lumens per second of the behavior. C) Duration times the frequency of a behavior. D) All of the above. Vote 22

Ask yourself 3 questions: 1. Is your dependent variable a behavior? 2. Is your definition: 3. Does your definition use: ü Objective? • Frequency? ü Clear? • Intensity? ü Complete? • Duration? 23

What are your tools (not program) to increase or decrease the behaviors on your list at a home visit while you are there? 24

25

Independent variable = the practice, treatment or intervention being tested that should produce observable changes in the dependent variable • Is it time limited? • Can it be started and stopped? • Why will the IV change DV? 26

What is your lever? Will it move your rock? 27

Which lever moved the rock? 28

The application of the independent variable is observed and measured over time 29

How do you test whether the lever moves the rock? • Withdrawal and reversal designs • Multiple baseline design • Selection of design depends on the research question and objectives 30

ABAB (Reversal) Design 9 baseline phase intervention phase 8 Frequency of behavior 7 6 5 BL Therapy 4 BL 3 Therapy 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 Days 13 14 15 16 17 18 19 20 21 31

ABAB (Reversal) Design Child’s Tasks Mastered Baseline Lottery 1 Baseline 2 Lottery 2 20 15 Benji & Mom 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Successive Home Visits ABA Example… Teaching parents to increase their developmental delayed child’s competencies Muir, K. A. , & Milan, M. A. (1982). Parent reinforcement for child achievement: the use of a lottery to maximize parent training effects. J Appl Behav Anal, 15(3), 455460. doi: 10. 1901/jaba. 1982. 15 -455 32

Step 1: Multiple Baseline Design 12 10 8 6 Participant 1 4 BL Therapy 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 33

Multiple Baseline Design with 3 Participants 34

Multiple Baseline Example… Teaching parents how to reduce children’s dangerous or risky behavior Embry, D. D. (1984). The safe-playing program: A case study of putting research into practice. In S. Paine & B. Bellamy (Eds. ), Human Services That Work: From Innovation to Standard Practice (pp. 624). Baltimore, MD: Brookes Co. 35

You decide… Reversal design Multiple baseline design When would you use each? Which is better for you? 36

Establish a Stable Baseline 7 6 5 4 3 2 1 0 1 2 3 4 5 7 stable or instable ? 6 5 4 3 2 1 37 0 1 2 3 4 5

Dependent variable is measured repeatedly to allow for identification of behavior patterns Measures need to be sensitive to change Measurement of dependent variable must be monitored for consistency by measuring inter-observer agreement 38

1 Must document three demonstrations of the experimental effect at three different points in time with a single participant or across different participants. 2 3 39

1 9 baseline phase 3 2 intervention phase baseline phase intervention phase 8 Frequency of behavior 7 6 5 BL Therapy 4 BL 3 Therapy 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 Days 13 14 15 16 17 18 19 20 21 40

Expanding your toolkit for influencing behavior change 41

FREE DOWNLOAD Embry, D. D. , & Biglan, A. (2008). Evidence-Based Kernels: Fundamental Units of Behavioral Influence. Clinical Child & Family Psychology Review, 39. Download at: www. pubmed. gov 42

Antecedent Kernel Happens BEFORE the behavior Reinforcement Kernel Happens AFTER the behavior Relational Frame Kernel Physiological Kernel Creates verbal Changes biochemistry of relations for behavior the behavior Four Types of Evidence. Based Kernels 43

Definition and properties of kernels Irreductable unit of behavioral influence (e. g. simple, cannot be made smaller) Have scientifically proof and peer reviewed publications showing effect using valid experimental design Produce observable behavior change quickly Often described in folk wisdom (e. g. , grandmother told you) Easy to explain Can be used alone or combined with other strategies Are trialable Low cost or no-cost Can be used at many levels Individual, family, classroom School or community Used in environmental policies 44

45

46

Training Family Therapists Training therapist to praise parent Change in parent & child behaviors Parent scolding, reprimanding, negative to child Parent praise for positive child behavior Negative child behavior Positive child behavior Isaacs, Embry & Baer (1982), Training Family Therapists, JABA, 15. 505 -520 47

Applied Behavior Analysis An evaluation using a singlesubject design (ABA model) needs back up if the initial intervention strategy fails to leverage change. ABA’s are not about finding a failure; rather about a solution. Takeaway: Build it until it works Huybers et al. J Appl Behav Anal. 2004 Winter; 37(4): 445 -56. 48

And. you can be big with single subject studies 49

Everyday scientists then and now 50

For more information… Kate Lyon, James Bell Associates (TEI/JBA) lyon@jbassoc. com Julie Morales, James Bell Associates (TEI/JBA) morales@jbassoc. com Aleta Meyer, Office of Planning Research and Evaluation aleta. meyer@acf. hhs. gov Carol Gage, Administration for Children and Families (ACF) carol. gage@acf. hhs. gov Bridget Shea Westfall, ACF bridget. shea@acf. hhs. gov Moushumi Beltangady, ACF moushumi. beltangady@acf. hhs. gov 51

Questions and Comments 52

The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP 23320095644 WC. TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings. TEI 1 was awarded to MDRC; James Bell Associates, Inc. ; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health. For more information on TEI contact: Nicole Denmark Federal Project Officer Office of Planning Research and Evaluation nicole. denmark@acf. hhs. gov Kate Lyon Project Director James Bell Associates, Inc. lyon@jbassoc. com 53