Examining Early Child Development Issues Relating to Measurement

Examining Early Child Development: Issues Relating to Measurement Lia Fernald, Ph. D. School of Public Health University of California, Berkeley March 21, 2013, Universidad de Chile Desarrollo en la primera infancia: Evidencia longitudinal

In memory: Patrice Lee Engle 1944 -2012

Today • • Measuring child development Domains of development to be measured Theoretical decisions in selecting instruments Chile: Early Childhood Longitudinal Survey

Introduction: Why measure child development? >200 million disadvantaged children worldwide Percentage of disadvantaged children under 5 years old by country in 2004 Grantham-Mc. Gregor et al. , Lancet (2007)

Differential risk and vulnerability • Children’s development from 0 -5 is dependent on quality of early environments and relationship with caregiver. • Young children growing up in poverty are disproportionately exposed to a wide range of risk factors: • Poor nutrition • Less stimulating learning environments • Poor sanitation • Stressful life events • Exposure to environmental risks

Poverty and cumulative risk • Number of risk factors increases over time. – Cumulative effect of risk factors becomes more evident as children get older • Higher cumulative levels of risk are associated with: – Poorer cognitive development – Psychological distress and behavioral problems – Slower and lower quality communicative development

Conceptual framework Direct & Indirect Effects From Walker and al. Lancet, 2007 • Environmental factors – Psychosocial risks: harsh disciplinary techniques, maternal depression – Biological risks: malnutrition and infectious diseases • Poverty and socio-cultural factors increase likelihood of both types of risks

Timeline of development Timing of human brain development, from Grantham-Mc. Gregor, et al. , 2007 • Early childhood is characterized by developmental spurts and plateaus • Skills emerge at different rates and ages

Ecological model of child development Adapted representation of Bronfenbrenner’s ecological model of child development (Wortham, 2007)

The “Toolkit” • Importance of measuring child development • Domains of development to be measured • Theoretical decisions in selecting instruments • Modification, adaptation and standardization of existing tests • Creation of new tests • Training and quality control • Conclusions and recommendations

Domains of development • Domains of development: – – Cognitive Language Motor Executive function/selfregulatory – Social/emotional • Domains are overlapping and mutually influencing • Every effort should be made to include multiple domains when assessing children’s development

STEP 1: Define purpose of assessment Step 1: Define purpose of assessment For example: 1. To plan interventions or services; 2. To monitor programs; 3. To conduct impact evaluations; 4. To investigate the effect of interventions or programs on specific outcomes of interest; 5. To design a curriculum for a particular child; or 6. To diagnose and assess child progress STEP 2: Determine type of assessment Abilities Screening Brief assessment; identifies children likely to have problems based on cutoffs derived in test population. Does not yield continuous scores. Useful for examples 1 -4 above. Detailed assessment of child’s maximum skill level for age. Provides continuous scores that allow comparisons within and across children/groups. Suitable for all examples above. STEP 3: Determine mode of assessment Direct Ratings/ Reports Observation STEP 4: Determine which assessment to use (examples below) Denver (DDST II) Ages and Stages Questionnaires Naturalistic sample or structured sampling Bayley Scales III Woodcock. Johnson WPPSI Stanford-Binet Kaufman-ABC Executive function tasks Mac. Arthur Communicativ e Inventories Naturalistic sample or structured sampling (see IEA’s Child Coding System)

Key Questions in Selecting Instruments • What are the goals of the assessment/evaluation? • What dimensions of child’s development do you expect to be affected by the intervention? – What developmental systems are most vulnerable at a given age range? – What are immediate outcomes and longer term outcomes? • What are the mechanisms at work? – What physiologic processes are influenced by iodine/iron/poverty? • What are key elements of context that must be considered in selecting the test? – Urban/rural, level of poverty, parent education. • At what level will effect be measured? – Individual? Household? Population (then consider test such as EDI)? • How will the sample be selected? – Population sample? Sub-sample? • What is the analytic plan? – Are norms relevant and/or available? Will a cut-off score be used?

STEP 1: Define purpose of assessment Step 2: Determine type of assessment For example: 1. To plan interventions or services; 2. To monitor programs; 3. To conduct impact evaluations; 4. To investigate the effect of interventions or programs on specific outcomes of interest; 5. To design a curriculum for a particular child; or 6. To diagnose and assess child progress STEP 2: Determine type of assessment Abilities Screening Brief assessment; identifies children likely to have problems based on cutoffs derived in test population. Does not yield continuous scores. Useful for examples 1 -4 above. Detailed assessment of child’s maximum skill level for age. Provides continuous scores that allow comparisons within and across children/groups. Suitable for all examples above. STEP 3: Determine mode of assessment Direct Ratings/ Reports Observation STEP 4: Determine which assessment to use (examples below) Denver (DDST II) Ages and Stages Questionnaires Naturalistic sample or structured sampling Bayley Scales III Woodcock. Johnson WPPSI Stanford-Binet Kaufman-ABC Executive function tasks Mac. Arthur Communicativ e Inventories Naturalistic sample or structured sampling (see IEA’s Child Coding System)

Screening versus ability test • Screening tests: brief assessments to identify children who are at risk of having development problems – Inexpensive, quick, and relatively easy to administer – Classify children into categories • Cutoffs used in one population to classify children should not be applied to another population! • Ability tests: longer tests that assess the maximum skill level for a child at any given age – Continuous scores that can be used to compare children’s developmental levels with more precision

STEP 1: Define purpose of assessment Step 3: Determine mode of assessment For example: 1. To plan interventions or services; 2. To monitor programs; 3. To conduct impact evaluations; 4. To investigate the effect of interventions or programs on specific outcomes of interest; 5. To design a curriculum for a particular child; or 6. To diagnose and assess child progress STEP 2: Determine type of assessment Abilities Screening Brief assessment; identifies children likely to have problems based on cutoffs derived in test population. Does not yield continuous scores. Useful for examples 1 -4 above. Detailed assessment of child’s maximum skill level for age. Provides continuous scores that allow comparisons within and across children/groups. Suitable for all examples above. STEP 3: Determine mode of assessment Direct Ratings/ Reports Observation STEP 4: Determine which assessment to use (examples below) Denver (DDST II) Ages and Stages Questionnaires Naturalistic sample or structured sampling Bayley Scales III Woodcock. Johnson WPPSI Stanford-Binet Kaufman-ABC Executive function tasks Mac. Arthur Communicativ e Inventories Naturalistic sample or structured sampling (see IEA’s Child Coding System)

Types of assessments: Direct tests • Pros: – Data are gathered first hand – Data can be less biased than parental reports – Potentially wider range of outcomes can be assessed – Many of the “cons” can be overcome with careful planning and preparation • Cons: – Young children can be difficult to test (sleeping, hungry) – Testers need a lot of training and oversight – Accuracy depends on testing demands and child must be familiar with parameters (e. g. best v. worst)

Types of assessments: Parent report • Pros – Easy to administer and require minimal training and instruction – Often are quick and easy to complete and to score – Parents can become involved and express concerns – Often correlate well with direct assessments – Teachers can be an additional source of information as children get older • Cons – Parents and teachers may artificially inflate scores – Parents may not accurately report abilities – Parents and teachers may have different interpretations of items in different cultures

Types of assessments: Observation Types of observation: Naturalistic observation, Sampled observation, Structured situation • Pros: – Highly valid – Measures behavior in an identified context – Can provide additional or confirmatory information for other types of assessments • Cons: – Requires a lot of time and training – Need to identify if culturally appropriate – Difficult coding since observational codes and definitions are not always clearly defined

STEP 1: Define purpose of assessment Step 4: Determine which assessment to use For example: 1. To plan interventions or services; 2. To monitor programs; 3. To conduct impact evaluations; 4. To investigate the effect of interventions or programs on specific outcomes of interest; 5. To design a curriculum for a particular child; or 6. To diagnose and assess child progress STEP 2: Determine type of assessment Abilities Screening Brief assessment; identifies children likely to have problems based on cutoffs derived in test population. Does not yield continuous scores. Useful for examples 1 -4 above. Detailed assessment of child’s maximum skill level for age. Provides continuous scores that allow comparisons within and across children/groups. Suitable for all examples above. STEP 3: Determine mode of assessment Direct Ratings/ Reports Observation STEP 4: Determine which assessment to use (examples below) Denver (DDST II) Ages and Stages Questionnaires Naturalistic sample or structured sampling Bayley Scales III Woodcock. Johnson WPPSI Stanford-Binet Kaufman-ABC Executive function tasks Mac. Arthur Communicativ e Inventories Naturalistic sample or structured sampling (see IEA’s Child Coding System)

Other constraints to consider • Budget: Tests can be very expensive (e. g. $1000 for Bayley); administration time is a budget issue, too. • Copyright issues: Must obtain permission for most tests. • Time allocated for testing: Direct assessment v. parent rep. • Training: Capacity for administration. • Test setting: Set-up, lighting, noise, observers • Capacity of respondent: Education/knowledge of parent • Language and cultural differences: Words used in testing materials, approach used for testing (e. g. speedy response) • Materials: Must be familiar and/or available (e. g. mirror, ball)

Ethical risks and responsibilities • All assessment protocols must be reviewed and approved by an ethical review board • Accuracy and validity are extremely important especially if test scores are being used to identify children “with delays” • Follow-up (e. g. referrals for at-risk children) should be mandatory even in the context of a developing country.

Broad recommendations • Assess characteristics of the child that the intervention is intending to affect. – Make sure to measure variables that could also be contributing to the outcomes (e. g. maternal responsiveness, home environment) • Decide on the type of outcome measure that is appropriate for the evaluation. • Rely upon multiple measures of children’s development. – Include assessments of executive function and socio-emotional development • Consider the cultural context and how it may affect children’s development and school readiness – Always work with local collaborators! • Look for national level tests where possible and use parent/teacher report when possible. • Begin following children early in life.

Criteria for being recommended • Psychometrically adequate, valid and reliable; • Balanced in terms of number of items at the lower end to avoid children with low scores; • Enjoyable for children to take (e. g. interactive, colorful materials); • Relatively easy to adapt to various cultures; • Easy to use in low-resource settings, e. g. not requiring much material; • Not too difficult to obtain or too expensive; • Able to be used in a wide age range.

ELPI 2010 Area Test EEDP (Scale of Psychomotor Development Evaluation) Overall Development BDI (Battelle Developmental Inventory) TEPSI (Psychomotor Development Test) TVIP / PPVT (Peabody Picture Vocabulary Test) Area Test Scale WAIS Digit Cognitive Abilities Evaluation Proposal Validity, Reliability and Norms Evaluation Proposal Validity, Reliability and Norms Scale WAIS Vocabulary Validity, Reliability and Norms

ELPI 2012 Area Test Evaluation Proposal TADI (Test of Learning and Child Development) Made it by CEDEP Overall BDI-ST 2 (Battelle Developmental Inventory - Validity, Reliability and Development Screening Test 2) Norms TVIP / PPVT (Peabody Picture Vocabulary Test) SDT (Snack Delay Task) PTT (Pencil Tapping Task) Validity, Reliability and Norms Executive Function BDS (Backward Digit Span Task) HTKS (Head Toes Knees Shoulders Task) Area Test Scale WAIS Digit Cognitive Abilities Validity, Reliability and Norms Evaluation Proposal Validity, Reliability and Norms Scale WAIS Vocabulary Validity, Reliability and Norms

ELPI: Home stimulation (HOME score)

ELPI: Wealth gradients

ELPI (PPVT): gradients over time

Advantages of longitudinal cohort • Document specific trajectories of development • Examine shape and timing of emergence of particular skills • Steepening of gradient at different ages due to individual or household shocks • Potential for interventions to affect developmental trajectories

Acknowledgements • Funding and support from World Bank for the Toolkit – – Barbara Bruns Sophie Naudeau Harold Alderman Ariel Fitzbein • External reviewers for the Toolkit – – – – Frances Aboud, Mc. Gill University Santiago Cueto, Catholic University, Peru Ed Frongillo, University of South Carolina Jane Kvalsvig, University of Kwa-Zulu Natal, South Africa Ann Weber, University of California, Berkeley Paul Wassenich, University of California, Berkeley Michelle Neuman, The World Bank Mary Eming Young, The World Bank • Research assistants – Robin Dean (UC Berkeley), Kallista Bley (UC Berkeley), Melissa Hidrobo (UC Berkeley), Anna Moore (Cal Poly), Rose Calnin Kagawa • Photo credits for photographs included in presentation – Lia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber, Tricia Kariger

Thank you! Contact information: Lia Fernald School of Public Health University of California, Berkeley fernald@berkeley. edu (510) 643 -9113