Econometric analysis informing policies UNICEF workshop 13 May

Econometric analysis informing policies UNICEF workshop, 13 May 2008 Christian Stoff Statistics Division, UNESCAP, stoff@un. org

Outline • • • Causality in experiments Confounding factors Quasi experiments: Difference estimators Difference-in-difference estimators Possible questions

A quick intro: The ideal situation • Causality means a specific action leads to a specific, measurable consequence • Ideal: Randomized controlled experiment • Subjects are randomly allocated to either control or treatment group • ONLY difference between two groups is treatment

The reality • In practice experiments are rare • Subjects are NOT randomly assigned so that sorting out of other relevant factors is difficult • Econometrics provides the tools for controlling these other factors

Challenge: Confounding factors • Regress test score on student-teacher ratio • But what about number of students in class still learning English? – Omitted variable? • Omitted variable correlated with explanatory and dependent variable (low student-teacher ratios -> high % English learners -> bad scores) • Thus a policy of increasing no. teachers may not increase test scores because high English learners (%) are the real problem • Solution: Control for differences in English learners (%), i. e. regress test score on student-teacher ratio AND English learners (%)

Limits to controlling these factors • Many years of cross-country research • However, countries often have such different settings and the “causal” relationships are only specific to the country and the time period • Therefore in search of quasi or natural experiments between units that are “not too different” • Danger lies in… – Possible correlation between error term and explanatory variable (i. e. treatment not assigned at random) – Teachers try especially hard in areas with programs – General equilibrium effects: when program is enlarged additional factors may arise (external validity)

Difference estimators: Using MICS 3 • Define unit of analysis (households, districts, provinces, countries) • Selected units gone through policy program (i. e. treatment) AND assignment was “as if” random • If is binary, then no functional form assumption needed; it is simply the difference in the conditional expectations • If can take on multiple values, then the above regression assumes linearity • But often there are pre-treatment differences between control and treatment group…

Difference-in-difference estimators: Using MICS 2 and MICS 3 • Types of datasets: Cross-section, panel and time-series • Includes observations on same units before and after experiment • OLS estimator is the difference in the group means of • Control for district-level context constant over time through fixed or random effects or adjust standard errors for clustering • Advantage over difference estimator: 1. More efficient; 2. Eliminates pre-treatment differences

Some possible questions • • • Education research: – Study drop-out rates and relate it to child labour questions – Study effect of different child disciplining strategies (punishment, praise, etc. ) on a child’s “success” in school – Combine MICS data with GIS disaster data and study effect of disasters on school attendance – Combine with policy data between 2000 -2005 and evaluate the effectiveness of policies aimed at promoting higher school attendance Child health: – Effect of different fuel types for cooking on child-health indicators? – Effects of different types of access to water and sanitation on a child’s probability of having diarrhoea or succeeding in school? – How does Vitamin A affect a child’s health? – Impact of different health service facilities Adult’s knowledge and attitude towards violence: – What is the effect of having information access (TV or radio) on knowledge about HIV or contraception? – What is the effect of having information access (TV or radio) on education methods or attitudes towards domestic violence?