Experiments and QuasiExperiments MingChing Luoh p Idealized Experiments

  • Slides: 79
Download presentation
Experiments and Quasi-Experiments Ming-Ching Luoh

Experiments and Quasi-Experiments Ming-Ching Luoh

p Idealized Experiments and Causal Effects n n n Estimation Internal Validity External Validity

p Idealized Experiments and Causal Effects n n n Estimation Internal Validity External Validity Regression Estimators Using Experimental Data p The Tennessee Class Size Experiment p Quasi-Experiments p Threats to Quasi-Experiments p Average Treatment Effect p n n OLS IV Regression

Why study experiments? p Ideal randomized controlled experiments provide a benchmark for assessing observational

Why study experiments? p Ideal randomized controlled experiments provide a benchmark for assessing observational studies. p Actual experiments are rare ($$$) but influential. p Experiments can solve threats to internal validity of observational studies, but they have their own threats to internal validity. p Thinking about experiments helps us to understand quasi-experiments, or “natural experiments, ” in which there some variation is “as if” randomly assigned.

Terminology: experiments and quasiexperiments An experiment is designed and implemented consciously by human researchers.

Terminology: experiments and quasiexperiments An experiment is designed and implemented consciously by human researchers. An experiment entails conscious use of a treatment and control group with random assignment (e. g. clinical trials of a drug. ) p A quasi-experiment or natural experiment has a source of randomization that is “as if” randomly assigned, but this variation was not part of a conscious randomized treatment and control design. p Program evaluation is the field of statistics aimed at evaluating the effect of a program or policy, for example, an ad campaign to cut smoking. p

Different types of experiments: three examples p Clinical drug trial: does a proposed drug

Different types of experiments: three examples p Clinical drug trial: does a proposed drug lower cholesterol? n n p Job training program (Job Training Partnership Act) n n p Y = cholesterol level X = treatment or control group (or dose of drug) Y = has a job, or not (or Y = wage income) X = went through experimental program, or not Class size effect (Tennessee class size experiment) n n Y = test score (Stanford Achievement Test) X = class size treatment group (regular, regular + aide, small)

Brief outline of discussing experiments: Why (precisely) do ideal randomized controlled experiments provide estimates

Brief outline of discussing experiments: Why (precisely) do ideal randomized controlled experiments provide estimates of causal effects? p What are the main threats to the validity (internal and external) of actual experiments - that is, experiments actually conducted with human subjects? p Flaws in actual experiments can result in X and u being correlated (threats to internal validity). p Some of these threats can be addressed using the regression estimation methods we have used so far— multiple regression, panel data, IV regression. p

Idealized Experiments and Causal Effects An ideal randomized controlled experiment randomly assigns subjects to

Idealized Experiments and Causal Effects An ideal randomized controlled experiment randomly assigns subjects to treatment and control groups. p More generally, the treatment level X is randomly assigned: p If X is randomly assigned (for example, by computer), then u and X are independently distributed and , so OLS yields an unbiased estimator of. p The causal effect is the population value of in an ideal randomized controlled experiment. p

Estimation of causal effects in an ideal randomized controlled experiment p Random assignment of

Estimation of causal effects in an ideal randomized controlled experiment p Random assignment of X implies that Thus the OLS estimator is consistent. p When the treatment is binary, is just the difference in mean outcome (Y ) in the treatment vs. control group p p This differences in means is sometimes called the differences estimator.

Potential Problems with Experiments in Practice Threats to Internal Validity 1. Failure to randomize

Potential Problems with Experiments in Practice Threats to Internal Validity 1. Failure to randomize (or imperfect randomization) p for example, openings in job treatment program are filled on first-come, first-serve basis; latecomers are controls. p result is correlation between X and u.

2. Failure to follow treatment protocol (or “partial compliance”) p some controls get the

2. Failure to follow treatment protocol (or “partial compliance”) p some controls get the treatment. p some “treated” get controls. p “errors-in-variables” bias: p attrition (some subjects drop out). p suppose the controls who get jobs move out of town, then

3. Experimental effects p experimenter bias (conscious or subconscious): treatment X is associated with

3. Experimental effects p experimenter bias (conscious or subconscious): treatment X is associated with “extra effort” or “extra care, ” so p subject behavior might be affected by being in an experiment, so (Hawthorne effect) Just as in regression analysis with observational data, threats to the internal validity of regression with experimental data implies that , so OLS (the differences estimator) is biased.

Threats to External Validity Nonrepresentative sample. p Nonrepresentative “treatment” (that is, program or policy).

Threats to External Validity Nonrepresentative sample. p Nonrepresentative “treatment” (that is, program or policy). p General equilibrium effects (effect of a program can depend on its scale). p Treatment v. s. eligibility effects (which is it we want to measure: effect on those who take the program, or the effect on those are eligible). p

Regression Estimators Using Experimental Data Focus on the case that X is binary (treatment/control).

Regression Estimators Using Experimental Data Focus on the case that X is binary (treatment/control). p Often we observe subject characteristics, p Extensions of the differences estimator: p n n can improve efficiency (reduce standard errors) can eliminate bias that arises when: treatment and control groups differ p there is “conditional randomization” p there is partial compliance p p These extensions involve methods we have already seen - multiple regression, panel data, IV regression.

The differences-in-differences estimator Suppose the treatment and control groups differ systematically; maybe the control

The differences-in-differences estimator Suppose the treatment and control groups differ systematically; maybe the control group is healthier (wealthier; better educated; etc. ). p Then X is correlated with u, and the differences estimator is biased. p The differences-in-differences estimator adjusts for pre-experimental differences by subtracting off each subject’s pre-experimental value of Y. p n n n = value of Y for subject i before the experiment. = value of Y for subject i after the experiment. change over the course of experiment.

The differences-in-differences estimator (1) “Differences” formulation: where is the diffs-in-diffs estimator because is

The differences-in-differences estimator (1) “Differences” formulation: where is the diffs-in-diffs estimator because is

(2) Equivalent “panel data” version: Where t = 1 (before experiment), 2 (after experiment)

(2) Equivalent “panel data” version: Where t = 1 (before experiment), 2 (after experiment) Dit = 0 for t = 1, Dit = 1 for t = 2 Git = 0 for control group, Git = 1 for treatment group Xit = Dit × Git = interaction effect of being in treatment group in the second period:

is the diffs-in-diffs estimator because Therefore,

is the diffs-in-diffs estimator because Therefore,

Including additional subject characteristics (W’s) Typically you observe additional subject characteristics, W 1 i

Including additional subject characteristics (W’s) Typically you observe additional subject characteristics, W 1 i , … , Wri. p Differences estimator with additional regressors: p p Differences-in-differences estimator with W’s: where

Why include additional subject characteristics (W’s)? Efficiency: more precise estimator of (smaller standard errors).

Why include additional subject characteristics (W’s)? Efficiency: more precise estimator of (smaller standard errors). p Check for randomization. If X is randomly assigned, then the OLS estimators with and without the W’s should be similar - if they aren’t, this suggests that X wasn’t randomly designed (a problem with the experiment. ) p n p Note: To check directly for randomization, regress X on the W’s and do a F-test. Adjust for conditional randomization (we’ll return to this later. )

Estimation when there is partial compliance, TSLS Consider diffs-in-diffs estimator, X = actual treatment

Estimation when there is partial compliance, TSLS Consider diffs-in-diffs estimator, X = actual treatment Suppose there is partial compliance: some of the treated don’t take the drug; some of the controls go to job training anyway. p Then X is correlated with u, and OLS is biased. p Suppose initial assignment, Z, is random. p Then (1) corr (Z, X) ≠ 0 and (2) corr (Z, u) = 0. p Thus can be estimated by TSLS, with instrumental variable Z = initial assignment p This can be extended to W’s (included exog. variables) p

The Tennessee Class Size Experiment Project STAR (Student-Teacher Achievement Ratio) p 4 -year study,

The Tennessee Class Size Experiment Project STAR (Student-Teacher Achievement Ratio) p 4 -year study, $12 million. p Upon entering the school system, a student was randomly assigned to one of three groups: n n n regular class (22 - 25 students) regular class + aide small class (13 - 17 students) regular class students re-randomized after first year to regular or regular+aide. p Y = Stanford Achievement Test scores. p

Deviations fromexperimental design p Partial compliance: n n p 10% of students switched treatment

Deviations fromexperimental design p Partial compliance: n n p 10% of students switched treatment groups because of “incompatibility” and “behavior problems” - how much of this was because of parental pressure? Newcomers: incomplete receipt of treatment for those who move into district after grade 1. Attrition n n students move out of district students leave for private/religious schools

Regression analysis p The “differences” regression model: where Small. Classi = 1 if in

Regression analysis p The “differences” regression model: where Small. Classi = 1 if in a small class Reg. Aidei = 1 if in regular class with aide p Additional regressors (W’s) n n n teacher experience. free lunch eligibility. gender, race.

Differences estimates (no W’s)

Differences estimates (no W’s)

How big are these estimated effects? Put on same basis by dividing by std.

How big are these estimated effects? Put on same basis by dividing by std. dev. of Y. p Units are now standard deviations of test scores. p

How do these estimates compare to those from the California, Mass. observational studies?

How do these estimates compare to those from the California, Mass. observational studies?

Summary: The Tennessee Class Size Experiment Remaining threats to internal validity p partial compliance/incomplete

Summary: The Tennessee Class Size Experiment Remaining threats to internal validity p partial compliance/incomplete treatment n n can use TSLS with Z = initial assignment. Turns out, TSLS and OLS estimates are similar (Krueger (1999)), so this bias seems not to be large. Main findings: p The effects are small quantitatively (same size as gender difference). p Effect is sustained but not cumulative or increasing biggest effect at the youngest grades.

underline. Effect of teacher experience on test scores More on the design of Project

underline. Effect of teacher experience on test scores More on the design of Project STAR: p Teachers didn’t change school because of the experiment. p Within their normal school, teachers were randomly assigned to small/regular/reg+aide classrooms. p What is the effect of X = years of teacher education?

The design implies conditional mean independence: p W = school binary indicator p Given

The design implies conditional mean independence: p W = school binary indicator p Given W (school), X is randomly assigned. p That is, E(u|X, W) = E(u|W). p Wis plausibly correlated with u (nonzero school fixed effects: some schools are better/richer/etc than others)

Example: teacher experience, ctd. Without school fixed effects (2), the estimated effect of an

Example: teacher experience, ctd. Without school fixed effects (2), the estimated effect of an additional year of experience is 1. 47 (SE =. 17). p “Controlling for the school” (3), the estimated effect of an additional year of experience is. 74 (SE =. 17). p Direction of bias makes sense: p n n p less experienced teachers at worse schools. years of experience picks up this school effect. OLS estimator of coefficient on years of experience is biased up without school effects, with school effects, OLS yields unbiased estimator of causal effect.

Quasi-Experiments A quasi-experiment or natural experiment has a source of randomization that is “as

Quasi-Experiments A quasi-experiment or natural experiment has a source of randomization that is “as if” randomly assigned, but this variation was not part of a conscious randomized treatment and control design. Two cases: (a) Treatment (X) is “as if” randomly assigned (OLS). (b) A variable (Z) that influences treatment (X) is “as if” randomly assigned (IV).

Two types of quasi-experiments (a) Treatment (X) is “as if” randomly assigned (perhaps conditional

Two types of quasi-experiments (a) Treatment (X) is “as if” randomly assigned (perhaps conditional on some control variables W. ) p Ex: Effect of marginal tax rates on labor supply n X = marginal tax rate (rate changes in one state, not another; state is “as if” randomly assigned) (b) A variable (Z) that influences treatment (X) is “as if” randomly assigned (IV) p Effect on survival of cardiac catheterization X = cardiac catheterization; Z = differential distance to CC hospital

Example 1: Labor market effects of immigration. Does immigration reduce wages? p Immigration are

Example 1: Labor market effects of immigration. Does immigration reduce wages? p Immigration are attracted to cities with higher labor demand. Immigration is endogenous. p Card (1990) used a quasi-experiment in which a large number of Cuban immigrants entered the Miami, Folrida labor market, which resulted from a temporary lifting of restrictions on emigration from Cuba in 1980. p He concluded that the influx of immigrants had a negligible effect on wages of less-skilled workers. p

Example 2: Effects of military service on civilian earnings. Does serving in the military

Example 2: Effects of military service on civilian earnings. Does serving in the military improve your prospects on the labor market? p Military servics is determined, at least in part, by individual choices and characteristics. p Angrist (1990) examined labor market histories of Vietname. War veterans. p Whether a young was drafted was determined in part by a national lottery system: men randomly assigned low lottery numbers were eligible to be drafted. p

Example 2: ctd. Being draft-eligible serves as an IV that partially determines military service

Example 2: ctd. Being draft-eligible serves as an IV that partially determines military service but is randomly assigned. p Angrist concluded that the long-term effect of military service was to reduce earnigs of white, but nonwhite, veterans. p

Econometric methods (a) Treatment (X) is “as if” randomly assigned (OLS) Diffs-in-diffs estimator using

Econometric methods (a) Treatment (X) is “as if” randomly assigned (OLS) Diffs-in-diffs estimator using panel data methods: where t = 1 (before experiment), 2 (after experiment) Dit = 0 for t = 1, Dit = 1 for t = 2 Git = 0 for control group, Git = 1 for treatment group Xit = Dit × Git = interaction effect of being in treatment group in the second period: p is the diffs-in-diffs estimator.

The panel data diffs-in-diffs estimator simplifies to the “changes” diffs-in-diffs estimator when T =

The panel data diffs-in-diffs estimator simplifies to the “changes” diffs-in-diffs estimator when T = 2

Differences-in-differences with control variables =1 if the treatment is received, = 0 otherwise =

Differences-in-differences with control variables =1 if the treatment is received, = 0 otherwise = (= 1 for treatment group in second period) p If the treatment (X) is “as if” randomly assigned, given W, then u is conditionally mean independent of X: OLS is a consistent estimator of 1, the causal effect of a change in X. p In general, the OLS estimators of the other p

(b) A variable (Z) that influences treatment (X) is “as if” randomly assigned (IV)

(b) A variable (Z) that influences treatment (X) is “as if” randomly assigned (IV)

TSLS: p X = endogenous regressor p D, G, W 1, … , Wr

TSLS: p X = endogenous regressor p D, G, W 1, … , Wr = included exogenous variables p Z = instrumental variable

Potential Threats to Quasi-Experiments The threats to the internal validity of a quasiexperiment are

Potential Threats to Quasi-Experiments The threats to the internal validity of a quasiexperiment are the same as for a true experiment, with one addition. p Failure to randomize (imperfect randomization). Is the “as if” randomization really random, so that X (or Z) is uncorrelated with u? p Failure to follow treatment protocol & attrition. p No experimental effects. p Instrument invalidity (relevance + exogeneity) (Maybe healthier patients do live closer to CC hospitals – they might have better access to care in

The threats to the external validity of a quasiexperiment are the same as for

The threats to the external validity of a quasiexperiment are the same as for an observational study. p Nonrepresentative sample. p Nonrepresentative “treatment” (that is, program or policy). Example: Cardiac catheterization p The CC study has better external validity than controlled clinical trials because the CC study uses observational data based on real-world implementation of cardiac catheterization. However that study used data from the early 90’s - do its findings apply to CC usage today?

Experimental and Quasi-Experimental Estimates in Heterogeneous Populations We have discussed “the” treatment effect. p

Experimental and Quasi-Experimental Estimates in Heterogeneous Populations We have discussed “the” treatment effect. p But the treatment effect could vary across individuals: p n n Effect of job training program probably depends on education, years of education, etc. Effect of a cholesterol-lowering drug could depend other health factors (smoking, age, diabetes, . . . )

If this variation depends on observed variable, then this can be solved by interaction

If this variation depends on observed variable, then this can be solved by interaction variables. p But what if the source of variation is unobserved? p

Heterogeneity of causal effects When the causal effect (treatment effect) varies among individuals, the

Heterogeneity of causal effects When the causal effect (treatment effect) varies among individuals, the population is said to be heterogeneous. When there are heterogeneous causal effects that are not linked to an observed variable: p What do we want to estimate? n n p Often, the average causal effect in the population. But there are other choices, for example the average causal effect for those who participate (effect of treatment on the treated). What do we actually estimate? n using OLS? using TSLS?

Population regression model with heterogeneous causal effects: is the causal effect (treatment effect) for

Population regression model with heterogeneous causal effects: is the causal effect (treatment effect) for the ith individual in the sample. p For example, in the JTPA experiment, could be zero if person i already has good job search skills. p What do we want to estimate? p n n n effect of the program on a randomly selected person (the “average causal effect”)— our main focus. effect on those most (or least) benefited. effect on those who choose to go into the program.

The Average Causal Effect The average causal effect (or average treatment effect) is the

The Average Causal Effect The average causal effect (or average treatment effect) is the mean value of in the population. p We can think of as a random variable: it has a distribution in the population, and drawing a different person yields a different value of (just like X and Y ). p For example, for person #34 the treatment effect is not random - it is her true treatment effect - but before she is selected at random from the population, her value of 1 can be thought of as randomly distributed. p

The average causal effect, ctd. The average causal effect is E. 1 i /.

The average causal effect, ctd. The average causal effect is E. 1 i /. p What does OLS estimate? p n n When the conditional mean of u given X is zero. Answer: the average treatment effect among those treated. Under the stronger assumption that X is randomly assigned (as in a randomized experiment). Anwser: the average treatment effect (average causal effect). See the following discussion.

OLS with Heterogeneous Causal Effects (a) Suppose E(ui|Xi ) = 0, so Cov(ui ,

OLS with Heterogeneous Causal Effects (a) Suppose E(ui|Xi ) = 0, so Cov(ui , Xi ) = 0. p If X is binary (treated/untreated), p estimates the causal effect among those who receive the treatment. p Why? For those treated, treated reflects the effect of the treatment on them. But we don’t know how the untreated would have responded had they been treated!

The math: suppose X is binary and E(ui|Xi ) = 0. Then For the

The math: suppose X is binary and E(ui|Xi ) = 0. Then For the treated: For the controls: = average effect of the treatment on the treated

OLS with heterogeneous treatment effects: general X with E(ui|Xi) = 0 Without heterogeneity, p

OLS with heterogeneous treatment effects: general X with E(ui|Xi) = 0 Without heterogeneity, p With heterogeneity, if X is binary, this simplifies to the “effect of treatment on the treated. ” p In general, the treatment effects of individuals with large values of X are given the most weight. p

(b) Now make a stronger assumption that X is randomly assigned (experiment or quasi-experiment).

(b) Now make a stronger assumption that X is randomly assigned (experiment or quasi-experiment). Then what does OLS actually estimate? p If Xi is randomly assigned, it is distributed independently of , so there is no difference between the population of controls and the population in the treatment group. p Thus the effect of treatment on the treated = the average treatment effect in the population.

Summary: If Xi and are independent (Xi is randomly assigned), OLS estimates the average

Summary: If Xi and are independent (Xi is randomly assigned), OLS estimates the average treatment effect. p If Xi is not randomly assigned but E(ui|Xi) = 0, OLS estimates the effect of treatment on the treated. p Without heterogeneity, the effect of treatment on the treated and the average treatment effect are the same. p

IV Regression with Heterogeneous Causal Effects Suppose the treatment effect is heterogeneous and the

IV Regression with Heterogeneous Causal Effects Suppose the treatment effect is heterogeneous and the effect of the instrument on X is heterogeneous: (equation of interest) (first stage of TSLS) In general, TSLS estimates the causal effect for those whose value of X (probability of treatment) is most influenced by the instrument— something called the Local Average Treatment Effect (LATE).

IV with heterogeneous causal effects, ctd. Intuition: p Suppose ’s were known. If for

IV with heterogeneous causal effects, ctd. Intuition: p Suppose ’s were known. If for some people = 0, then their predicted value of Xi wouldn’t depend on Z, so the IV estimator would ignore them. p The IV estimator puts most of the weight on individuals for whom Z has a large influence on X. p TSLS measures the treatment effect for those whose probability of treatment is most influenced by X.

The math: To simplify things, suppose: p and are distributed independently of p =

The math: To simplify things, suppose: p and are distributed independently of p = 0 and = 0. p , then (derived later) p TSLS estimates the causal effect for those individuals for whom Z is most influential (those with large ).

Derivation of because since , and

Derivation of because since , and

p where since p and are independent; because , and Z are independent; p

p where since p and are independent; because , and Z are independent; p because p independent of p Therefore, . and are

The Local Average Treatment Effect (LATE): TSLS estimates the causal effect for those individuals

The Local Average Treatment Effect (LATE): TSLS estimates the causal effect for those individuals for whom Z is most influential (those with large ) p The limit, , is called the local average treatment effect—it is the average treatment effect for those in a “local” region who are most heavily influenced by the IV Z. p In general, LATE is neither the average causal effect (a. k. a. average treatment effect) nor the effect of treatment on the treated. p

Recall the covariance fact, so

Recall the covariance fact, so

If the treatment effect is large for individuals for whom the effect of the

If the treatment effect is large for individuals for whom the effect of the instrument is also large, then > 0 and LATE > ATE (if > 0). p In the binary case, LATE is the treatment effect for those whose probability of receipt of treatment is most heavily influenced by Z. p n If you always (or never) get treated, you don’t show up in limit of the IV estimator (in LATE).

When there are heterogeneous causal effects, what TSLS estimates depends on the choice of

When there are heterogeneous causal effects, what TSLS estimates depends on the choice of instruments! With different instruments, TSLS estimates different weighted averages!!! p Suppose you have two instruments, Z 1 and Z 2. p n n n In general these instruments will be influential for different members of the population. Using Z 1, TSLS will estimate the treatment effect for those people whose probability of treatment (X) is most influenced by Z 1. The treatment effect for those most influenced by Z 1 might differ from the treatment effect for those most influenced by Z 2.

When does TSLS estimate the average causal effect? p LATE= the average causal effect,

When does TSLS estimate the average causal effect? p LATE= the average causal effect, that is, if : n n n p If If If and are independent. = (no heterogeneity in equation of interest). = (no heterogeneity in first stage equation). But in general does not estimate !

Example: Cardiac catheterization Yi = survival time (days) for AMI patients Xi = received

Example: Cardiac catheterization Yi = survival time (days) for AMI patients Xi = received cardiac catheterization (or not) Zi = differential distance to CC hospital Equation of interest: First stage (linear probability model): For whom does distance have the great effect on the probability of treatment? p For those patients, what is their causal effect ? p

Equation of interest: First stage (linear probability model): TSLS estimates the causal effect for

Equation of interest: First stage (linear probability model): TSLS estimates the causal effect for those whose value of Xi is most heavily influenced by Zi. p TSLS estimates the causal effect for those for whom distance most influences the probability of treatment. p What is their causal effect? p This is one explanation of why the TSLS estimate is smaller than the clinical trial OLS estimate. p

Heterogeneous Causal Effects: Summary Heterogeneous causal effects means that the causal (or treatment) effect

Heterogeneous Causal Effects: Summary Heterogeneous causal effects means that the causal (or treatment) effect varies across individuals. p When these differences depend on observable variables, heterogeneous causal effects can be estimated using interactions (nothing new here). p When these differences are unobserved ( ) the average causal (or treatment) effect is the average value in the population, E( ). p When causal effects are heterogeneous, OLS and TSLS estimate are different. p

TSLS with Heterogeneous Causal Effects TSLS estimates the causal effect for those individuals for

TSLS with Heterogeneous Causal Effects TSLS estimates the causal effect for those individuals for whom Z is most influential (those with large ). p What TSLS estimates depends on the choice of Z!! p In CC example, these were the individuals for whom the decision to drive to a CC lab was heavily influenced by the extra distance. p Thus TSLS also estimates a causal effect: the average effect of treatment on those most influenced by the instrument. p n In general, this is neither the average causal effect nor the effect of treatment on the treated.

Summary: Experiments and Quasi. Experiments Average causal effects are defined as expected values of

Summary: Experiments and Quasi. Experiments Average causal effects are defined as expected values of ideal randomized controlled experiments. p Actual experiments have threats to internal validity. p These threats to internal validity can be addressed (in part) by: p n n n panel methods (differences-in-differences) multiple regression IV (using initial assignment as an instrument)

Quasi-experiments: Quasi-experiments have an “as-if” randomly assigned source of variation. p This as-if random

Quasi-experiments: Quasi-experiments have an “as-if” randomly assigned source of variation. p This as-if random variation can generate: p n n p Xi which satisfies E(ui|Xi) = 0 (so estimation proceeds using OLS); or instrumental variable(s) which satisfy E(ui|Zi) = 0 (so estimation proceeds using TSLS) Quasi-experiments also have threats to internal vaidity.

Two additional subtle issues: p What is a control variable? n n A variable

Two additional subtle issues: p What is a control variable? n n A variable W for which X and u are uncorrelated, given the value of W (conditional mean independence: E(ui|Xi , Wi) = E(ui|Wi). Example: STAR & effect of teacher experience within their school, teachers were randomly assigned to regular/reg+aide/small class p OLS provides an unbiased estimator of the causal effect, but only after controlling for school effects. p

Summary, ctd. What do OLS and TSLS estimate when there is unobserved heterogeneity of

Summary, ctd. What do OLS and TSLS estimate when there is unobserved heterogeneity of causal effects? p In general, weighted averages of causal effects. p If X is randomly assigned, then OLS estimates the average causal effect. p If Xi is not randomly assigned but E(ui|Xi) = 0 , OLS estimates the average effect of treatment on the treated. p If E(ui|Zi) = 0, TSLS estimates the average effect of treatment on those most influenced by Zi. p