Selection Models Evaluation Research 8521 Prof Jesse Lecy

Selection Models Evaluation Research (8521) Prof. Jesse Lecy 1

The “selection” problem Program Outcome Choice to participate When happens when people are given the choice to participate in a program? When do problems arise? How do we estimate program impact in this case? 2

Microfinance example of bias from selection Number of people per category Not an Entrepreneur No Loan 30 15 Takes a Loan 20 35 Average weekly income per category Not an Entrepreneur No Loan $10 $20 Takes a Loan $10 $20 The loan appears to have an impact even when we know it didn’t! 3

Charter school example 4

Propensity score matching Program Outcome This is the “propensity” Selection bias is just another case of omitted variable bias 5

Propensity score matching ID# Age Group Outcome Program? #1 40 Blue $75 0 #2 43 Blue $64 1 #3 54 Blue $69 0 #4 23 Red $32 0 #5 41 Red $49 0 #6 38 Red $87 1 6

http: //www. youtube. com/watch? v=p. Nn_o. Jw. Z 2 sg&feature=related 7

Probability Models Linear Probability Model (OLS) Program? Logit Model Program? 1 1 0 0 Age These models predict the probability of participation in the program, i. e. the “propensity” of an individual 8

Propensity score matching ID# Age Race Outcome Program? Predicted Propensity #1 40 Blue $75 0 0. 87 #2 43 Blue $64 1 0. 98 #3 54 Blue $69 0 0. 45 #4 23 Red $32 0 0. 53 #5 41 Red $49 0 1. 02 #6 38 Red $87 1 1. 11 We add a variable for the predicted propensity score. Remember not to use your outcome in your propensity score calculations. 9

Propensity score matching ID# Age Race Outcome Program? Predicted Propensity #1 40 Blue $75 0 0. 87 #2 43 Blue $64 1 0. 98 #3 54 Blue $69 0 0. 45 #4 23 Red $32 0 0. 53 #5 41 Red $49 0 1. 02 #6 38 Red $87 1 1. 11 Then matches are made to form a synthetic control group: #5 <--> #6 #2 <--> #1 10

Harry Potter Example Propensity Score 11

Harry Potter Example 12

Harry Potter Example 13

How do we know matching has been successful? 14

What are the problems with Propensity Scores? http: //www. youtube. com/watch? v=Zg 7 gx. AG 8 YS 4&feature=related 15

What are the problems with Propensity Scores? You can only match on observables! http: //www. youtube. com/watch? v=Zg 7 gx. AG 8 YS 4&feature=related 16

Marriage and happiness (Stutzer & Frey 2006) “Marriage is one of the most important institutions affecting people’s life and wellbeing…This commitment has positive effects, for instance on spouses’ health and their earnings on the labor market. ” Married people are happier. . . So we should get married to be happy, right? 17

Marriage and happiness “Is marriage an institution for the happy and joyful crowd that finds a partner? This question summarizes the selection hypothesis in research on marriage and wellbeing. It proposes that those who get married are intrinsically happier people. ” “In order to test the selection hypothesis, we follow a simple approach and compare two different groups of singles. The level of subjective well-being of singles who marry later in life is contrasted with the well-being of those who stay single, controlling for numerous observable characteristics. ” For any given age, a comparison of the average life satisfaction in these two groups indicates systematic heterogeneity to some extent. However, it has to be taken into consideration that the years immediately before marriage might not be representative for a person’s intrinsic happiness level. People might live in a marriage-like relation, as cohabitants, thinking and planning their joint future in a loving relationship. As these years end in marriage, they are more likely to be the best years in life. 18

Marriage and happiness Married Young Married Later Never Married 19

Marriage and happiness Does this gap represent the happiness bump? 20

Marriage and happiness “For any given age, a comparison of the average life satisfaction in these two groups indicates systematic heterogeneity to some extent. However, it has to be taken into consideration that the years immediately before marriage might not be representative for a person’s intrinsic happiness level. People might live in a marriage-like relation, as cohabitants, thinking and planning their joint future in a loving relationship. As these years end in marriage, they are more likely to be the best years in life. ” 21

Marriage and happiness Bummer…