Multiple Regression II 41112 Categorical explanatory variables Adjusted

Multiple Regression II 4/11/12 • Categorical explanatory variables • Adjusted R 2 Not in book Professor Kari Lock Morgan Duke University

Project 1 Regrade Requests • Regrade requests must be submitted in writing, with the original project, by Friday, 4/13/12, at 5 pm • If regraded, I will grade the entire project, and the grade may go up or down

To Do • Project 2 Proposal (due TODAY, 5 pm) • Homework 9 (due Monday, 4/16) • Project 2 Presentation (Thursday, 4/19) • Project 2 Paper (Wednesday, 4/25)

Chapter 9 Odd Solutions • Chapter 9 odd solutions available

Lab • Your group members are in the same lab (mostly) so this is a great time to meet with your group in person • There is no “On Your Own” and I’ve tried to make the lab short, so you should have some time with your group to get started with your project • If you want to work through the lab on your own before coming to class, you can have the whole lab time with your group to work on the project • Come to lab tomorrow!

US States • We will build a model to predict the % of the state that voted Republican in the 2008 US presidential election, using the 50 states as cases • Sample? Population? • This can help us to understand how certain features of a state are associated with political beliefs

US States • Response Variable: • Our first explanatory variable is region of the country: Midwest, Northeast, South, or West

Categorical Variables • For this to make any sense, each x value has to be a number. • How do we include categorical variables in a regression setting?

Categorical Variables • Take one categorical variable, and replace it with several “dummy” variables • A dummy variable is 1 if the case falls into the category represented by the dummy variable, and 0 otherwise • Create one dummy variable for each category of the categorical variable

Dummy Variables State Region South West Northeast Midwest Alabama South 1 0 0 0 Alaska West 0 1 0 0 Arkansas South 1 0 0 0 California West 0 1 0 0 Colorado West 0 1 0 0 Connecticut Northeast 0 0 1 0 Delaware Northeast 0 0 1 0 Florida South 1 0 0 0 Georgia South 1 0 0 0 Hawaii West 0 1 0 0 … … …

Dummy Variables • When using dummy variables, one has to be left out of the model • The dummy variable left out is called the reference level • When using region of the country (Northeast, South, Midwest, West) to predict % Mc. Cain vote, how many dummy variables will be included? a) One b) Two c) Three d) Four

Dummy Variables • Predicting % vote for Mc. Cain with one categorical variable: region of the country • If “midwest” is the reference level:

Voting by Region Based on the output above, which region had the highest percent vote for Mc. Cain? a) b) c) d) Midwest Northeast South West

Voting by Region What is the predicted % Republican vote for a state in the northeast? a) b) c) d) – 10. 2% 48. 6% 38. 4% 58. 8%

Voting by Region What is the predicted % Republican vote for a state in the midwest? a) b) c) d) 50% 48. 6% 0% 58. 8%

Categorical Variables • The p-value for each dummy variable tests for a significant difference between that category and the reference level • For an overall p-value for the significance of the categorical variable with multiple categories, use a) b) c) d) z-test T-test Chi-square test ANOVA

Categorical Variables

Categorical Variables in R • R automatically creates dummy variables for you if you include a categorical explanatory variable • The first level alphabetically is usually the reference level • If you want to change the reference level, see me

Categorical Variables • Either all dummy variables associated with a categorical variable have to be included in the model, or none of them • Region. W is not significant, but leaving it out would clump West with the reference level, Midwest, which does not make sense

Variables • Let’s include some more explanatory variables! • What helps to predict % voting Republican?

Categorical Variables • Be careful not to include a categorical variable for which every case is it’s own category • Example: using “State” as an explanatory variable would be silly, even though R 2 = 1! • If you want to know how each state voted, it would make more sense to just look directly at Mc. Cain. Vote, rather than fitting a model and giving each state it’s own coefficient

Explanatory Variables • Also, be careful not to include explanatory variables that are essentially just another form of the response variable • For example, Obama. Mc. Cain is “M” if the state went for Mc. Cain, and “O” if the state went for Obama • This is certainly associated with the % of people in the state that voted for Mc. Cain, but tells us nothing interesting

Explanatory Variables • Models should be created either to learn about relationships between explanatory variables and the response, or for prediction • Make sure the explanatory variables you include in the model are not contradicting the point of the model

Visualization How would we visualize the association between region and % vote for Mc. Cain? a) b) c) d) e) Scatterplot Side-by-side boxplots Bar chart Pie chart Mosaic plot

Side-by-Side Boxplots

Test for Association How would we test for an association between region and % vote for Mc. Cain? a) b) c) d) e) t-test for difference in means test for a correlation ANOVA chi-square test for a difference in proportions

ANOVA

Visualization All of the other potential explanatory variables are quantitative. How would we visualize the association between each of them and % vote for Mc. Cain? a) b) c) d) e) Scatterplot Side-by-side boxplots Bar chart Pie chart Mosaic plot

What do you see?

Test for Association How would we test the association between each of these variables and % vote for Mc. Cain? a) b) c) d) e) t-test for difference in means test for a correlation ANOVA chi-square test for a difference in proportions

Test for Correlation

Regression Model

Physical Activity Given all the other variables in the model, states with a higher percentage of physically active citizens are more likely to vote (a) Republican (b) Democratic

West Region • With only region as an explanatory variable, interpret the meaning of the negative coefficient of Region. W. In this data set, states in the West voted less Republican than states in the Midwest. • With all the other explanatory variables included, interpret the meaning of the positive coefficient of Region. W. States in the West voted more Republican than would be expected based on the other variables in the model, as compared to states in the Midwest.

Goal of the Model? • If the goal of the model is to see what and how each variable is associated with a state’s voting patterns, given all the other variables in the model, then we are done • If the goal is to predict the % of the vote that will be Republican, say in the 2012 election, we want to prune out insignificant variables to improve the model

Over-fitting • It is possible to over-fit a model: to include too many explanatory variables • The fewer the coefficients being estimated, the better they will be estimated • Usually, a good model has pruned out explanatory variables that are not helping

R 2 • Adding more explanatory variables will only make R 2 increase or stay the same • Adding another explanatory variable can not make the model explain less, because the other variables are all still in the model • Is the best model always the one with the highest proportion of variability explained, and so the highest R 2? (a) Yes (b) No

Adjusted 2 R • Adjusted R 2 is like R 2, but takes into account the number of explanatory variables • As the number of explanatory variables increases, adjusted R 2 gets smaller than R 2 • One way to choose a model is to choose the model with the highest adjusted R 2

Adjusted R 2 You now know how to interpret all of these numbers!