Chapter 11 Analyzing Association Between Quantitative Variables Regression
Chapter 11 Analyzing Association Between Quantitative Variables: Regression Analysis l Learn…. To use regression analysis to explore the association between two quantitative variables Agresti/Franklin Statistics, 1 of 88
Ø Section 11. 1 How Can We “Model” How Two Variables Are Related? Agresti/Franklin Statistics, 2 of 88
Regression Analysis l The first step of a regression analysis is to identify the response and explanatory variables • We use y to denote the response variable • We use x to denote the explanatory variable Agresti/Franklin Statistics, 3 of 88
The Scatterplot l The first step in answering the question of association is to look at the data l A scatterplot is a graphical display of the relationship between two variables Agresti/Franklin Statistics, 4 of 88
Example: What Do We Learn from a Scatterplot in the Strength Study? l l An experiment was designed to measure the strength of female athletes The goal of the experiment was to find the maximum number of pounds that each individual athlete could bench press Agresti/Franklin Statistics, 5 of 88
Example: What Do We Learn from a Scatterplot in the Strength Study? l l 57 high school female athletes participated in the study The data consisted of the following variables: • x: • the number of 60 -pound bench presses an athlete could do y: maximum bench press Agresti/Franklin Statistics, 6 of 88
Example: What Do We Learn from a Scatterplot in the Strength Study? l For the 57 girls in this study, these variables are summarized by: • x: • y: mean = 11. 0, st. deviation = 7. 1 mean = 79. 9 lbs, st. dev. = 13. 3 lbs Agresti/Franklin Statistics, 7 of 88
Example: What Do We Learn from a Scatterplot in the Strength Study? Agresti/Franklin Statistics, 8 of 88
The Regression Line Equation l l When the scatterplot shows a linear trend, a straight line fitted through the data points describes that trend The regression line is: is the predicted value of the response variable y is the y-intercept and is the slope Agresti/Franklin Statistics, 9 of 88
Example: Which Regression Line Predicts Maximum Bench Press? “t” test for slope t=(b-0)/se df=N-2 Agresti/Franklin Statistics, 10 of 88
Example: What Do We Learn from a Scatterplot in the Strength Study? l The MINITAB output shows the following regression equation: l BP = 63. 5 + 1. 49 (BP_60) l The y-intercept is 63. 5 and the slope is 1. 49 The slope of 1. 49 tells us that predicted maximum bench press increases by about 1. 5 pounds for every additional 60 -pound bench press an athlete can do l Agresti/Franklin Statistics, 11 of 88
Outliers l Check for outliers by plotting the data l The regression line can be pulled toward an outlier and away from the general trend of points Agresti/Franklin Statistics, 12 of 88
Influential Points l An observation can be influential in affecting the regression line when two thing happen: • Its x value is low or high compared to the • rest of the data It does not fall in the straight-line pattern that the rest of the data have Agresti/Franklin Statistics, 13 of 88
Residuals are Prediction Errors l The regression equation is often called a prediction equation l The difference between an observed outcome and its predicted value is the prediction error, called a residual Agresti/Franklin Statistics, 14 of 88
Residuals l Each observation has a residual l A residual is the vertical distance between the data point and the regression line Agresti/Franklin Statistics, 15 of 88
Residuals l We can summarize how near the regression line the data points fall by l The regression line has the smallest sum of squared residuals and is called the least squares line Agresti/Franklin Statistics, 16 of 88
Regression Model: A Line Describes How the Mean of y Depends on x l At a given value of x, the equation: l Predicts a single value of the response variable l But… we should not expect all subjects at that value of x to have the same value of y • Variability occurs in the y values Agresti/Franklin Statistics, 17 of 88
The Regression Line l The regression line connects the estimated means of y at the various x values l In summary, l Describes the relationship between x and the estimated means of y at the various values of x Agresti/Franklin Statistics, 18 of 88
The Population Regression Equation l l The population regression equation describes the relationship in the population between x and the means of y The equation is: Agresti/Franklin Statistics, 19 of 88
The Population Regression Equation l In the population regression equation, α is a population y-intercept and β is a population slope • These are parameters l In practice we estimate the population regression equation using the prediction equation for the sample data Agresti/Franklin Statistics, 20 of 88
The Population Regression Equation l l l The population regression equation merely approximates the actual relationship between x and the population means of y It is a model A model is a simple approximation for how variable relate in the population Agresti/Franklin Statistics, 21 of 88
The Regression Model Agresti/Franklin Statistics, 22 of 88
The Regression Model l If the true relationship is far from a straight line, this regression model may be a poor one Agresti/Franklin Statistics, 23 of 88
Variability about the Line l At each fixed value of x, variability occurs in the y values around their mean, µy l The probability distribution of y values at a fixed value of x is a conditional distribution l At each value of x, there is a conditional distribution of y values l An additional parameter σ describes the standard deviation of each conditional distribution Agresti/Franklin Statistics, 24 of 88
A Statistical Model l A statistical model never holds exactly in practice. l It is merely a simple approximation for reality l Even though it does not describe reality exactly, a model is useful if the true relationship is close to what the model predicts Agresti/Franklin Statistics, 25 of 88
For recent data on several nations, the prediction equation relating y = fertility rate to x = female economic activity (the female labor force as a percentage of the male labor force) is: Find the predicted fertility for Vietnam, which had the highest value of x = 91. a. 5. 25 b. 469. 2 c. 1. 196 d. 10. 73 Agresti/Franklin Statistics, 26 of 88
For recent data on several nations, the prediction equation relating y = fertility rate to x = female economic activity (the female labor force as a percentage of the male labor force) is: Find the residual for Vietnam, which had y = 2. 3. a. -2. 136 b. 1. 104 c. -1. 104 d. 2. 136 Agresti/Franklin Statistics, 27 of 88
Ø Section 11. 2 How Can We Describe Strength of Association? Agresti/Franklin Statistics, 28 of 88
Correlation l l The correlation, denoted by r, describes linear association The correlation ‘r’ has the same sign as the slope ‘b’ The correlation ‘r’ always falls between -1 and +1 The larger the absolute value of r, the stronger the linear association Agresti/Franklin Statistics, 29 of 88
Correlation and Slope l We can’t use the slope to describe the strength of the association between two variables because the slope’s numerical value depends on the units of measurement Agresti/Franklin Statistics, 30 of 88
Correlation and Slope l l The correlation is a standardized version of the slope The correlation does not depend on units of measurement Agresti/Franklin Statistics, 31 of 88
Correlation and Slope l The correlation and the slope are related in the following way: *** Exam I: xbar and ybar reminder Agresti/Franklin Statistics, 32 of 88
Example: What’s the Correlation for Predicting Strength? l l For the female athlete strength study: • • x: y: number of 60 -pound bench presses maximum bench press mean = 11. 0, st. dev. =7. 1 mean= 79. 9 lbs. , st. dev. = 13. 3 lbs. Regression equation: Agresti/Franklin Statistics, 33 of 88
Example: What’s the Correlation for Predicting Strength? l The variables have a strong, positive association Agresti/Franklin Statistics, 34 of 88
The Squared Correlation l Another way to describe the strength of association refers to how close predictions for y tend to be to observed y values l The variables are strongly associated if you can predict y much better by substituting x values into the prediction equation than by merely using the sample mean y and ignoring x Agresti/Franklin Statistics, 35 of 88
The Squared Correlation l Consider the prediction error: the difference between the observed and predicted values of y • Using the regression line to make a prediction, each error is: • Using only the sample mean, y, to make a prediction, each error is: Agresti/Franklin Statistics, 36 of 88
The Squared Correlation l When we predict y using y (that is, ignoring x), the error summary equals: l This is called the total sum of squares Agresti/Franklin Statistics, 37 of 88
The Squared Correlation l When we predict y using x with the regression equation, the error summary is: l This is called the residual sum of squares Agresti/Franklin Statistics, 38 of 88
The Squared Correlation l When a strong linear association exists, the regression equation predictions tend to be much better than the predictions using y l We measure the proportional reduction in error and call it, r 2 Agresti/Franklin Statistics, 39 of 88
The Squared Correlation l We use the notation r 2 for this measure because it equals the square of the correlation r Agresti/Franklin Statistics, 40 of 88
Example: What Does r 2 Tell Us in the Strength Study? l For the female athlete strength study: • • • x: number of 60 -pund bench presses y: maximum bench press The correlation value was found to be r = 0. 80 l We can calculate r 2 from r: (0. 80)2=0. 64 l For predicting maximum bench press, the regression equation has 64% less error than y has Agresti/Franklin Statistics, 41 of 88
Correlation r and Its Square r 2 l Both r and r 2 describe the strength of association l ‘r’ falls between -1 and +1 l • It represents the slope of the regression line when x and y have been standardized ‘r 2’ falls between 0 and 1 • It summarizes the reduction in sum of squared errors in predicting y using the regression line instead of using y Agresti/Franklin Statistics, 42 of 88
All Students who attend Lake Woebegone College must take the math and verbal SAT exams. Both exams have a mean of 500 and a standard deviation of 100. The regression equation relating y = math SAT score and x = verbal SAT score is: Find the predicted math SAT score for a student who has the verbal SAT score of 800. a. 250 b. 500 c. 650 d. 750 See example 10 on pg 544 Agresti/Franklin Statistics, 43 of 88
All Students who attend Lake Woebegone College must take the math and verbal SAT exams. Both exams have a mean of 500 and a standard deviation of 100. The regression equation relating y = math SAT score and x = verbal SAT score is: Find the r-value. a. . 5 b. . 25 c. 1. 00 d. . 75 Agresti/Franklin Statistics, 44 of 88
All Students who attend Lake Woebegone College must take the math and verbal SAT exams. Both exams have a mean of 500 and a standard deviation of 100. The regression equation relating y = math SAT score and x = verbal SAT score is: Find the r 2 value. a. . 5 b. . 25 c. 1. 00 d. . 75 Agresti/Franklin Statistics, 45 of 88
Ø Section 11. 3 How Can We make Inferences About the Association? Agresti/Franklin Statistics, 46 of 88
Descriptive and Inferential Parts of Regression l The sample regression equation, r, and r 2 are descriptive parts of a regression analysis l The inferential parts of regression use the tools of confidence intervals and significance tests to provide inference about the regression equation, the correlation and r-squared in the population of interest Agresti/Franklin Statistics, 47 of 88
Assumptions for Regression Analysis l Basic assumption for using regression line for description: • The population means of y at different values of x have a straight-line relationship with x, that is: • This assumption states that a straight-line • regression model is valid This can be verified with a scatterplot. Agresti/Franklin Statistics, 48 of 88
Assumptions for Regression Analysis l Extra assumptions for using regression to make statistical inference: • The data were gathered using randomization • The population values of y at each value of x follow a normal distribution, with the same standard deviation at each x value Agresti/Franklin Statistics, 49 of 88
Assumptions for Regression Analysis l Models, such as the regression model, merely approximate the true relationship between the variables l A relationship will not be exactly linear, with exactly normal distributions for y at each x and with exactly the same standard deviation of y values at each x value Agresti/Franklin Statistics, 50 of 88
Testing Independence between Quantitative Variables Suppose that the slope β of the regression line equals 0 Then… l • • The mean of y is identical at each x value The two variables, x and y, are statistically independent: • The outcome for y does not depend on the value of x • It does not help us to know the value of x if we want to predict the value of y Agresti/Franklin Statistics, 51 of 88
Testing Independence between Quantitative Variables Agresti/Franklin Statistics, 52 of 88
Testing Independence between Quantitative Variables Steps of Two-Sided Significance Test about a Population Slope β: 1. Assumptions: l • The population satisfies regression line: • Randomization • The population values of y at each value of x follow a normal distribution, with the same standard deviation at each x value Agresti/Franklin Statistics, 53 of 88
Testing Independence between Quantitative Variables l Steps of Two-Sided Significance Test about a Population Slope β: 2. Hypotheses: H 0: β = 0, Ha: β ≠ 0 3. Test statistic: l Software supplies sample slope b and its se Agresti/Franklin Statistics, 54 of 88
Testing Independence between Quantitative Variables l Steps of Two-Sided Significance Test about a Population Slope β: 4. P-value: Two-tail probability of t test statistic value more extreme than observed: Use t distribution with df = n-2 5. Conclusions: Interpret P-value in context • If decision needed, reject H 0 if P-value ≤ significance level Agresti/Franklin Statistics, 55 of 88
Example: Is Strength Associated with 60 -Pound Bench Press? Agresti/Franklin Statistics, 56 of 88
Example: Is Strength Associated with 60 -Pound Bench Press? l l Conduct a two-sided significance test of the null hypothesis of independence Assumptions: • • • A scatterplot of the data revealed a linear trend so the straight-line regression model seems appropriate The scatter of points have a similar spread at different x values The sample was a convenience sample, not a random sample, so this is a concern Agresti/Franklin Statistics, 57 of 88
Example: Is Strength Associated with 60 -Pound Bench Press? l Hypotheses: H 0: β = 0, Ha: β ≠ 0 l Test statistic: l P-value: 0. 000 Conclusion: An association exists between the number of 60 -pound bench presses and maximum bench press l Agresti/Franklin Statistics, 58 of 88
A Confidence Interval for β l A small P-value in the significance test of H 0: β = 0 suggests that the population regression line has a nonzero slope l To learn how far the slope β falls from 0, we construct a confidence interval: Agresti/Franklin Statistics, 59 of 88
Example: Estimating the Slope for Predicting Maximum Bench Press l Construct a 95% confidence interval for β l Based on a 95% CI, we can conclude, on average, the maximum bench press increases by between 1. 2 and 1. 8 pounds for each additional 60 -pound bench press that an athlete can do Agresti/Franklin Statistics, 60 of 88
Example: Estimating the Slope for Predicting Maximum Bench Press l Let’s estimate the effect of a 10 -unit increase in x: • Since the 95% CI for β is (1. 2, 1. 8), the • 95% CI for 10β is (12, 18) On the average, we infer that the maximum bench press increases by at least 12 pounds and at most 18 pounds, for an increase of 10 in the number of 60 -pound bench presses Agresti/Franklin Statistics, 61 of 88
- Slides: 61