HAWKES LEARNING SYSTEMS math courseware specialists Copyright 2010

HAWKES LEARNING SYSTEMS math courseware specialists Copyright © 2010 by Hawkes Learning Systems/Quant Systems, Inc. All rights reserved. Chapter 13 Regression, Inference, and Model Building

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 1 Assumptions of the Simple Linear Model Objectives: • To define the assumptions of a simple linear regression equation.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 1 Assumptions of the Simple Linear Model Definitions: • The population regression line is given by the formula: • The sample regression line is given by the formula: where b 1 and b 0 are estimates of their population counterparts. b 0 is an estimate of b 1 is an estimate of , and. • Incorporating error terms produces the simple linear regression model:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 1 Assumptions of the Simple Linear Model Definitions: • The error term, εi, represents the variation in the y variable not accounted for by the linear regression model. • Assumptions about the error term in the linear model: o The εi are presumed to be normally distributed with a mean of 0 and a variance of. o The εi are presumed to be independent.

HAWKES LEARNING SYSTEMS math courseware specialists Definitions: • is estimated by the parameter Regression, Inference, and Model Building Section 13. 1 Assumptions of the Simple Linear Model

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Objectives: • To construct confidence intervals of the slope of a simple linear regression equation.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Definitions: • Confidence Interval for β 1: • The sample estimate of the variance of b 1 is given by: • Therefore the standard deviation of the sample estimate of b 1 is:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Definitions: • The confidence interval for β 1 is given by where is the critical value for a t-distribution with n-2 degrees of freedom. The expression above creates the following interval.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: In a linear regression model there are 20 observations. A 95% confidence interval is required. Determine the critical value for the confidence interval. Solution: Since and if then The critical value is given by

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: In the table to the right the left hand column is the ages of Saturn SL 1 s and the right hand column is the corresponding asking prices. Construct a simple linear regression model that uses the age of Saturn SL 1 s to predict their price. Then construct a 95% confidence interval for β 1. Solution: Asking price v. Age (Years) Asking Price 1. 0 $11, 875 1. 0 10, 995 2. 0 9, 995 2. 0 8, 500 3. 0 8, 995 The sample coefficients for β 0 and β 1 are $12, 519. 30 and -$1391. 09 respectively. 4. 0 6, 995 5. 0 4, 450 Therefore the simple linear regression model is: 5. 0 5, 500 6. 0 4, 400 6. 0 4, 800

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Construct a simple linear regression model that uses the age of Saturn SL 1 s to predict their price. Then construct a 95% confidence interval for β 1. Solution: Since and if then The critical value is given by

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Construct a simple linear regression model that uses the age of Saturn SL 1 s to predict their price. Then construct a 95% confidence interval for β 1. Solution: We have already found b 1 and the critical value, the only value left to be found in the formula is the standard deviation of b 1. The sum of the squared errors, The variance of errors, , is 4, 284, 979.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Construct a simple linear regression model that uses the age of Saturn SL 1 s to predict their price. Then construct a 95% confidence interval for β 1. Solution: Since the standard deviation of b 1, is given by and

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Construct a simple linear regression model that uses the age of Saturn SL 1 s to predict their price. Then construct a 95% confidence interval for β 1. Solution: Now we can use the formula: Which yields the interval:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Using the same data from the previous example, construct a 99% confidence interval. Solution: The only value from the formula That will change is the critical value, Since and if then The critical value is given by .

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 2 Inference – How Good is the Estimate of β 1? Confidence Interval: Using the same data from the previous example, construct a 99% confidence interval. Solution: Now we can use the formula: Which yields the interval:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Objectives: • Construct prediction intervals and compare them with confidence intervals for the average value of Y given X.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Definition: • Confidence interval for the Average Value of Y given X: is the predicted value of Y when is the critical value associated with confidence level, is the standard deviation of the error terms, and Measures how far xp is away from in relation to the total variation of the X. The further that xp is from , the larger this ratio will become and consequently the wider the confidence interval.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Definition: • Steps to calculate the confidence interval for the average value of Y given X: o Use the estimated regression line to calculate given for X = xp, for the value o Find a t-value corresponding to the level of confidence and the degrees of freedom associated with the data used to estimate the model, o Determine the standard deviation of the error terms, o Compute the term

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Confidence Interval: In the table to the right the left hand column is the ages of Saturn SL 1 s and the right hand column is the corresponding asking prices. Construct a 95% confidence interval for the average value of the Price when the Age equals two years. Asking price v. Age (Years) Asking Price 1. 0 $11, 875 1. 0 10, 995 Solution: 2. 0 9, 995 The simple linear regression model is: 2. 0 8, 500 3. 0 8, 995 4. 0 6, 995 5. 0 4, 450 5. 0 5, 500 6. 0 4, 400 6. 0 4, 800 The predicted value of Y when xp = 2 is:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Confidence Interval: Construct a 95% confidence interval for the average value of the Price when the Age equals two years. Solution: The standard deviation of the error terms is: For a 95% confidence interval, and the degrees of freedom = n − 2 = 10 − 2 = 8. Therefore, The term

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Confidence Interval: Construct a 95% confidence interval for the average value of the Price when the Age equals two years. Solution: Now that all of the necessary terms have been gathered,

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Definition: • Confidence interval for the predicted value of Y given X: is the predicted value of Y when is the critical value associated with confidence level, is the standard deviation of the error terms, and Measures how far xp is away from in relation to the total variation of the X. The further that xp is from , the larger this ratio will become and consequently the wider the confidence interval.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Definition: • Steps to calculate the confidence interval for the predicted value of Y given X: o Use the estimated regression line to calculate given for X = xp, for the value o Find a t-value corresponding to the level of confidence and the degrees of freedom associated with the data used to estimate the model, o Determine the standard deviation of the error terms, o Compute the term

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Interval: In the table to the right the left hand column is the ages of Saturn SL 1 s and the right hand column is the corresponding asking prices. Construct a 95% prediction interval for the Price when the Age equals two years. Asking price v. Age (Years) Asking Price 1. 0 $11, 875 1. 0 10, 995 Solution: 2. 0 9, 995 The simple linear regression model is: 2. 0 8, 500 3. 0 8, 995 4. 0 6, 995 5. 0 4, 450 5. 0 5, 500 6. 0 4, 400 6. 0 4, 800 The predicted value of Y when xp = 2 is:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Interval: Construct a 95% prediction interval for the average value of the Price when the Age equals two years. Solution: The standard deviation of the error terms is: For a 95% prediction interval, , and the degrees of freedom = n − 2 = 10 − 2 = 8. Therefore, the critical value is The term

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 4 Inference Concerning a Model’s Prediction Interval: Construct a 95% prediction interval for the Price when the Age equals two years. Solution: Now that all of the necessary terms have been gathered,

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Objectives: • To define and describe a hypothesis test for a multiple regression equation. • To define the F-test statistic of a multiple regression hypothesis test. • To conduct a hypothesis test for a multiple regression equation. • To learn how to analyze a regression output to construct a confidence interval of a single coefficient in a multiple regression model. • To learn how to analyze a regression output to conduct a hypothesis test of a single coefficient in a multiple regression model.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Definition: • The multiple regression model is defined by the equation each βi represents the coefficient of a independent variable xi. • To test if the multiple regression model is valid we test to see if all of the coefficients equal zero: • The test is a F-test where the null and alternative hypotheses are: • The null hypothesis (in words) states that all of the coefficients equal zero, while the alternative hypothesis states at least one of the coefficients is not equal to zero.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Steps for Hypothesis Testing Multiple Regression: 1) (Define the hypothesis in plain English. ) Is the overall model useful in explaining variation in the dependent variable. 2) (Select the appropriate statistical measure to rephrase the hypothesis. ) If some of the model’s independent variables are useful predictors of Y, then at least one of the coefficients, βi, of these variables will have a non-zero value. 3) (Determine whether the hypothesis should be one-sided or two-sided. ) The F-test for multiple regression will always be one-sided. 4) (State the hypothesis using the statistical measure selected in Step 2. )

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Steps for Hypothesis Testing Multiple Regression: 5) Specify the significance level of the test 6) Select the appropriate test statistic 7) Determine the necessary critical value. 8) Collect sample data and determine the test-statistic. 9) Make the decision. Is the value of the test statistic in the rejection region? 10) State the conclusion in terms of the original question. Anyone trying to build a model for predictive purposes hopes the null hypothesis is rejected, since rejecting the null implies the model can explain some of the variation in the dependent variable.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Definition: • F-statistic for multiple regression. Where, k = the number of independent variables, and n = the number of observations.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. The multiple regression equation for the model is given below along with the ANOVA table for the regression model. Conduct a hypothesis test at the 0. 05 level to determine the model’s usefulness as a whole in explaining the variation in home prices.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test at the 0. 05 level to determine the model’s usefulness as a whole in explaining the variation in home prices. Solution: 1) H 0: The overall model as a whole is useful in explaining the variation in home prices. Ha: The overall model as a whole is not useful in explaining the variation in home prices. 2) If the model’s independent variables are useful predictors of Y, then at least one of the coefficients, , will have a non zero value. 3) A hypothesis test for model usefulness is always a one-sided test.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test at the 0. 05 level to determine the model’s usefulness as a whole in explaining the variation in home prices. Solution: 4) 5) 6) The appropriate test statistic is the F-statistic. 7) Since 8) , k = 3, n = 31, and the test is one-sided,

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test at the 0. 05 level to determine the model’s usefulness as a whole in explaining the variation in home prices. Solution: 9) Since , reject the null hypothesis. 10) There is significant evidence at the 0. 05 level that the overall multiple regression model is useful in explaining the variation in home prices.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: The rejection region can be graphed on the real number line, And then the test statistic can be compared against the rejection region on the real number line.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Definition: • The confidence interval for an individual bi in a multiple regression equation is similar to that of simple linear regression confidence intervals for the slope. It is defined by: • • The standard deviation of the independent variable can be found in the column labeled “SE Coef” in a Minitab output and in the column labeled “Standard Error” for an Excel output.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Confidence Interval: A model for a home’s price is based on the square footage, and the number of bedrooms. Construct a 95% confidence interval for the independent variable “Square Footage” (b 1). The necessary summary output is given below along with the multiple regression model.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Confidence Interval: A model for a home’s price is based on the square footage, and the number of bedrooms. Construct a 95% confidence interval for the independent variable “Square Footage” (b 1). Solution: Since and if then The critical value is given by The standard deviation of the variable (from the summary output) is given as 9. 318.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Confidence Interval: A model for a home’s price is based on the square footage, and the number of bedrooms. Construct a 95% confidence interval for the independent variable “Square Footage” (b 1). Solution: We are 95% confident that the true value of β 1, the increase in the price of a house per square foot given a particular number of bedrooms and age of the home, will be between $48. 08 and $86. 32.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Definition: • Sometimes while an F-test of a multiple regression model as a whole will validate the model, there may be individual independent variables that are statistically insignificant. • Hypothesis testing an individual independent variable (in multiple or simple regression) is a two-sided t-Test given by the formula: where, • The null and alternative hypothesis are:

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test of the variable “Age of Home” at the 0. 05 level. The necessary summary output is below along with the multiple regression model.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test of the variable “Age of Home” at the 0. 05 level. Solution: 1) H 0: The variable “Age of Home” is a useful predictor of the price of home. Ha: The variable “Age of Home” is not a useful predictor of the price of home. 2) If age is not a useful predictor of home price, then the sample estimate 3) The hypothesis should be two-sided since the relationship between the age of the home and home price can be positive or negative.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test of the variable “Age of Home” at the 0. 05 level. Solution: 4) 5) The significant level of the test given in the problem is 6) The appropriate test is a t-test and the appropriate test statistic is t. 7) Since , n = 31, k = 3, and the test is two-sided, 8) The test statistic is

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model Hypothesis Testing: A model for a home’s price is based on the square footage, and the number of bedrooms. Conduct a hypothesis test of the variable “Age of Home” at the 0. 05 level. Solution: 9) Since , fail to reject the null hypothesis. 10) There is not significant evidence at the 0. 05 level that the variable “Age of Home” is a useful predictor of the price of a home given the other variables currently in the model.

HAWKES LEARNING SYSTEMS math courseware specialists Regression, Inference, and Model Building Section 13. 8 Inference Concerning the Coefficients of the Multiple Regression Model The rejection regions can be graphed on the real number line, And then the test statistic can be compared against the rejection regions on the real number line.