Basic Business Statistics 9 th Edition Chapter 13

Chapter Topics n n n n Types of Regression Models Determining the Simple Linear Regression Equation Measures of Variation Assumptions of Regression and Correlation Residual Analysis Measuring Autocorrelation Inferences about the Slope © 2004 Prentice-Hall, Inc. 2

Chapter Topics n n n (continued) Correlation - Measuring the Strength of the Association Estimation of Mean Values and Prediction of Individual Values Pitfalls in Regression and Ethical Issues © 2004 Prentice-Hall, Inc. 3

Purpose of Regression Analysis n Regression Analysis is Used Primarily to Model Causality and Provide Prediction n n Predict the values of a dependent (response) variable based on values of at least one independent (explanatory) variable Explain the effect of the independent variables on the dependent variable © 2004 Prentice-Hall, Inc. 4

Types of Regression Models Positive Linear Relationship Negative Linear Relationship © 2004 Prentice-Hall, Inc.

Simple Linear Regression Model n n n Relationship between Variables is Described by a Linear Function The Change of One Variable Causes the Other Variable to Change A Dependency of One Variable on the Other © 2004 Prentice-Hall, Inc. 6

Simple Linear Regression Model (continued) Population regression line is a straight line that describes the dependence of the average value (conditional mean) of one variable on the other Population Slope Coefficient Population Y Intercept Dependent (Response) Variable © 2004 Prentice-Hall, Inc. Population Regression Line (Conditional Mean) Random Error Independent (Explanatory) Variable 7

Simple Linear Regression Model (continued) Y (Observed Value of Y) = = Random Error

Linear Regression Equation Sample regression line provides an estimate of the population regression line as well as a predicted value of Y Sample Y Intercept Sample Slope Coefficient Residual Simple Regression Equation (Fitted Regression Line, Predicted Value) © 2004 Prentice-Hall, Inc. 9

Linear Regression Equation (continued) n and are obtained by finding the values of and

Linear Regression Equation (continued) Y X Observed Value © 2004 Prentice-Hall, Inc. 11

Interpretation of the Slope and Intercept is the average value of Y when n the value of X is zero measures the change in n the average value of Y as a result of a oneunit change in X © 2004 Prentice-Hall, Inc. 12

Interpretation of the Slope and Intercept (continued) is the estimated average n value of Y when the value of X is zero is the estimated change n in the average value of Y as a result of a oneunit change in X © 2004 Prentice-Hall, Inc. 13

Simple Linear Regression: Example You wish to examine the linear dependency of the annual sales of produce stores on their sizes in square footage. Sample data for 7 stores were obtained. Find the equation of the straight line that fits the data best. © 2004 Prentice-Hall, Inc. Store Square Feet Annual Sales ($1000) 1 2 3 4 5 6 7 1, 726 1, 542 2, 816 5, 555 1, 292 2, 208 1, 313 3, 681 3, 395 6, 653 9, 543 3, 318 5, 563 3, 760 14

Scatter Diagram: Example Excel Output © 2004 Prentice-Hall, Inc. 15

Simple Linear Regression Equation: Example From Excel Printout: © 2004 Prentice-Hall, Inc. 16

Graph of the Simple Linear Regression Equation: Example Xi 7 8. 4 1 =

Interpretation of Results: Example The slope of 1. 487 means that for each increase of one unit in X, we predict the average of Y to increase by an estimated 1. 487 units. The equation estimates that for each increase of 1 square foot in the size of the store, the expected annual sales are predicted to increase by $1487. © 2004 Prentice-Hall, Inc. 18

Simple Linear Regression in PHStat n n In Excel, use PHStat | Regression |

Measures of Variation: The Sum of Squares SST = Total = Sample Variability ©

Measures of Variation: The Sum of Squares (continued) n SST = Total Sum of Squares n n SSR = Regression Sum of Squares n n Measures the variation of the Yi values around their mean, Explained variation attributable to the relationship between X and Y SSE = Error Sum of Squares n Variation attributable to factors other than the relationship between X and Y © 2004 Prentice-Hall, Inc. 21

Measures of Variation: The Sum of Squares (continued) Y SST = (Yi - SSE

Venn Diagrams and Explanatory Power of Regression Variations in store Sizes not used in explaining variation in Sales Sizes © 2004 Prentice-Hall, Inc. Sales Variations in Sales explained by the error term or unexplained by Sizes Variations in Sales explained by Sizes or variations in Sizes used in explaining variation in Sales 23

The ANOVA Table in Excel ANOVA df Regression k SS MS SSR MSR =SSR/k

Measures of Variation The Sum of Squares: Example Excel Output for Produce Stores Degrees

The Coefficient of Determination n n Measures the proportion of variation in Y that

Venn Diagrams and Explanatory Power of Regression Sales Sizes © 2004 Prentice-Hall, Inc. 27

Coefficients of Determination (r 2) and Correlation (r) Y r 2 = 1, r = +1 ^=b +b X Y i 0 1 i Y r 2 = 1, r = -1 ^=b +b X Y i 0 X Y r 2 =. 81, r = +0. 9 X © 2004 Prentice-Hall, Inc. X Y ^=b +b X Y i 0 1 i r 2 = 0, r = 0 ^=b +b X Y i 0 1 i X 28

Standard Error of Estimate n n Measures the standard deviation (variation) of the Y

Measures of Variation: Produce Store Example Excel Output for Produce Stores r 2 =. 94 n Syx 94% of the variation in annual sales can be explained by the variability in the size of the store as measured by square footage. © 2004 Prentice-Hall, Inc. 30

Linear Regression Assumptions n Normality n n Y values are normally distributed for each X Probability distribution of error is normal Homoscedasticity (Constant Variance) Independence of Errors © 2004 Prentice-Hall, Inc. 31

Consequences of Violation of the Assumptions n n Non-normality (error not normally distributed) Heteroscedasticity (variance not constant) n n Autocorrelation (errors are not independent) n n Usually happens in time-series data Consequences of Any Violation of the Assumptions n n n Usually happens in cross-sectional data Predictions and estimations obtained from the sample regression line will not be accurate Hypothesis testing results will not be reliable It is Important to Verify the Assumptions © 2004 Prentice-Hall, Inc. 32

Variation of Errors Around the Regression Line f(e) • Y values are normally distributed around the regression line. • For each X value, the “spread” or variance around the regression line is the same. Y X 2 X X 1 © 2004 Prentice-Hall, Inc. Sample Regression Line 33

Residual Analysis n Purposes n n n Examine linearity Evaluate violations of assumptions Graphical

Residual Analysis for Linearity Y Y 0 e X 0 X X 0 Not

Residual Analysis for Homoscedasticity Y Y 0 SR X 0 X SR Heteroscedasticity ©

Residual Analysis: Excel Output for Produce Stores Example Excel Output © 2004 Prentice-Hall, Inc.

Residual Analysis for Independence n The Durbin-Watson Statistic n n Used when data are collected over time to detect autocorrelation (residuals in one time period are related to residuals in another period) Measures violation of independence assumption Should be close to 2. If not, examine the model for autocorrelation. © 2004 Prentice-Hall, Inc. 38

Durbin-Watson Statistic in PHStat | Regression | Simple Linear Regression … n Check the

Obtaining the Critical Values of Durbin-Watson Statistic Table 13. 4 Finding Critical Values of

Using the Durbin-Watson Statistic : No autocorrelation (error terms are independent) : There is autocorrelation (error terms are not independent) Reject H 0 (positive autocorrelation) 0 © 2004 Prentice-Hall, Inc. d. L Inconclusive Do not reject H 0 (no autocorrelation) d. U 2 4 -d. U Reject H 0 (negative autocorrelation) 4 -d. L 4 41

Residual Analysis for Independence Graphical Approach Not Independent e 0 Time 0 Cyclical Pattern Time No Particular Pattern Residual is Plotted Against Time to Detect Any Autocorrelation © 2004 Prentice-Hall, Inc. 42

Inference about the Slope: t Test n t Test for a Population Slope n n Null and Alternative Hypotheses n n n Is there a linear relationship between Y and X ? H 0 : 1 = 0 H 1 : 1 0 (no linear relationship) (linear relationship) Test Statistic n n © 2004 Prentice-Hall, Inc. 43

Example: Produce Store Data for 7 Stores: Store Square Feet Annual Sales ($000) 1 2 3 4 5 6 7 1, 726 1, 542 2, 816 5, 555 1, 292 2, 208 1, 313 3, 681 3, 395 6, 653 9, 543 3, 318 5, 563 3, 760 © 2004 Prentice-Hall, Inc. Estimated Regression Equation: The slope of this model is 1. 487. Are square footage and annual sales linearly related? 44

Inferences about the Slope: t Test Example H 0 : 1 = 0 H 1 : 1 0 . 05 df 7 - 2 = 5 Critical Value(s): Reject. 025 Test Statistic: From Excel Printout Reject. 025 -2. 5706 0 2. 5706 © 2004 Prentice-Hall, Inc. Decision: Reject H 0. t p-value Conclusion: There is evidence that square footage is linearly related to annual sales. 45

Inferences about the Slope: Confidence Interval Example Confidence Interval Estimate of the Slope: Excel Printout for Produce Stores At 95% level of confidence, the confidence interval for the slope is (1. 062, 1. 911). Does not include 0. Conclusion: There is a significant linear relationship between annual sales and the size of the store. © 2004 Prentice-Hall, Inc. 46

Inferences about the Slope: F Test n F Test for a Population Slope n n Null and Alternative Hypotheses n n n Is there a linear relationship between Y and X ? H 0 : 1 = 0 H 1 : 1 0 (no linear relationship) (linear relationship) Test Statistic n © 2004 Prentice-Hall, Inc. n Numerator d. f. =1, denominator d. f. =n-2 47

Relationship between a t Test and an F Test n Null and Alternative Hypotheses n n H 0 : 1 = 0 H 1 : 1 0 (no linear relationship) (linear relationship) n n n The p –value of a t Test and the p –value of an F Test are Exactly the Same The Rejection Region of an F Test is Always in the Upper Tail © 2004 Prentice-Hall, Inc. 48

Inferences about the Slope: F Test Example Test Statistic: H 0 : 1 = 0 H 1 : 1 0 . 05 numerator df = 1 denominator df 7 - 2 = 5 From Excel Printout p-value Reject =. 05 0 © 2004 Prentice-Hall, Inc. 6. 61 Decision: Reject H 0. Conclusion: There is evidence that square footage is linearly related to annual sales. 49

Purpose of Correlation Analysis n Correlation Analysis is Used to Measure Strength of Association (Linear Relationship) Between 2 Numerical Variables n n Only strength of the relationship is concerned No causal effect is implied © 2004 Prentice-Hall, Inc. 50

Purpose of Correlation Analysis (continued) n Population Correlation Coefficient (Rho) is Used to Measure

Purpose of Correlation Analysis (continued) n Sample Correlation Coefficient r is an Estimate of and is Used to Measure the Strength of the Linear Relationship in the Sample Observations © 2004 Prentice-Hall, Inc. 52

Sample Observations from Various r Values Y Y r = -1 X Y r

Features of and r n n n Unit Free Range between -1 and 1 The Closer to -1, the Stronger the Negative Linear Relationship The Closer to 1, the Stronger the Positive Linear Relationship The Closer to 0, the Weaker the Linear Relationship © 2004 Prentice-Hall, Inc. 54

t Test for Correlation n Hypotheses n n n H 0: = 0 (no

Example: Produce Stores From Excel Printout Is there any evidence of linear relationship between annual sales of a store and its square footage at. 05 level of significance? © 2004 Prentice-Hall, Inc. r H 0: = 0 (no association) H 1: 0 (association) . 05 df 7 - 2 = 5 56

Example: Produce Stores Solution Decision: Reject H 0. Critical Value(s): Reject. 025 -2. 5706 0 2. 5706 © 2004 Prentice-Hall, Inc. Conclusion: There is evidence of a linear relationship at 5% level of significance. The value of the t statistic is exactly the same as the t statistic value for test on the slope coefficient. 57

Estimation of Mean Values Confidence Interval Estimate for : The Mean of Y Given a Particular Xi Standard error of the estimate Size of interval varies according to distance away from mean, t value from table with df=n-2 © 2004 Prentice-Hall, Inc. 58

Prediction of Individual Values Prediction Interval for Individual Response Yi at a Particular Xi

Interval Estimates for Different Values of X Prediction Interval for an Individual Yi Y

Example: Produce Stores Data for 7 Stores: Store Square Feet Annual Sales ($000) 1 2 3 4 5 6 7 1, 726 1, 542 2, 816 5, 555 1, 292 2, 208 1, 313 3, 681 3, 395 6, 653 9, 543 3, 318 5, 563 3, 760 © 2004 Prentice-Hall, Inc. Consider a store with 2000 square feet. Regression Model Obtained: Yi = 1636. 415 +1. 487 Xi 61

Estimation of Mean Values: Example Confidence Interval Estimate for Find the 95% confidence interval for the average annual sales for stores of 2, 000 square feet. Predicted Sales Yi = 1636. 415 +1. 487 Xi = 4610. 45 (in $000) X = 2350. 29 © 2004 Prentice-Hall, Inc. SYX = 611. 75 tn-2 = t 5 = 2. 5706 62

Prediction Interval for Y : Example Prediction Interval for Individual Find the 95% prediction interval for annual sales of one particular store of 2, 000 square feet. Predicted Sales Yi = 1636. 415 +1. 487 Xi = 4610. 45 (in $000) X = 2350. 29 © 2004 Prentice-Hall, Inc. SYX = 611. 75 tn-2 = t 5 = 2. 5706 63

Estimation of Mean Values and Prediction of Individual Values in PHStat n In Excel, use PHStat | Regression | Simple Linear Regression … n n Check the “Confidence and Prediction Interval for X=” box Excel Spreadsheet of Regression Sales on Footage © 2004 Prentice-Hall, Inc. 64

Pitfalls of Regression Analysis n n Lacking an Awareness of the Assumptions Underlining Least-Squares Regression Not Knowing How to Evaluate the Assumptions Not Knowing What the Alternatives to Least. Squares Regression are if a Particular Assumption is Violated Using a Regression Model Without Knowledge of the Subject Matter © 2004 Prentice-Hall, Inc. 65

Strategy for Avoiding the Pitfalls of Regression n Start with a scatter plot to observe possible relationship between X on Y Perform residual analysis to check the assumptions Use a histogram, stem-and-leaf display, boxand-whisker plot, or normal probability plot of the residuals to uncover possible nonnormality © 2004 Prentice-Hall, Inc. 66

Strategy for Avoiding the Pitfalls of Regression (continued) n n If there is violation of any assumption, use alternative methods (e. g. , least absolute deviation regression or least median of squares regression) to least-squares regression or alternative least-squares models (e. g. , curvilinear or multiple regression) If there is no evidence of assumption violation, then test for the significance of the regression coefficients and construct confidence intervals and prediction intervals © 2004 Prentice-Hall, Inc. 67

Chapter Summary n n n Introduced Types of Regression Models Discussed Determining the Simple Linear Regression Equation Described Measures of Variation Addressed Assumptions of Regression and Correlation Discussed Residual Analysis Addressed Measuring Autocorrelation © 2004 Prentice-Hall, Inc. 68

Chapter Summary n n (continued) Described Inference about the Slope Discussed Correlation - Measuring the Strength of the Association Addressed Estimation of Mean Values and Prediction of Individual Values Discussed Pitfalls in Regression and Ethical Issues © 2004 Prentice-Hall, Inc. 69