Forecasting Methods Quantitative Causal Time Series Smoothing Trend

Forecasting Methods Quantitative Causal Time Series Smoothing Trend Projection Qualitative Trend Projection Adjusted for Seasonal Influence Slide 1

General Linear Model n Models in which the parameters ( 0, 1, . . . , p ) all have exponents of one are called linear models. It does not imply that the relationship between y and the xi’s is linear. n A general linear model involving p independent variables is n Each of the independent variables z is a function of x 1, x 2, . . . , xk (the variables for which data have been collected). © 2006 by Thomson Learning, a division of Thomson Asia Pte Ltd. . Slide 2

General Linear Model n The simplest case is when we have collected data for just one variable x 1 and want to estimate y by using a straight-line relationship. In this case z 1 = x 1. n This model is called a simple first-order model with one predictor variable. © 2006 by Thomson Learning, a division of Thomson Asia Pte Ltd. . Slide 3

Estimated Multiple Regression Equation A simple random sample is used to compute sample statistics b 0, b 1, b 2, . . . , bp that are used as the point estimators of the parameters b 0, b 1, b 2, . . . , bp. The estimated multiple regression equation is: ^ y y = b 0 + b 1 x 1 + b 2 x 2 +. . . + bpxp Slide 4

Estimation Process Multiple Regression Model E(y) = 0 + 1 x 1 + 2 x 2 +. . . + pxp + Multiple Regression Equation E(y) = 0 + 1 x 1 + 2 x 2 +. . . + pxp Unknown parameters are Sample Data: x 1 x 2. . . xp y. . . . 0, 1, 2, . . . , p b 0, b 1, b 2, . . . , bp Estimated Multiple Regression Equation provide estimates of 0, 1, 2, . . . , p Sample statistics are b 0, b 1, b 2, . . . , bp Slide 5

Least Squares Method n Least Squares Criterion n Computation of Coefficient Values The formulas for the regression coefficients b 0, b 1, b 2, . . . , bp involve the use of matrix algebra. We will rely on computer software packages to perform the calculations. Slide 6

Multiple Regression Equation n Example: Butler Trucking Company To develop better work schedules, the managers want to estimate the total daily travel time for their drivers Data Slide 7

Multiple Regression Equation n MINITAB Output Slide 8

Multiple Regression Model n Example: Programmer Salary Survey A software firm collected data for a sample of 20 computer programmers. A suggestion was made that regression analysis could be used to determine if salary was related to the years of experience and the score on the firm’s programmer aptitude test. The years of experience, score on the aptitude test, and corresponding annual salary ($1000 s) for a sample of 20 programmers is shown on the next slide. Slide 9

Multiple Regression Model Exper. Score Salary 4 7 1 5 8 10 0 1 6 6 78 100 86 82 86 84 75 80 83 91 24 43 23. 7 34. 3 35. 8 38 22. 2 23. 1 30 33 9 2 10 5 6 8 4 6 3 3 88 73 75 81 74 87 79 94 70 89 38 26. 6 36. 2 31. 6 29 34 30. 1 33. 9 28. 2 30 Slide 10

Multiple Regression Model Suppose we believe that salary (y) is related to the years of experience (x 1) and the score on the programmer aptitude test (x 2) by the following regression model: y = 0 + 1 x 1 + 2 x 2 + where y = annual salary ($1000) x 1 = years of experience x 2 = score on programmer aptitude test Slide 11

Solving for the Estimates of 0, 1, 2 Least Squares Output Input Data x 1 x 2 y 4 78 24 7 100 43. . . 3 89 30 Computer Package for Solving Multiple Regression Problems b 0 = b 1 = b 2 = R 2 = etc. Slide 12

Solving for the Estimates of 0, 1, 2 n Excel Worksheet (showing partial data entered) Note: Rows 10 -21 are not shown. Slide 13

Solving for the Estimates of 0, 1, 2 n Excel’s Regression Dialog Box Slide 14

Solving for the Estimates of 0, 1, 2 n Excel’s Regression Equation Output Note: Columns F-I are not shown. Slide 15

Estimated Regression Equation SALARY = 3. 174 + 1. 404(EXPER) + 0. 251(SCORE) Note: Predicted salary will be in thousands of dollars. Slide 16

Interpreting the Coefficients In multiple regression analysis, we interpret each regression coefficient as follows: bi represents an estimate of the change in y corresponding to a 1 -unit increase in xi when all other independent variables are held constant. Slide 17

Interpreting the Coefficients b 1 = 1. 404 Salary is expected to increase by $1, 404 for each additional year of experience (when the variable score on programmer attitude test is held constant). Slide 18

Interpreting the Coefficients b 2 = 0. 251 Salary is expected to increase by $251 for each additional point scored on the programmer aptitude test (when the variable years of experience is held constant). Slide 19

Multiple Coefficient of Determination n Relationship Among SST, SSR, SSE SST = SSR + SSE where: SST = total sum of squares SSR = sum of squares due to regression SSE = sum of squares due to error Slide 20

Multiple Coefficient of Determination n Excel’s ANOVA Output SSR SST Slide 21

Multiple Coefficient of Determination R 2 = SSR/SST R 2 = 500. 3285/599. 7855 =. 83418 • • In general, R 2 always increases as independent variables are added to the model. adjusting R 2 for the number of independent variables to avoid overestimating the impact of adding an independent variable Slide 22

Adjusted Multiple Coefficient of Determination • • n denoting the number of observations p denoting the number of independent variables Slide 23

Adjusted Multiple Coefficient of Determination n Excel’s Regression Statistics Slide 24

Assumptions About the Error Term The error is a random variable with mean of zero. The variance of , denoted by 2, is the same for all values of the independent variables. The values of are independent. The error is a normally distributed random variable reflecting the deviation between the y value and the expected value of y given by 0 + 1 x 1 + 2 x 2 +. . . + pxp. Slide 25

Multiple Regression Analysis with Two Independent Variables n Graph Slide 26

Testing for Significance In simple linear regression, the F and t tests provide the same conclusion. In multiple regression, the F and t tests have different purposes. Slide 27

Testing for Significance: F Test The F test is used to determine whether a significant relationship exists between the dependent variable and the set of all the independent variables. The F test is referred to as the test for overall significance. Slide 28

Testing for Significance: t Test If the F test shows an overall significance, the t test is used to determine whether each of the individual independent variables is significant. A separate t test is conducted for each of the independent variables in the model. We refer to each of these t tests as a test for individual significance. Slide 29

Testing for Significance: F Test Hypotheses H 0: 1 = 2 =. . . = p = 0 Ha: One or more of the parameters is not equal to zero. Test Statistics F = MSR/MSE Rejection Rule Reject H 0 if p-value < or if F > F , where F is based on an F distribution with p d. f. in the numerator and n - p - 1 d. f. in the denominator. Slide 30

Testing for Significance: F Test n ANOVA Table for A Multiple Regression Model with p Independent Variables Slide 31

F Test for Overall Significance Hypotheses Rejection Rule H 0: 1 = 2 = 0 Ha: One or both of the parameters is not equal to zero. For =. 05 and d. f. = 2, 17; F. 05 = 3. 59 Reject H 0 if p-value <. 05 or F > 3. 59 Slide 32

F Test for Overall Significance n Excel’s ANOVA Output p-value used to test for overall significance Slide 33

F Test for Overall Significance Test Statistics Conclusion F = MSR/MSE = 250. 16/5. 85 = 42. 76 p-value <. 05, so we can reject H 0. (Also, F = 42. 76 > 3. 59) Slide 34

Testing for Significance: t Test Hypotheses Test Statistics Rejection Rule Reject H 0 if p-value < or if t < -t or t > t where t is based on a t distribution with n - p - 1 degrees of freedom. Slide 35

t Test for Significance of Individual Parameters Hypotheses Rejection Rule For =. 05 and d. f. = 17, t. 025 = 2. 11 Reject H 0 if p-value <. 05 or if t > 2. 11 Slide 36

t Test for Significance of Individual Parameters n Excel’s Regression Equation Output Note: Columns F-I are not shown. t statistic and p-value used to test for the individual significance of “Experience” Slide 37

t Test for Significance of Individual Parameters n Excel’s Regression Equation Output Note: Columns F-I are not shown. t statistic and p-value used to test for the individual significance of “Test Score” Slide 38

t Test for Significance of Individual Parameters Test Statistics Conclusions Reject both H 0: 1 = 0 and H 0: 2 = 0. Both independent variables are significant. Slide 39

Testing for Significance: Multicollinearity The term multicollinearity refers to the correlation among the independent variables. When the independent variables are highly correlated (say, |r | >. 7), it is not possible to determine the separate effect of any particular independent variable on the dependent variable. Slide 40

Testing for Significance: Multicollinearity If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually not a serious problem. Every attempt should be made to avoid including independent variables that are highly correlated. Slide 41

Modeling Curvilinear Relationships n To account for a curvilinear relationship, we might set z 1 = x 1 and z 2 =. n This model is called a second-order model with one predictor variable. Slide 42

Modeling Curvilinear Relationships n n n Example: Reynolds, Inc. , Managers at Reynolds want to investigate the relationship between length of employment of their salespeople and the number of electronic laboratory scales sold. Data Slide 43

Modeling Curvilinear Relationships n Scatter Diagram for the Reynolds Example Slide 44

Modeling Curvilinear Relationships n Let us consider a simple first-order model and the estimated regression is Sales = 111 + 2. 38 Months, where : Sales = number of electronic laboratory scales sold, Months = the number of months the salesperson has been employed Slide 45

Modeling Curvilinear Relationships n MINITAB output – first-order model Slide 46

Modeling Curvilinear Relationships n n Standardized Residual plot – first-order model The standardized residual plot suggests that a curvilinear relationship is needed Slide 47

Modeling Curvilinear Relationships n n Reynolds Example : The second-order model The estimated regression equation is Sales = 45. 3 + 6. 34 Months +. 0345 Months. Sq where : Sales = number of electronic laboratory scales sold, Months. Sq = the square of the number of months the salesperson has been employed Slide 48

Modeling Curvilinear Relationships n MINITAB output –second-order model Slide 49

Modeling Curvilinear Relationships n Standardized Residual plot – second-order model Slide 50

Variable Selection Procedures n Stepwise Regression n Forward Selection n Backward Elimination Iterative; one independent variable at a time is added or deleted based on the F statistic Slide 51

Variable Selection: Stepwise Regression Any p-value < alpha to enter ? Compute F stat. and p-value for each indep. variable not in model No Any p-value > alpha to remove ? Yes Compute F stat. and p-value for each indep. variable in model Indep. variable with largest p-value is removed from model next iteration No Yes Stop Indep. variable with smallest p-value is entered into model Start with no indep. variables in model Slide 52

Variable Selection: Forward Selection Start with no indep. variables in model Compute F stat. and p-value for each indep. variable not in model Any p-value < alpha to enter ? Yes Indep. variable with smallest p-value is entered into model No Stop Slide 53

Variable Selection: Backward Elimination Start with all indep. variables in model Compute F stat. and p-value for each indep. variable in model Any p-value > alpha to remove ? Yes Indep. variable with largest p-value is removed from model No Stop Slide 54

Qualitative Independent Variables In many situations we must work with qualitative independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. For example, x 2 might represent gender where x 2 = 0 indicates male and x 2 = 1 indicates female. In this case, x 2 is called a dummy or indicator variable. Slide 55

Qualitative Independent Variables n Example: Programmer Salary Survey As an extension of the problem involving the computer programmer salary survey, suppose that management also believes that the annual salary is related to whether the individual has a graduate degree in computer science or information systems. The years of experience, the score on the programmer aptitude test, whether the individual has a relevant graduate degree, and the annual salary ($1000) for each of the sampled 20 programmers are shown on the next slide. Slide 56

Qualitative Independent Variables Exper. Score Degr. Salary 4 7 1 5 8 10 0 1 6 6 78 100 86 82 86 84 75 80 83 91 No Yes Yes No No No Yes 24 43 23. 7 34. 3 35. 8 38 22. 2 23. 1 30 33 Exper. Score Degr. Salary 9 2 10 5 6 8 4 6 3 3 88 73 75 81 74 87 79 94 70 89 Yes No No 38 26. 6 36. 2 31. 6 29 34 30. 1 33. 9 28. 2 30 Slide 57

Estimated Regression Equation y = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 where: y^ = annual salary ($1000) x 1 = years of experience x 2 = score on programmer aptitude test x 3 = 0 if individual does not have a graduate degree 1 if individual does have a graduate degree x 3 is a dummy variable Slide 58

Qualitative Independent Variables n Excel’s Regression Statistics Slide 59

Qualitative Independent Variables n Excel’s ANOVA Output Slide 60

Qualitative Independent Variables n Excel’s Regression Equation Output Note: Columns F-I are not shown. Not significant Slide 61

More Complex Qualitative Variables If a qualitative variable has k levels, k - 1 dummy variables are required, with each dummy variable being coded as 0 or 1. For example, a variable with levels A, B, and C could be represented by x 1 and x 2 values of (0, 0) for A, (1, 0) for B, and (0, 1) for C. Care must be taken in defining and interpreting the dummy variables. Slide 62

More Complex Qualitative Variables For example, a variable indicating level of education could be represented by x 1 and x 2 values as follows: Highest Degree Bachelor’s Master’s Ph. D. x 1 x 2 0 1 0 0 0 1 Slide 63

Interaction n If the original data set consists of observations for y and two independent variables x 1 and x 2 we might develop a second-order model with two predictor variables. n In this model, the variable z 5 = x 1 x 2 is added to account for the potential effects of the two variables acting together. n This type of effect is called interaction. Slide 64

Interaction n Example: Tyler Personal Care New shampoo products, two factors believed to have the most influence on sales are unit selling price and advertising expenditure. Data Slide 65

Interaction n Mean Unit Sales (1000 s) for the Tyler Personal Care Example n At higher selling prices, the effect of increased advertising expenditure diminishes. These observations provide evidence of interaction between the price and advertising expenditure variables. Slide 66

Interaction n Mean Sales as a Function of Selling Price and Advertising Expenditure Slide 67

Interaction n To account for the effect of interaction, use the following regression model where : y = unit sales (1000 s), x 1 = price ($), x 2 = advertising expenditure ($1000 s). Slide 68

Interaction n General Linear Model involving three independent variables (z 1, z 2, and z 3) where : y= Sales = unit sales (1000 s) z 1 = x 1 (price) = price of the product ($) z 2 = x 2 (Adv. Exp) = advertising expenditure ($1000 s) z 3 = x 1 x 2 (Price. Adv) = interaction term (Price times Adv. Exp) Slide 69

Interaction n MINITAB Output for the Tyler Personal Care Example Slide 70