Chapter 12 Simple Linear Regression and Correlation Copyright

12. 1 The Simple Linear Regression Model Copyright (c) 2004 Brooks/Cole, a division of

Linear Relationship The simplest deterministic mathematical relationship between two variables x and y is

Terminology The variable whose value is fixed by the experimenter, denoted x, is the

The Simple Linear Regression Model There exists parameters such that for any fixed value

Linear Regression Model (x 1, y 1) True regression line x 1 Copyright (c)

Distribution of Normal, mean = 0, standard deviation 0 Copyright (c) 2004 Brooks/Cole, a

Distribution of Y for Different Values of x x 1 x 2 x 3

12. 2 Estimating Model Parameters Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning,

Principle of Least Squares The vertical deviation of the point (xi, yi) from the

Principle of Least Squares The least-squares (regression) line for the data is given by

Ex. Find the equation of least-squares for the data Sum: x 1 2 y

Fitted Values and Residuals The fitted (predicted) values are obtained by substituting into the

Error Sum of Squares The error sum of squares, denoted SSE, is and the

Computational Formula A computational formula for the SSE, is Copyright (c) 2004 Brooks/Cole, a

Total Sum of Squares The total sum of squares, denoted SST, is Copyright (c)

Coefficient of Determination The coefficient of determination, denoted by r 2, is given by

Regression Sum of Squares SSR = SST – SSE Regression sum of squares is

12. 3 Inferences About the Slope Parameter Copyright (c) 2004 Brooks/Cole, a division of

1. The mean of 2. The variance and standard deviation are 3. has a

T Variable The assumptions of the simple linear regression model imply that the standardized

Confidence Interval of the true regression line is Copyright (c) 2004 Brooks/Cole, a division

Hypothesis-Testing Procedures Null hypothesis: Test statistic value: Copyright (c) 2004 Brooks/Cole, a division of

Hypothesis-Testing Procedures Alternative Hypothesis Rejection Region for Approx. Level Test or A P-value based

Hypothesis-Testing The model utility test is the test of in which case the test

ANOVA Table Source of Variation df Sum of squares Mean Square Regression 1 SSR

12. 4 Inferences Concerning and the Prediction of Future Y Values Copyright (c) 2004

is some fixed value of x. 1. The mean of is 2. Variance and

2. (continued) 3. has a normal distibution. Copyright (c) 2004 Brooks/Cole, a division of

T Variable The variable has a t distribution with n – 2 df. Copyright

Confidence Interval expected value of Y when x = x*, is Copyright (c) 2004

Prediction Interval A future value of Y is not a parameter but instead a

Prediction Interval A PI for a future Y observation to be made when x

12. 5 Correlation Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Sample Correlation Coefficient The sample correlation coefficient, denoted r, of n pairs (x 1,

Ex. Find the correlation coefficient for the least-squares line from the points = 0.

Properties of r Important properties of r 1. The value of r does not

Properties of r 4. r = 1 iff all (xi, yi) pairs lie on

Different Values of r r near 1 r near 0, no relationship r near

The Population Correlation Coefficient where depending on whether (X, Y) is discrete or continuous.

Estimator Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Assumption The joint probability distribution of (X, Y ) is specified by is called

Testing for the Absence of Correlation When statistic: is true, the test Has a

Hypothesis-Testing Alternative Hypothesis Rejection Region for Approx. Level Test or A P-value based on

Other Inferences Concerning When (X 1, Y 1), …, (Xn, Yn) is a sample

The test statistic for testing Alternative Hypothesis Rejection Region for Level Test or Copyright

CI for where c 1 and c 2 are the left and right endpoints,

Slides: 47

Download presentation

Linear Relationship The simplest deterministic mathematical relationship between two variables x and y is a linear relationship The set of pairs (x, y) for which determines a straight line. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Terminology The variable whose value is fixed by the experimenter, denoted x, is the independent (predictor, explanatory) variable. For a fixed x, the second variable will be a random variable Y with observed value y, referred to as the dependent (response) variable. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

The Simple Linear Regression Model There exists parameters such that for any fixed value of x, the dependent variable is related to x through the model equation is a random variable (called the random deviation) with Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Principle of Least Squares The vertical deviation of the point (xi, yi) from the line y = b 0 + b 1 x is yi – (b 0 + b 1 xi) The sum of squared vertical deviations from the points to the line is: Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Fitted Values and Residuals The fitted (predicted) values are obtained by substituting into the equation of the estimated regression line: The residuals are the vertical deviations from the estimated line. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Coefficient of Determination The coefficient of determination, denoted by r 2, is given by It is interpreted as the proportion of observed y variation that can be explained by the simple linear regression model. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Regression Sum of Squares SSR = SST – SSE Regression sum of squares is interpreted as the amount of variation that is explained by the model. We have Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

T Variable The assumptions of the simple linear regression model imply that the standardized variable has a t distribution with n – 2 df. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Hypothesis-Testing Procedures Alternative Hypothesis Rejection Region for Approx. Level Test or A P-value based on n – 2 df can be calculated as in Chap 8 and 9. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Prediction Interval A future value of Y is not a parameter but instead a random variable; its interval of plausible values is referred to as a prediction interval. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Properties of r Important properties of r 1. The value of r does not depend on which of the two variables under study is labeled x and which is labeled y. 2. The value of r is independent of the units in which x and y are measured. 3. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Properties of r 4. r = 1 iff all (xi, yi) pairs lie on straight line with positive slope, and r = – 1 iff all (xi, yi) pairs lie on a straight line with negative slope. 5. The square of the sample correlation coefficient gives the value of the coefficient of determination that would result from fitting the simple linear regression model. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Hypothesis-Testing Alternative Hypothesis Rejection Region for Approx. Level Test or A P-value based on n – 2 df can be calculated as described previously. Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

Other Inferences Concerning When (X 1, Y 1), …, (Xn, Yn) is a sample from a bivariate normal distribution, the rv has approximately a normal distribution with mean and variance Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.