Introduction to Regression Analysis Dependent variable response variable

Introduction to Regression Analysis

Dependent variable (response variable) n Measures an outcome of a study q q n Dependent variable = Mean (expected value) + random error q n Income GRE scores y = E(y) + ε If y is normally distributed, know the mean and the standard deviation, we can make a probability statement

Probability statement n n Let’s say the mean cholesterol level for graduate students= 250 Standard deviation= 50 units What does this distribution look like? “the probability that ____’s cholesterol will fall within 2 standard deviations of the mean is. 95”

Independent variables (predictor variables) n n explains or causes changes in the response variables (The effect of the IV on the DV) (Predicting the DV based on the IV) What independent variables might help us predict cholesterol levels?

Examples n n The effect of a reading intervention program on student achievement in reading Predict state revenues Predict GPA based on SAT predict reaction time from blood alcohol level

Regression Analysis n Build a model that can be used to predict one variable (y) based on other variables (x 1, x 2, x 3, … xk, ) q n Model: a prediction equation relating y to x 1, x 2, x 3, … xk, Predict with a small amount of error

Typical Strategy for Regression Analysis Start Conduct exploratory data analysis Develop one or more tentative models Identify most suitable model Make inferences based on model Stop

Fitting the Model: Least Squares Method n n Model: an equation that describes the relationship between variables Let’s look at the persistence example

Method of Least Squares Let’s look at the persistence example

Finding the Least Squares Line n Slope: n Intercept: n The line that makes the vertical distances of the data points from the line as small as possible q q The SE [Sum of Errors (deviations from the line, residuals)] equals 0 The SSE (Sum of Squared Errors) is smaller than for any other straight-line model with SE=0.

Regression Line n Has the form y = a + bx q q b is the slope, the amount by which y changes when x increases by 1 unit a is the y-intercept, the value of y when x = 0 (or the point at which the line cuts through the x-axis)

Simplest of the probabilistic models: Straight-Line Regression Model First order linear model n Equation: y = β 0 + β 1 x + ε n Where y = dependent variable x = independent variable β 0 = y-intercept β 1 = slope of the line ε = random error component n

Let’s look at the relationship between two variables and construct the line of best fit example: Beers and BAC n Minitab