Regression Analysis GOAL Determine a function that best

Regression Analysis • • GOAL: Determine a function that “best” maps a set of inputs to corresponding outputs. Regression is one form of data mining. Regression both describes the data (finds human-interpretable patterns that describe the data) and through interpolation and extrapolation predicts unknown or future values of variables. Regression analysis relies upon the concept of minimizing the squares of the errors.

Regression Analysis: The problem • Given: a set of inputs x, corresponding outputs y, and a desired parameterized function type (e. g. linear, quadratic, exponential, logarithmic, Gaussian, etc. ), determine the curve of best fitby finding the parameter values than minimize the sum of the squares of the errors in the data.

Linear Regression • We will illustrate the concept beginning with the simplest regression problem type: fitting a line to a set of single variable inputs x and corresponding single variable outputs y.

Linear Regression: Deriving the Equations • Given the data we wish to determine the values of the parameters and for which the line minimizes the sum

Linear Regression: Deriving the Equations • Substituting, we wish to minimize the function • This occurs where • That is, where

Linear Regression: Deriving the Equations • Or,

Linear Regression: Deriving the Equations • Solving, we have, • where

Linear Regression: Deriving the Equations • Pearson Correlation Coefficient for Linear Regression

Linear Regression: Analysis • • r values “close” to +1 indicate a strong positive correlation r values “close” to -1 indicate a strong negative correlation r values “close” to 0 indicate little correlation The “coefficient of determination”, r 2, represents the proportion of the variation that can be explained by the linear relationship between x and y provided by the regression equation.

Linear Regression: Excel • Demonsration

Regression Extensions • • Multi-regression: Extension of regression to multiple inputs Non-linear regression: Replacing the linear relationship between inputs x and outputs y with a non-linear relationship where the are the parameters of f. • Similar equation derivations: For each , we have

Regression Extensions • Certain non-linear relations can be solved via linear regression upon logarithms of the variables • Power laws: • Taking logarithms of both sides yields which is linear in log(x) and log(y) • Logarithmic laws: which is linear in log(x) and y • Exponential laws: • Taking logarithms of both sides, which is linear in log(y-b) and x.