Prediction variance in Linear Regression Assumptions on noise
- Slides: 12
Prediction variance in Linear Regression • Assumptions on noise in linear regression allow us to estimate the prediction variance due to the noise at any point. • Prediction variance is usually large when you are far from a data point. • We distinguish between interpolation, when we are in the convex hull of the data points, and extrapolation where we are outside. • Extrapolation is associated with larger errors, and in high dimensions it usually cannot be avoided.
Linear Regression •
Model based error for linear regression •
Prediction variance • Linear regression model • Define • With some algebra • Standard error then
Interpolation, extrapolation and regression • Interpolation is often contrasted to regression or least-squares fit • As important is the contrast between interpolation and extrapolation • Extrapolation occurs when we are outside the convex hull of the data points • For high dimensional spaces we must have extrapolation!
2 D example of convex hull • By generating 20 points at random in the unit square we end up with substantial region near the origin where we will need to use extrapolation • Using the data in the notes, give a couple of alternative sets of alphas Approximately for the point (0. 4, 0. 4)
Example of prediction variance • For a linear polynomial RS y=b 1+b 2 x 1+b 3 x 2 find the prediction variance in the region • (a) For data at three vertices (omitting (1, 1))
Interpolation vs. Extrapolation • At origin . At 3 vertices . At (1, 1)
Standard error contours •
Data at four vertices • Now • And • Error at vertices • At the origin minimum is • How can we reduce error without adding points?
Graphical Comparison of Standard Errors Three points Four points
Homework • Redo the four point example, when the data points are not at the corners but inside the domain, at +-0. 8. What does the difference in the results tells you? • For a grid of 3 x 3 data points, compare the standard errors for a linear and quadratic fits.
- Simple linear regression and multiple linear regression
- Classical linear regression model assumptions
- Mahalanobis distance spss
- Multiple linear regression variance
- Multiple linear regression variance
- Survival analysis vs logistic regression
- Logistic regression vs linear regression
- Regression linear model
- Prediction error variance
- Empirical bayesian kriging regression prediction
- Standard costing formula
- Noise variance
- Linear regression spss