Linear Least Squares Paul Heckbert Computer Science Department
Linear Least Squares Paul Heckbert Computer Science Department Carnegie Mellon University 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 1
Orthogonal and Hermitian Matrix A square matrix Q is orthogonal iff Q-1=QT QTQ = QQT = I its rows are orthonormal its columns are orthonormal note: orthogonal matrices are often named Q generalization: a matrix is Hermitian iff Q-1=QH where superscript H denotes complex conjugate transpose 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 2
Householder Transformations The Householder transformation determined by vector v is: outer product, n n matrix inner product, scalar To apply it to a vector x, compute: scalar 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 3
Householder Geometry • Hx is x reflected through the hyperplane perpendicular to v (p : p. Tv=0) 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 4
Householder Properties • H is symmetric, since • H is orthogonal, since 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 5
Householder to Zero Matrix Elements We’ll use Householder transformations to zero subdiagonal elements of a matrix. Given any vector a, find the v that determines an H such that, Now solve for v: 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 6
Choosing the Vector v 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 7
Applying Householder Transforms • Don’t compute Hx explicitly, that costs 3 n 2 flops. • Instead use the formula given previously, which costs 4 n flops (if you pre-compute v. Tv or prenormalize v. Tv=2). • Typically, when using Householder transformations, you never compute the matrix H; it’s only used in derivation and analysis. 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 8
QR Decomposition • Householder transformations are a good way to zero out subdiagonal elements of a matrix. • A is decomposed: • where QT=Hn…H 2 H 1 is the orthogonal (prove!) product of Householders and R is upper triangular. • Overdetermined system Ax=b is transformed into the easy-to-solve 27 Sept. 2000 15 -859 B - Introduction to Scientific Computing 9
- Slides: 9