Conjugate Gradient Method invented by Hestenes and Stiefel
Conjugate Gradient Method invented by Hestenes and Stiefel around 1951 Conjugate Gradient Method It is an iterative method to solve the linear system of equations
Conjugate Gradient Method Example: Solve: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 0 x 1 x 2 x 3 X 4 k=1 k=2 0 0. 4716 0. 9964 0 1. 9651 1. 9766 0 -0. 8646 -0. 9098 0 1. 1791 1. 0976 k=3 k=4 1. 0015 1. 9833 -1. 0099 1. 0197 1. 0000 2. 0000 -1. 0000 31. 7 5. 1503 1. 0433 0. 1929 0. 0000
Quadratic function We want to solve the following linear system Define: Example: quadratic function
Quadratic Function Example: Remark:
Minimization equivalent ot linear system Remark: Problem (1) IDEA: Search for the minimum Problem (2)
Conjugate Gradient Method Example: minimum
Minimum IDEA: Remark: Search for the minimum
Conjugate Gradient Method: “search direction” “step length” Method:
Conjugate Gradient Method vectors constants
Conjugate Gradient Method:
Conjugate Gradient Method: Conjugate Gradient Method
INNER PRODUCT
Inner Product DEF: We say that Is an inner product if Example: A is SPD We define the norm
Inner Product DEF: where A is SPD Example:
Conjugate Gradient
Conjugate Gradient Method: Conjugate Gradient Method
Conjugate Gradient Method:
Conjugate Gradient Method: Conjugate Gradient Method HW:
Conjugate Gradient Method Lemma: [Elman, Silvester, Wathen Book] vectors Orthogonal A-Orthogonal
Error and Residual vectors REMARK Orthogonal A-Orthogonal REMARK Minimizes the A-norm of the error
Conjugate Gradient Method 0. 0000 0. 4716 0. 9964 1. 0015 1. 9651 1. 9766 1. 9833 -0. 8646 -0. 9098 -1. 0099 1. 1791 1. 0976 1. 0197 1. 0000 2. 0000 -1. 0000 6. 0000 4. 9781 -0. 1681 -0. 0123 0. 0000 25. 0000 -0. 5464 0. 0516 0. 1166 -0. 0000 -11. 0000 -0. 1526 -0. 8202 0. 0985 0. 0000 15. 0000 -1. 1925 -0. 6203 -0. 1172 -0. 0000 6. 0000 5. 1362 0. 0427 -0. 0108 25. 0000 0. 1121 0. 0562 0. 1185 -11. 0000 -0. 4424 -0. 8384 0. 0698 15. 0000 -0. 7974 -0. 6530 -0. 1395 0. 0786 0. 1022 0. 1193 0. 1411 0. 0263 0. 0410 0. 0342 0. 0713
Connection to Lanczos
Introduction to Krylov Subspace Methods DEF: Krylov sequence Example: Krylov sequence 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 1 1 11 12 10 10 118 141 100 106 1239 1651 989 1171 12717 19446 9546 13332
Introduction to Krylov Subspace Methods DEF: Krylov subspace Example: Krylov subspace 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 DEF: Krylov matrix Example:
Introduction to Krylov Subspace Methods DEF: Krylov matrix Remark: Example:
Lanczos method Lanczos: The Lanczos algorithm is defined as follows An orthogonal basis for
- Slides: 26