Numerical Linear Algebra Mo Mu March 15 6

Numerical Linear Algebra Mo Mu March 15, 6: 30 -9: 20

Linear Algebra Problems • Solving Linear System Equations Au=b • Matrix Factorizations with Applications in Data Analysis, Eigenvalue/Singular Value Computation, etc

Solving Linear System Equations • Direct Methods • Gaussian elimination, pivoting, LU factorization • Band solvers • Sparse solvers • Minimum fill-in • Nested disection • Multi-frontal • Iterative Methods

Solving Linear System Equations • Iterative Methods • Basic iterative methods • Simple (one step) linear iterative methods • Optimization based directional searching methods • Advanced iterative techniques • • • Preconditioning Polynomial acceleration Multigrid Domain decomposition Subspace decomposition • Others: Min. Res, GMRes, etc.

Basic Linear Iterative Methods • Simple (one step) linear iterative methods (Linear fixed point iteration with the fixed point being the solution of the original linear system) • un+1 = g(un) =Gun + k • Residual correction (RF method, with G = I-A) • rn = b – Aun • un+1 = un + rn = (I - A) un + b • Matrix splitting for fixed point, with A = Q-N, or A = D – L –U specifically, which leads to • Jacobi method: D u = (L+U) u = b, where Q = D • Gauss-Seidel method: (D-L)u = U u + b, where Q = D-L • SOR method, extrapolation with Gauss-Seidel • Error correction, Improved RF, or Preconditioning

Basic Linear Iterative Methods, continued • Error correction, Improved RF • Compute rn = b – Aun; • Solve error equation approximately • Aen = rn , or compute en =A-1 rn, by e = Brn, with B being an approximation to A-1, called a preconditioner, and easily computable for Brn • Error correction • un+1 = un + e • Preconditioning • un+1= un + B rn = (I - BA) un + Bb, • G = I – B A = I – Q-1 A (in D. Young, where Q is called a splitting matrix, with A = Q-N) in the linear fixed point iteration; • This may be viewed as by applying B = Q-1 to the original system (preconditioning), then do RF

Directional Searching Methods • Optimization • Directional searching • Steepest descent method • Conjugate gradient method • A BRIEF INTRODUCTION TO THE CONJUGATE GRADIENT METHOD

Linear Iteration and Preconditioners (J. Xu) • Proposition 2. 2 (J. XU) A symmetric iterative scheme gives rise to a preconditioner B for PCG, and the rate of the convergence of the iterative scheme may be accelerated by using PCG • Proposition 2. 3 (J. XU) Any preconditioner can also be used to construct a linear iterative scheme by the extrapolated preconditioned RF, with the optimal choice of the parameter: • un+1= un + ωB rn

Multigrid Methods

Domain Decompsition

Subspace Correction

Matrix Factorizations