Chapter 11 Gaussian Elimination I Speaker LungSheng Chien

Chapter 11 Gaussian Elimination (I) Speaker: Lung-Sheng Chien Reference book: David Kincaid, Numerical Analysis

Out. Line • Basic operation of matrix - representation - three elementary matrices • Example of Gaussian Elimination (GE) • Formal description of GE • MATLAB usage

Matrix notation in MATLAB 6 -2 2 4 12 -8 6 10 3 -13 9 3 -6 4 1 -18

Matrix-vector product Inner-product based 6 -2 2 4 x 1 6 x 1 + (-2) x 2 + 2 x 3 + 4 x 4 12 -8 6 10 x 2 12 x 1 + (-8) x 2 + 6 x 3 + 10 x 4 3 -13 9 3 x 3 3 x 1 + (-13) x 2 + 9 x 3 + 3 x 4 -6 4 1 -18 x 4 (-6) x 1 + 4 x 2 + 1 x 3 + (-18) x 4 outer-product based -2 6 -2 2 4 x 1 12 -8 6 10 x 2 3 -13 9 3 x 3 3 -13 9 3 -6 4 1 -18 x 4 -6 4 1 -18 x 1 12 x 2 -8 x 3 6 4 x 4 10

Matrix-vector product: MATLAB implementation matvec. m Inner-product based outer-product based Question: which one is better

Column-major nature in MATLAB physical index : 1 D Logical index : 2 D 6 1 12 2 3 3 -6 4 -2 5 -8 6 6 -2 2 4 -13 7 12 -8 6 10 4 8 3 -13 9 3 2 9 -6 4 1 -18 6 10 9 11 1 12 4 13 10 14 3 15 -18 16 Question: how does column-major affect inner-product based matrix-vector product and outer-product based matrix-vector product?

Matrix representation: outer-product where is outer-product representation

Elementary matrix [1] (1) The interchange of two rows in A: Define permutation matrix How to explain? Question 1: why Question 2: how to easily obtain

Concatenation of permutation matrices such that Question: implies Direct calculation

Elementary matrix [2] (2) Multiplying one row by a nonzero constant: Define scaling matrix How to explain? since

Elementary matrix [3] (3) Adding to one row a multiple of another: Define GE (Gaussian Elimination) matrix How to explain? outer-product representation

Use MATLAB notation

Concatenation of GE matrices such that Suppose such that Question:

Out. Line • Basic operation of matrix • Example of Gaussian Elimination (GE) - forward elimination to upper triangle form - backward substitution • Formal description of GE • MATLAB usage

6 -2 2 4 x 1 12 12 -8 6 10 x 2 34 3 -13 9 3 x 3 27 -6 4 1 -18 x 4 -38 12 -8 6 10 34 6 -2 2 4 12 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 3 -13 9 3 x 3 27 0 -4 2 2 10 -6 4 1 -18 x 4 -38 3 -13 9 3 27 6 -2 2 4 x 1 12 6 -2 2 4 12 0 -4 2 2 x 2 10 0 -12 8 1 x 3 21 -6 4 1 -18 x 4 -38 0 -12 8 1 21

-6 4 1 -18 -38 6 -2 2 4 12 0 2 3 -14 -26 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 -12 8 1 x 3 21 0 2 3 -14 x 4 -26 -12 8 1 21 -4 2 2 10 0 2 -5 -9 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 -12 8 1 x 3 21 0 2 3 -14 x 4 -26 First row does not change thereafter 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 2 3 -14 x 4 -26

2 3 -14 -26 -4 2 2 10 0 4 -13 -21 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 4 -13 x 4 -21 4 -13 -21 2 -5 -9 0 -3 -3 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 4 -13 x 4 -21 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 0 -3 x 4 -3

Backward substitution: inner-product-based 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 0 -3 x 4 -3 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 x 4 6 -2 2 4 x 1 x 2 x 3 x 4 12

Backward substitution: outer-product-based 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 0 -3 x 4 -3 6 -2 2 x 1 12 0 -4 2 x 2 10 0 0 2 x 3 -9 6 -2 x 1 8 0 -4 x 2 8 6 x 1 12 x 4 x 3 x 2 -2 4 8 2 8 -5 -4 2 12 6

Out. Line • Basic operation of matrix • Example of Gaussian Elimination (GE) • Formal description of GE - component-wise and column-wise representation - recursive structure • MATLAB usage

6 -2 2 4 x 1 12 12 -8 6 10 x 2 34 3 -13 9 3 x 3 27 -6 4 1 -18 x 4 -38 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 3 -13 9 3 x 3 27 -6 4 1 -18 x 4 -38 6 -2 2 4 x 1 12 12 -8 6 10 x 2 34 0 -12 8 1 x 3 21 3 -13 9 3 x 3 27 -6 4 1 -18 x 4 -38 0 2 3 -14 x 4 -26

Eliminate first column: component-wise 6 -2 2 4 x 1 12 12 -8 6 10 x 2 34 0 -4 2 2 x 2 10 3 -13 9 3 x 3 27 0 -12 8 1 x 3 21 -6 4 1 -18 x 4 -38 0 2 3 -14 x 4 -26 where Exercise: check by MATLAB

Eliminate first column: column-wise define and then Exercise: check Notation: by MATLAB

Eliminate second column: component-wise 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 -12 8 1 x 3 21 0 2 3 -14 x 4 -26 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 4 -13 x 4 -21 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 2 3 -14 x 4 -26

Eliminate second column: column-wise define Exercise: check Notation: , why? Give a simple explanation.

Eliminate third column: component-wise define then 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 4 -13 x 4 -21 6 -2 2 4 x 1 12 0 -4 2 2 x 2 10 0 0 2 -5 x 3 -9 0 0 0 -3 x 4 -3

LU-decomposition Notation: LU-decomposition Exercise: check Question: Do you have any effective method to write down matrix Question: why don’t we care about right hand side vector

Problems about LU-decomposition Question 1: what condition does LU-decomposition fail? Question 2: does any invertible matrix has LU-decomposition? Question 3: How to measure “goodness of LU-decomposition“? Question 4: what is order of performance of LU-decomposition (operation count)? Question 5: How to parallelize LU-decomposition? Question 6: How to implement LU-decomposition with help of GPU?

Out. Line • • Basic operation of matrix Example of Gaussian Elimination (GE) Formal description of GE MATLAB usage - website resource - “help” command - M-file

MATLAB website http: //www. mathworks. com/access/helpdesk/help/techdoc/matlab. html

MATLAB: create matrix

MATLAB: LU-factorization

Start MATLAB 2008

Command “help” : documentation != in C

Loop in MATLAB

Decision-making in MATLAB

IO in MATLAB

LU-factorization in MATLAB

M-file in MATLAB matvec. m Description of function matvec, write specificaiton of input parameter Consistency check Inner-product based

Move to working directory M-file

Execute M-file Construct matrix Execute M-file, like function call in C-language

Exercise: forward / backward substitution Forward substitution since backward substitution since Write MATLAB code to do forward substitution and backward substitution

Exercise: LU-decomposition You should find recursive structure of the decomposition