Lecture 9 Some Linear Algebra Nicholas Ruozzi University

Lecture 9: Some Linear Algebra Nicholas Ruozzi University of Texas at Dallas

Length of Vectors • 2

Linear Combinations • 3

Linear Independence • Linearly independent vectors 4

Linear Independence • Linearly dependent vectors 5

Linear Independence • Orthogonal vectors 6

Matrix Rank • 7

Orthonormal Vectors • 8

Orthonormal Vectors • 9

Eigenvalues • 10

Eigenvalues of Symmetric Matrices • 11

Examples • 12

Eigen-decomposition • 13

Eigen-decomposition • 14

Eigen-decomposition • 15

Useful Properties of Positive Semidefinite Matrices • 16

Useful Properties of Positive Semidefinite Matrices • 17

Useful Properties of Positive Semidefinite Matrices • 18

Least Square Solutions to Linear Systems • 19

Least Square Solutions to Linear Systems • 20

Least Square Solutions to Linear Systems • 21

Least Square Solutions to Linear Systems • 22

Least Square Solutions to Linear Systems • 23

Least Square Solutions to Linear Systems • 24

Least Square Solutions to Linear Systems • 25

The Positive Semidefinite Matrices • 26

Projection on PSD Matrices • 27

Projection on PSD Matrices • 28

Projection on PSD Matrices • 29

Projection on PSD Matrices • 30

Inner Products for Matrices • 31

Inner Products for Matrices • 32

Projection on PSD Matrices • 33

Projection on PSD Matrices • 34

Projection on PSD Matrices • 35

Semidefinite Programming (SDP) • 36

Semidefinite Programming • 37

Rayleigh-Ritz Theorem • 38

Rayleigh-Ritz Theorem • 39

Computing Eigenvalues • There a variety of methods for computing the eigenvalues/eigenvectors of a matrix • Power iteration: an iterative method that only requires performing matrix multiplication and dot products • QR factorization: based on Gram-Schmidt orthogonalization (we won’t cover it here, take CS 4334: Numerical Analysis!) 40

Power Iteration • 41