Engineering Analysis Chapter 6 Eigenvalues Basil Hamed we

Engineering Analysis Chapter 6 Eigenvalues Basil Hamed

• we will be concerned with the equation Ax = λx. This equation occurs in many applications of linear algebra. If the equation has a nonzero solution x, then λ is said to be an eigenvalue of A and x is said to be an eigenvector belonging to λ. • Wherever there are vibrations, there are eigenvalues, the natural frequencies of the vibrations. If you have ever tuned a guitar, you have solved an eigenvalue problem. When engineers design structures, they are concerned with the frequencies of vibration of the structure. • The eigenvalues of a boundary value problem can be used to determine the energy states of an atom or critical loads that cause buckling in a beam. Basil Hamed 2

6. 1 Eigenvalues and Eigenvectors Many application problems involve applying a linear transformation repeatedly to a given vector. The key to solving these problems is to choose a coordinate system or basis that is in some sense natural for the operator and for which it will be simpler to do calculations involving the operator. With respect to these new basis vectors (eigenvectors) we associate scaling factors (eigenvalues) that represent the natural frequencies of the operator. Basil Hamed 3

6. 1 Eigenvalues and Eigenvectors Definition Let A be an n×n matrix. A scalar λ is said to be an eigenvalue or a characteristic Value of A if there exists a nonzero vector x such that A x = λ x. The vector x is said to be an eigenvector or a characteristic vector belonging to λ. EXAMPLE 2 Let Solution Basil Hamed 4

6. 1 Eigenvalues and Eigenvectors The equation Ax = λx can be written in the form Thus, λ is an eigenvalue of A if and only if (1) has a nontrivial solution. Basil Hamed 5

6. 1 Eigenvalues and Eigenvectors Equation (1) will have a nontrivial solution if and only if A − λI is singular, or, equivalently det(A − λI) =0 (2) If the determinant in (2) is expanded, we obtain an nth-degree polynomial in the variable λ: • This polynomial is called the characteristic polynomial, and equation (2) is called the characteristic equation, for the matrix A. • The roots of the characteristic polynomial are the eigenvalues of A. • If roots are counted according to multiplicity, then the characteristic polynomial will have exactly n roots • Thus, A will have n eigenvalues, some of which may be repeated and some of which may be complex numbers. Basil Hamed 6

6. 1 Eigenvalues and Eigenvectors EXAMPLE 3 Find the eigenvalues and the corresponding eigenvectors of the matrix Basil Hamed 7

6. 1 Eigenvalues and Eigenvectors Basil Hamed 8

6. 1 Eigenvalues and Eigenvectors Basil Hamed 9

6. 1 Eigenvalues and Eigenvectors Basil Hamed 10

6. 1 Eigenvalues and Eigenvectors Basil Hamed 11

6. 1 Eigenvalues and Eigenvectors Example Find the eigenvalues of Solution The characteristic polynomial of A is The eigenvalues of A must therefore satisfy the cubic equation Thus the eigenvalues of A are Basil Hamed 12

6. 1 Eigenvalues and Eigenvectors EXAMPLE 4 Let Find the eigenvalues and the corresponding eigenspaces Solution Thus, the characteristic polynomial has roots λ 1 = 0, λ 2 = λ 3 = 1. The eigenspace corresponding to λ 1 = 0 is N(A), which we determine in the usual manner: Basil Hamed 13

6. 1 Eigenvalues and Eigenvectors Basil Hamed 14

6. 1 Eigenvalues and Eigenvectors Basil Hamed 15

6. 1 Eigenvalues and Eigenvectors If , λ = 2 then 3 becomes Solving this system using Gaussian elimination yields Since are linearly independent, these vectors form a basis for the eigenspace corresponding to λ = 2. Basil Hamed 16

6. 1 Eigenvalues and Eigenvectors If λ = 1 , then 3 becomes Solving this system yields Thus the eigenvectors corresponding to λ = 1 are the nonzero vectors of the form . is a basis for the eigenspace corresponding to λ = 1 Basil Hamed . 17

6. 1 Eigenvalues and Eigenvectors EXAMPLE Let Find the eigenvalues and the corresponding eigenvector Solution Basil Hamed 18

6. 1 Eigenvalues and Eigenvectors Basil Hamed 19

6. 1 Eigenvalues and Eigenvectors Basil Hamed 20

6. 1 Eigenvalues and Eigenvectors Basil Hamed 21

6. 1 Eigenvalues and Eigenvectors Basil Hamed 22

6. 1 Eigenvalues and Eigenvectors EXAMPLE 5 Let Compute the eigenvalues of A and find bases for the corresponding eigenspaces. Solution The roots of the characteristic polynomial are λ 1 = 1 + 2 i, λ 2 = 1 − 2 i. Basil Hamed 23

6. 1 Eigenvalues and Eigenvectors A square matrix A is invertible if and only if λ= 0 is not an eigenvalue of A. Basil Hamed 24

6. 1 Eigenvalues and Eigenvectors The sum of the diagonal elements of A is called the trace of A and is denoted by tr(A). Basil Hamed 25

6. 1 Eigenvalues and Eigenvectors EXAMPLE 6 If Then det(A) = − 5 + 18 = 13 and The characteristic polynomial of A is given by tr(A) = 5 − 1 = 4 and hence the eigenvalues of A are λ 1 = 2 + 3 i and λ 2 = 2 − 3 i. Note that Basil Hamed 26

6. 1 Eigenvalues and Eigenvectors If A is an n x n triangular matrix (upper triangular, lower triangular, or diagonal), then the eigenvalues of A are the entries on the main diagonal of A Example By inspection, the eigenvalues of the lower triangular matrix Basil Hamed 27

6. 1 Eigenvalues and Eigenvectors Basil Hamed 28

6. 1 Eigenvalues and Eigenvectors Basil Hamed 29

6. 2 Systems of Linear Differential Equations A differential equation is an equation that involves an unknown function and its derivatives. An important, simple example of a differential equation is where r is a constant. The idea here is to find a function x(t) that will satisfy the given differential equation. This differential equation is discussed further subsequently. linear algebra is helpful in the formulation and solution of differential equations. Homogeneous Linear Systems We consider the first-order homogeneous linear system of differential equations, Basil Hamed 30

6. 2 Systems of Linear Differential Equations We can write above eq. in matrix form by letting Basil Hamed 31

6. 2 Systems of Linear Differential Equations Eigenvalues play an important role in the solution of systems of linear differential equations. In this section, we see how they are used in the solution of systems of linear differential equations with constant coefficients. We begin by considering systems of first-order equations of the form Basil Hamed 32

6. 2 Systems of Linear Differential Equations then the system can be written in the form Let us consider the simplest case first. When n = 1, the system is simply y Clearly, any function of the form Basil Hamed 33

6. 2 Systems of Linear Differential Equations Basil Hamed 34

6. 2 Systems of Linear Differential Equations is a solution of the system. Basil Hamed 35

6. 2 Systems of Linear Differential Equations Example For the system the matrix has eigenvalues λ 1 = 2 and λ 2 = 3 with respective associated eigenvectors These eigenvectors are automatically linearly independent, since they are associated with distinct eigenvalues Hence the general solution to the given system is Basil Hamed 36

6. 2 Systems of Linear Differential Equations In terms of components, this can be written as Example Consider the following homogeneous linear system of differential equations: Solution The characteristic polynomial of A is Basil Hamed 37

6. 2 Systems of Linear Differential Equations so the eigenvalues of A are λ 1 =1, λ 2=2, and λ 3=4. Associated eigenvectors are respectively. The general solution is then given by where b 1, b 2, and b 3 are arbitrary constants. Basil Hamed 38

6. 2 Systems of Linear Differential Equations EXAMPLE For the linear system of previous Example, solve the initial value problem determined by the initial conditions x 1(0)=4, x 2(0)= 6, and x 3(0) = 8. Solution We write our general solution in the form x = Pu as or Therefore, the solution to the initial value problem is Basil Hamed 39

6. 2 Systems of Linear Differential Equations Basil Hamed 40

6. 2 Systems of Linear Differential Equations we see that Basil Hamed 41

6. 2 Systems of Linear Differential Equations EXAMPLE 2 Solve the system Solution Let Basil Hamed 42

6. 2 Systems of Linear Differential Equations Basil Hamed 43

6. 2 Systems of Linear Differential Equations Higher Order Systems Given a second-order system of the form we may translate it into a first-order system by setting Basil Hamed 44

6. 2 Systems of Linear Differential Equations and then and The equations can be combined to give the 2 n × 2 n first-order system Basil Hamed 45

6. 2 Systems of Linear Differential Equations EXAMPLE 3 Solve the initial value problem Solution Basil Hamed 46

6. 2 Systems of Linear Differential Equations The coefficient matrix for this system, has eigenvalues Corresponding to these eigenvalues are the eigenvectors Basil Hamed 47

6. 2 Systems of Linear Differential Equations Thus, the solution will be of the form We can use the initial conditions to find c 1, c 2, c 3, and c 4. For t = 0, we have or, equivalently, Basil Hamed 48

6. 2 Systems of Linear Differential Equations Therefore, Basil Hamed 49

6. 2 Systems of Linear Differential Equations In general, if we have an mth-order system of the form where each Ai is an n×n matrix, we can transform it into a first-order system by setting We will end up with a system of the form Basil Hamed 50

6. 3 Diagonalization Definition Theorem 6. 3. 2 An n × n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors. Basil Hamed 51

6. 3 Diagonalization Basil Hamed 52

6. 3 Diagonalization It follows from remark 4 that and, in general EXAMPLE 1 Let Basil Hamed 53

6. 3 Diagonalization It follows that and Basil Hamed 54

6. 3 Diagonalization EXAMPLE 2 Let It follows that =A Basil Hamed 55

6. 3 Diagonalization Even though λ = 1 is a multiple eigenvalue, the matrix can still be diagonalized since there are three linearly independent eigenvectors. Note also that If an n × n matrix A has fewer than n linearly independent eigenvectors, we say that A is defective. It follows from Theorem 6. 3. 2 that a defective matrix is not diagonalizable EXAMPLE 3 Let Basil Hamed 56

6. 3 Diagonalization Basil Hamed 57

6. 3 Diagonalization Basil Hamed 58

6. 3 Diagonalization EXAMPLE Finding a Matrix P That Diagonalizes a Matrix A Solution and we found the following bases (in section 6. 1) for the eigenspaces: There are three basis vectors in total, so the matrix A is diagonalizable and Basil Hamed 59

6. 3 Diagonalization diagonalizes A. As a check, the reader should verify that Basil Hamed 60

6. 3 Diagonalization EXAMPLE A Matrix That Is Not Diagonalizable Find a matrix P that diagonalizes Solution The characteristic polynomial of A is so the characteristic equation is Basil Hamed 61

6. 3 Diagonalization Thus the eigenvalues of A are λ 1 =1, λ 2, 3=2 and. We leave it for the reader to show that bases for the eigenspaces are Since A 3 x 3 is a matrix and there are only two basis vectors in total, A is not diagonalizable Basil Hamed 62

6. 3 Diagonalization EXAMPLE Let Basil Hamed 63

6. 3 Diagonalization The solutions are the vectors of the form Basil Hamed 64

6. 3 Diagonalization EXAMPLE 4 Let A and B both have the same eigenvalues Basil Hamed 65

6. 3 Diagonalization The matrix exponential (6) occurs in a wide variety of applications. In the case of a diagonal matrix Basil Hamed 66

6. 3 Diagonalization the matrix exponential is easy to compute: It is more difficult to compute the matrix exponential for a general n × n matrix A. If, however, A is diagonalizable, then Basil Hamed 67

6. 3 Diagonalization Basil Hamed 68

6. 3 Diagonalization The matrix exponential can be applied to the initial value problem studied in Section 6. 2. In the case of one equation in one unknown, the solution is Basil Hamed 69

6. 3 Diagonalization If, as in (8), we set and then Thus, the solution of is simply Basil Hamed 70

6. 3 Diagonalization The form of this solution looks where xi was an eigenvector belonging to λi for i = 1, . . . , n. The ci’s that satisfied the initial conditions were determined by solving a system with coefficient matrix X = (x 1, . . . , xn). If A is diagonalizable, we can write (9) in the form Basil Hamed 71

6. 3 Diagonalization Thus To summarize, the solution to the initial value problem (7) is given by If A is diagonalizable, this solution can be written in the form Basil Hamed 72

6. 3 Diagonalization EXAMPLE 7 Use the matrix exponential to solve the initial value problem where (This problem was solved in Example 1 of Section 6. 2. ) Basil Hamed 73

6. 3 Diagonalization and the solution is given by Basil Hamed 74

6. 3 Diagonalization EXAMPLE 8 Use the matrix exponential to solve the initial value problem where Basil Hamed 75

6. 3 Diagonalization The solution to the initial value problem is given by Basil Hamed 76

6. 7 Positive Definite Matrices Basil Hamed 77

6. 7 Positive Definite Matrices Quadratic Forms Up to now, we have been interested primarily in linear equations—that is, in equations of the form The expression on the left side of this equation, is a function of n variables, called a linear form. In a linear form, all variables occur to the first power, and there are no products of variables in the expression. Here, we will be concerned with quadratic forms, which are functions of the form Basil Hamed 78

6. 7 Positive Definite Matrices (1) (2) (3) The terms in a quadratic form that involve products of different variables are called the cross-product terms. Thus, in 2 the last term is a cross-product term, and in 3 the last three terms are cross-product terms Basil Hamed 79

6. 7 Positive Definite Matrices If we follow the convention of omitting brackets on the resulting 1 x 1 matrices, then 2 can be written in matrix form as (4) and 3 can be written as (5) Basil Hamed 80

6. 7 Positive Definite Matrices Basil Hamed 81

6. 7 Positive Definite Matrices DEFINITION Basil Hamed 82

6. 7 Positive Definite Matrices Now, it’s not always easy to tell if a matrix is positive definite. Quick, is this matrix? Hard to tell just by looking at it. One way to tell if a matrix is positive definite is to calculate all the eigenvalues and just check to see if they’re all positive calculating eigenvalues can be a real pain. Especially for large matrices. So we’re going to learn some easier ways tot ell if a matrix is positive definite. A matrix is positive definite if it’s symmetric and all its pivots are positive Basil Hamed 83

6. 7 Positive Definite Matrices Pivots are, in general, way easier to calculate than eigenvalues. Just perform elimination and examine the diagonal terms If we perform elimination (subtract 2× row 1 from row 2) we get The pivots are 1 and − 3. In particular, one of the pivots is − 3, and so the matrix is not positive definite. Basil Hamed 84

6. 7 Positive Definite Matrices THEOREM A symmetric matrix A is positive definite if and only if all the eigenvalues of A are positive. EXAMPLE Showing That a Matrix Is Positive Definite Solution The characteristic equation of A is Basil Hamed 85

6. 7 Positive Definite Matrices Our next objective is to give a criterion that can be used to determine whether a symmetric matrix is positive definite without finding its eigenvalues. To do this, it will be helpful to introduce some terminology. If Basil Hamed 86

6. 7 Positive Definite Matrices is a square matrix, then the principal submatrices of A are the submatrices formed from the first r rows and r columns of A for. These submatrices are THEOREM A symmetric matrix A is positive definite if and only if the determinant of every principal submatrix is positive. If det(A)=0, then A may be Indefinite or what is known Positive Semi definite or Negative Semi definite. Basil Hamed 87

6. 7 Positive Definite Matrices EXAMPLE Working with Principal Submatrices The matrix is positive definite since Basil Hamed 88

6. 7 Positive Definite Matrices Example- Is the following matrix positive definite? Basil Hamed 89

6. 7 Positive Definite Matrices Basil Hamed 90

6. 7 Positive Definite Matrices If A is an n × n diagonal matrix Then A is: 1. Positive definite if and only if di>0 for i= 1, 2, . . . , n, 2. Negative definite if and only if di<0 for i= 1, 2, . . . , n, 3. Positive semi definite if and only if di≥ 0 for i= 1, 2, . . . , n, 4. Negative semi definite if and only if di≤ 0 for i= 1, 2, . . . , n, 5. Indefinite if and only if di>0 for some indices i, 1≤i≤n, and negative for other indices Basil Hamed 91

6. 7 Positive Definite Matrices Example Consider the matrix Then and we have Therefore, even though all of the entries of A are positive, A is not positive definite. Basil Hamed 92

6. 7 Positive Definite Matrices Example Consider the matrix Then which can be seen to be always nonnegative. Furthermore, QA(x; y) = 0 if and only if x = y and y = 0, so for all nonzero vectors (x; y), QA(x; y) > 0 and A is positive definite, even though A does not have all positive entries. Basil Hamed 93

6. 7 Positive Definite Matrices Example Consider the diagonal matrix Then which can plainly be seen to be positive except when (x; y; z) = (0; 0; 0). Therefore, A is positive definite. Basil Hamed 94

6. 7 Positive Definite Matrices Example All of the principal minors are nonnegative (1, 0, 0) A is not positive Semi definite; it is actually indefinite because we have eigenvalues positive and negative Basil Hamed 95

6. 7 Positive Definite Matrices Example Let We have Basil Hamed 96

6. 7 Positive Definite Matrices Basil Hamed 97

6. 7 Positive Definite Matrices Example find the quadratic form Example determine if the following matrices are positive definite a) b) Basil Hamed 98

6. 7 Positive Definite Matrices Basil Hamed 99

6. 7 Positive Definite Matrices Basil Hamed 100