Linear Algebra Chapter 1 Linear Equations and Vectors

Linear Algebra Chapter 1 Linear Equations and Vectors 大葉大學 資訊 程系 黃鈴玲

1. 1 Matrices and Systems of Linear Equations Definition • An equation (方程式) in the variables (變數) x and y that can be written in the form ax+by=c, where a, b, and c are real constants (實數常數) (a and b not both zero), is called a linear equation (線性方程式). • The graph of this equation is a straight line in the x-y plane. • A pair of values of x and y that satisfy the equation is called a solution (解). system of linear equations (線性聯立方程式) Ch 1_2

Solutions for system of linear equations Figure 1. 1 Unique solution (唯一解) x+y= 5 2 x - y = 4 Lines intersect at (3, 2) Unique solution: x = 3, y = 2. Figure 1. 2 No solution (無解) – 2 x + y = 3 – 4 x + 2 y = 2 Lines are parallel. No point of intersection. No solutions. Figure 1. 3 Many solution (無限多解 ) 4 x – 2 y = 6 6 x – 3 y = 9 Both equations have the same graph. Any point on the graph is a solution. Ch 1_3 Many solutions.

Definition A linear equation in n variables x 1, x 2, x 3, …, xn has the form a 1 x 1 + a 2 x 2 + a 3 x 3 + … + an xn = b where the coefficients (係數) a 1, a 2, a 3, …, an and b are constants. 常見數系的英文名稱： natural number (自然數), integer (整數), rational number (有理數), real number (實數), complex number (複數) positive (正), negative (負) Ch 1_4

A linear equation in three variables corresponds to a plane in three-dimensional (三維) space. ※ Systems of three linear equations in three variables: Unique solution Ch 1_5

No solutions Many solutions Ch 1_6

How to solve a system of linear equations? Gauss-Jordan elimination. (高斯-喬登消去法) 1. 2節會介紹 Ch 1_7

Definition A matrix (矩陣) is a rectangular array of numbers. The numbers in the array are called the elements (元素) of the matrix. Matrices 注意矩陣左右兩邊是中括號不是直線，直線表示的是行列式。 Ch 1_8

Row (列) and Column (行) Submatrix (子矩陣) Ch 1_9

Size and Type Location aij表示在row i, column j 的元素值 也寫成 location (1, 3) = -4 Identity Matrices (單位矩陣) diagonal (對角線) 上都是 1，其餘都是 0，I 的下標表示size Ch 1_10

Relations between system of linear equations and matrices matrix of coefficient and augmented matrix 係數矩陣 擴大矩陣 隨堂作業： 5(f) Ch 1_11

Elementary Row Operations of Matrices 給定聯立方程式後， 不會改變解的一些轉換 Elementary Transformation 1. Interchange two equations. 2. Multiply both sides of an equation by a nonzero constant. 3. Add a multiple of one equation to another equation. 將左邊的轉換對應到矩陣上 Elementary Row Operation (基本列運算) 1. Interchange two rows of a matrix. (兩列交換) 2. Multiply the elements of a row by a nonzero constant. (某列的元素同乘一非零常數) 3. Add a multiple of the elements of one row to the corresponding elements of another row. (將一個列的倍數加進另一列裡) Ch 1_12

Example 1 Solving the following system of linear equation. 符號表示row equivalent Solution Equation Method Initial system: Eq 2+(– 2)Eq 1 Analogous Matrix Method Augmented matrix: Eq 3+(– 1)Eq 1 R 2+(– 2)R 1 R 3+(– 1)R 1 Ch 1_13

Eq 1+(– 1)Eq 2 Eq 3+(2)Eq 2 (– 1/5)Eq 3 Eq 1+(– 2)Eq 3 Eq 2+Eq 3 The solution is R 1+(– 1)R 2 R 3+(2)R 2 (– 1/5)R 3 R 1+(– 2)R 3 R 2+R 3 The solution is 隨堂作業： 7(d) Ch 1_14

Example 2 Solving the following system of linear equation. Solution (請先自行練習) Ch 1_17

Example 3 Solve the system Solution (請先自行練習) 隨堂作業： 10(d)(f) Ch 1_18

Summary Use row operations to [A: B] : A B i. e. , Def. [In : X] is called the reduced echelon form (簡化梯式 ) [AA: is. B]. Note. of 1. If the matrix of coefficients of a system of n equations in n variables that has a unique solution, then A is row equivalent to In (A In). 2. If A In, then the system has unique solution. Ch 1_19

Example 4 Many Systems Solving the following three systems of linear equation, all of which have the same matrix of coefficients. Solution R 2+(– 2)R 1 R 3+R 1 The solutions to the three systems are 隨堂作業： 13(b) Ch 1_20

Homework Exercise 1. 1： 1, 2, 4, 5, 6, 7, 10, 13 Ch 1_21

1. 2 Gauss-Jordan Elimination Definition A matrix is in reduced echelon form (簡化梯式) if 1. Any rows consisting entirely of zeros are grouped at the bottom of the matrix. (全為零的列都放在矩陣的下層) 2. The first nonzero element of each other row is 1. This element is called a leading 1. (每列第一個非零元素是 1，稱做leading 1) 3. The leading 1 of each row after the first is positioned to the right of the leading 1 of the previous row. (每列的leading 1出現在前一列leading 1的右邊，也就 是所有的leading 1會呈現由左上到右下的排列) 4. All other elements in a column that contains a leading 1 are zero. (包含leading 1的行裡所有其他元素都是 0) Ch 1_22

Examples for reduced echelon form ( ) ( ) 利用一連串的 elementary row operations，可讓 任何矩陣變成 reduced echelon form The reduced echelon form of a matrix is unique. 隨堂作業： 2(b)(d)(h) Ch 1_23

Gauss-Jordan Elimination System of linear equations augmented matrix reduced echelon form solution Ch 1_24

Example 1 Use the method of Gauss-Jordan elimination to find reduced echelon form of the following matrix. Solution pivot (樞軸，未來的 leading 1) pivot The matrix is the reduced echelon form of the given matrix. Ch 1_25

Example 2 Solve, if possible, the system of equations Solution (請先自行練習) The general solution (通解) to the system is 隨堂作業： 5(c) Ch 1_26

Example 3 This example illustrates that the general solution can involve a number of parameters. Solve the system of equations 變數個數 > 方程式個數 many sol. Solution Ch 1_27

Example 4 This example illustrates a system that has no solution. Let us try to solve the system Solution(自行練習) 0 x 1+0 x 2+0 x 3=1 The system has no solution. 隨堂作業： 5(d) Ch 1_28

Homogeneous System of linear Equations Definition A system of linear equations is said to be homogeneous (齊次) if all the constant terms (等號右邊的常數項) are zeros. Example: Observe that is a solution. Theorem 1. 1 A system of homogeneous linear equations in n variables always has the solution x 1 = 0, x 2 = 0. …, xn = 0. This solution is called the trivial solution. Ch 1_29

Homogeneous System of linear Equations Note. 除 trivial solution 外，可能還有其他解。 Example: The system has other nontrivial solutions. Theorem 1. 2 A system of homogeneous linear equations that has more variables than equations has many solutions. 隨堂作業： 8(e) Ch 1_30

Homework Exercise 1. 2: 2, 5, 8, 14 Ch 1_31

1. 3 The Vector Space Rn Rectangular Coordinate System (直角座標系) There are two ways of interpreting (5, 3) - it defines the location of a point in a plane - it defines the position vector • • • the origin (原點)：(0, 0) the position vector： the initial point of : O the terminal point of : A(5, 3) ordered pair (序對): (a, b) Figure 1. 5 Ch 1_32

Example 1 Sketch the position vectors Figure 1. 6 . Ch 1_33

R 2 → R 3 Figure 1. 7 Ch 1_34

Definition Let be a sequence of n real numbers. The set of all such sequences is called n-space and is denoted Rn. u 1 is the first component (分量) of , u 2 is the second component and so on. Example R 4 is the sets of sequences of four real numbers. For example, (1, 2, 3, 4) and (-1, 3, 5. 2, 0) are in R 4. R 5 is the set of sequences of five real numbers. For example, (-1, 2, 0, 3, 9) is in this set. Ch 1_35

Addition and Scalar Multiplication Definition Let be two elements of Rn. We say that u and v are equal if u 1 = v 1, …, un = vn. Thus two element of Rn are equal if their corresponding components are equal. Definition Let be elements of Rn and let c be a scalar. Addition and scalar multiplication are performed as follows Addition: Scalar multiplication : Note. (1) u, v Rn u+v Rn (Rn is closed under addition)(加法封閉性) (2) u Rn, c R cu Rn (Rn is closed under scalar Ch 1_36

Example 2 Let u = ( – 1, 4, 3, 7) and v = ( – 2, – 3, 1, 0) be vector in R 4. Find u + v and 3 u. Solution Example 3 Consider the vector (4, 1) and (2, 3), we get (4, 1) + (2, 3) = (6, 4). Figure 1. 8 Ch 1_37

In general, if u and v are vectors in the same vector space, then u + v is the diagonal (對角線) of the parallelogram (平行四邊 形) defined by u and v. Figure 1. 9 Ch 1_38

Example 4 Consider the scalar multiple of the vector (3, 2) by 2, we get 2(3, 2) = (6, 4) Observe in Figure 4. 6 that (6, 4) is a vector in the same direction as (3, 2), and 2 times it in length. Figure 1. 10 Ch 1_39

c>1 0<c<1 – 1 < c < 0 c < – 1 Figure 1. 11 In general, the direction of cu will be the same as the direction of u if c > 0, and the opposite direction to u if c < 0. The length of cu is |c| times the length of u. Ch 1_40

Special Vectors The vector (0, 0, …, 0), having n zero components, is called the zero vector of Rn and is denoted 0. Negative Vector The vector (– 1)u is written –u and is called the negative of u. It is a vector having the same magnitude (量) as u, but lies in the opposite direction to u. u -u Subtraction is performed on elements of Rn by subtracting corresponding components. For example, in R 3, (5, 3, -6) – (2, 1, 3) = (3, 2, -9) Ch 1_41

Theorem 1. 3 Let u, v, and w be vectors in Rn and let c and d be scalars. (a) u + v = v + u (b) u + (v + w) = (u + v) + w (c) u + 0 = 0 + u = u (d) u + (–u) = 0 (e) c(u + v) = cu + cv (f) (c + d)u = cu + du (g) c(du) = (cd)u (h) 1 u = u Figure 1. 12 Commutativity of vector addition u + v = v + u Ch 1_42

Linear Combinations of Vectors We call au +bv + cw a linear combination (線性組合) of the vectors u, v, and w. Example 5 Let u = (2, 5, – 3), v = ( – 4, 1, 9), w = (4, 0, 2). Determine the linear combination 2 u – 3 v + w. Solution 隨堂作業： 7(e) Ch 1_43

Column Vectors Row vector: Column vector: 向量的另一種表示法 We defined addition and scalar multiplication of column vectors in Rn in a componentwise manner: and 向量加法可視為矩陣運算 Ch 1_44

Subspaces of Rn We now introduce subsets of the vector space Rn that have all the algebraic properties of Rn. These subsets are called subspaces. Definition A subset S of Rn is a subspace if it is closed under addition and under scalar multiplication. Recall: (1) u, v S u+v S (S is closed under addition)(加法封閉性) (2) u S, c R cu S (S is closed under scalar multiplication) (純量乘法封閉性) Ch 1_45

Example 6 Consider the subset W of R 2 of vectors of the form (a, 2 a). Show that W is a subspace of R 2. 隨堂作業： 10(d) Proof Let u = (a, 2 a), v = (b, 2 b) W, and k R. u + v = (a, 2 a) + (b, 2 b) = (a+ b, 2 a + 2 b) = (a + b, 2(a + b)) W and ku = k(a, 2 a) = (ka, 2 ka) W Thus u + v W and ku W. W is closed under addition and scalar multiplication. W is a subspace of R 2. Observe: W is the set of vectors that can be written a(1, 2). Figure 1. 13 Ch 1_46

Example 7 Consider the homogeneous system of linear equations. It can be shown that there are many solutions x 1=2 r, x 2=5 r, x 3=r. We can write these solutions as vectors in R 3 as (2 r, 5 r, r). Show that the set of solutions W is a subspace of R 3. 隨堂作業： 13 Proof Let u = (2 r, 5 r, r), v = (2 s, 5 s, s) W, and k R. u + v = (2(r+s), 5(r+s), r+s) W and ku = (2 kr, 5 kr, kr) W Thus u + v W and ku W. W is a subspace of R 3. Observe: W is the set of vectors that can be written r(2, 5, 1). Figure 1. 14 Ch 1_47

Homework Exercise 1. 3: 7, 9, 10, 12, 13 Ch 1_48

1. 4 Basis and Dimension 向量空間的某些子集合(subset)本身也是向量空間，稱為子空間 (subspace) Basis: a set of vectors which are used to describe a vector space. Standard Basis of Rn Consider the vectors (1, 0, 0), (0, 1, 0), (0, 0, 1) in R 3. These vectors have two very important properties: (i) They are said to span R 3. That is, we can write an arbitrary vector (x, y, z) as a linear combination (線性組合) of the three vectors: For any (x, y, z) R 3 (x, y, z) = x(1, 0, 0) + y(0, 1, 0) + z(0, 0, 1) (ii) They are said to be linearly independent (線性獨立). If p(1, 0, 0) + q(0, 1, 0) +r(0, 0, 1) = (0, 0, 0) p = 0, q = 0, r = 0 is the unique solution. A set of vectors that satisfies the two preceding properties is called a basis. Ch 1_49

There are many bases for R 3 – sets that span R 3 and are linearly independent. For example, the set {(1, 2, 0), (0, 1, -1), (1, 1, 2)} is a basis for R 3. The set {(1, 0, 0), (0, 1, 0), (0, 0, 1)} is the most important basis for R 3. It is called the standard basis of R 3. R 2 : two-dimensional space (二維空間) R 3 : three-dimensional space 觀察: dimension數等於basis中的向量個數 (故成為dimension定義) The set {(1, 0, …, 0), (0, 1, …, 0), …, (0, …, 1)} of n vectors is the standard basis for Rn. The dimension (維度) of Rn is n. 隨堂作業： 1 Ch 1_50

Span, Linear Independence, and Basis The vectors v 1, v 2, and v 3 are said to span a space if every vector v in the space can be expressed as a linear combination of them, v = av 1 + bv 2 + cv 3. The vectors v 1, v 2, and v 3 are said to be linearly independent if the identity pv 1 + qv 2 + rvm = 0 is only true for p = 0, q = 0, r = 0. A basis for a space is a set that spans the space and is linearly independent. The number of vectors in a basis is called the dimension of the space. Ch 1_51

Example 1 Consider the subset W of R 3 consisting of vectors of the form (a, b, a+b). The vector (2, 5, 7) W, whereas (2, 5, 9) W. Show that W is a subspace of R 3. Proof. Let u=(a, b, a+b) and v=(c, d, c+d) be vectors in W and k be a scalar. (1) u+v = (a, b, a+b) + (c, d, c+d) = (a+c, b+d, (a+c)+(b+d)) u+v W (2) ku = k(a, b, a+b) = (ka, kb, ka+kb) ku W W is closed under addition and scalar multiplication. W is a subspace of R 3. Ch 1_52

Example 1 (continue) Separate the variables in the above arbitrary vector u. u = (a, b, a+b) = (a, 0, a) + (0, b, b) = a(1, 0, 1) + b(0, 1, 1) The vectors (1, 0, 1) and (0, 1, 1) thus span W. Furthermore, p(1, 0, 1) + q(0, 1, 1) = (0, 0, 0) leads to p=0 and q=0. The two vectors (1, 0, 1) and (0, 1, 1) are thus linearly independent. The set {(1, 0, 1), (0, 1, 1)} is therefore a basis for W. The dimension of W, dim(W)= 2. 有加法乘法封閉性 隨堂作業： 4(a) Ch 1_53

Example 2 Consider the subset V of R 3 of vectors of the form (a, 2 a, 3 a). Show that V is a subspace of R 3 and find a basis. Sol. (subspace) Let u=(a, 2 a, 3 a) and v=(b, 2 b, 3 b) be vectors in V and k be a scalar. (1) u+v = (a+b, 2(a+b), 3(a+b)) u+v V (2) ku = k(a, 2 a, 3 a) = (ka, 2 ka, 3 ka) ku V V is closed under addition and scalar multiplication. V is a subspace of R 3. (basis) u = (a, 2 a, 3 a) = a(1, 2, 3) {(1, 2, 3)} is a basis for V. dim(V) = 1. 隨堂作業： 4(c) Ch 1_54

Example 3 Consider the following system of homogeneous linear equations. It can be shown that there are many solutions x 1 = 3 r - 2 s, x 2 = 4 r, x 3 = r, x 4 = s. Write these solution as vectors in R 4, (3 r - 2 s, 4 r, r, s). It can be shown that this set of vectors is a subspace W of R 4. Find a basis for W and give its dimension. Sol. (1) (3 r - 2 s, 4 r, r, s) = r(3, 4, 1, 0) + s( -2, 0, 0, 1) (2) If p(3, 4, 1, 0) + q( -2, 0, 0, 1) = (0, 0, 0, 0), then p=0, q=0. {(3, 4, 1, 0), ( -2, 0, 0, 1) } is a basis for W. dim(W) = 2. 隨堂作業： 11 Ch 1_55

Exercise 7 State with a brief explanation whether the following statements are true or false. (a) The set {(1, 0, 0), (0, 1, 0)} is the basis for a two-dimensional subspace of R 3. (b) The set {(1, 0, 0)} is the basis for a one-dimensional subspace of R 3. (c) The vector (a, 2 a, b) is an arbitrary vector in the plane spanned by the vectors (1, 2, 0) and (0, 0, 1). (d) The vector (a, b, 2 a-b) is an arbitrary vector in the plane spanned by the vectors (1, 0, 2) and (0, 1, -1). (e) The set {(1, 0, 0), (0, 1, 0), (1, 1, 0)} is a basis for a subspace of R 3. (f) R 2 is a subspace of R 3. Ch 1_56

Homework Exercise 1. 4: 1, 4, 11, 12 Ch 1_57

1. 5 Dot Product, Norm, Angle, and Distance In this section we develop a geometry for the vector space Rn. Definition Let be two vectors in Rn. The dot product (內積) of u and v is denoted u‧v and is defined by. The dot product assigns a real number to each pair of vectors. The dot product is a tool that is used to build the geometry of Rn. Example Find the dot product of u = (1, – 2, 4) and v = (3, 0, 2) Solution Ch 1_58

Properties of the Dot Product Let u, v, and w be vectors in Rn and let c be a scalar. Then 1. u‧v = v‧u 2. (u + v)‧w = u‧w + v‧w 3. cu‧v = c(u‧v) = u‧cv 4. u‧u 0, and u‧u = 0 if and only if u = 0 Proof 1. 2. 隨堂作業： 2(b) Ch 1_59

Norm (length) of a Vector in Rn Figure 1. 17 Definition The norm (length or magnitude) of a vector u = (u 1, …, un) in Rn is denoted ||u|| and defined by Note: The norm of a vector can also be written in terms of the dot product Ch 1_60

Example Find the norm of the vectors u = (1, 3, 5) of R 3 and v = (3, 0, 1, 4) of R 4. Solution Definition A unit vector is a vector whose norm is 1. If v is a nonzero vector, then the vector is a unit vector in the direction of v. This procedure of constructing a unit vector in the same direction as a given vector is called normalizing the vector. Ch 1_61

Example 1 Find the norm of the vector (2, – 1, 3). Normalize this vector. Solution The norm of (2, – 1, 3) is The normalized vector is The vector may also be written This vector is a unit vector in the direction of (2, – 1, 3). 隨堂作業： 6(b), 10(d) Ch 1_62

Angle between Vectors (R 2) Let u=(a, b) and v=(c, d). Find the angle between u and v. The law of cosines gives We get that Figure 1. 18 0 Ch 1_63

Angle between Vectors (R n) Definition Let u and v be two nonzero vectors in Rn. The cosine of the angle between these vectors is Example 2 Determine the angle between the vectors u = (1, 0, 0) and v = (1, 0, 1) in R 3. Solution Thus the angle between u and v is /4 (or 45 ). 隨堂作業： 13(c) Ch 1_64

Definition Two nonzero vectors are orthogonal (正交) if the angle between them is a right angle (直角). Theorem 1. 4 Two nonzero vectors u and v are orthogonal if and only if u‧v = 0. 可寫成u v Proof Example The vectors (2, – 3, 1) and (1, 2, 4) are orthogonal since (2, – 3, 1)‧(1, 2, 4) = (2 1) + (– 3 0) + (1 4) = 2 – 6 + 4 = 0. Ch 1_65

Properties of Standard basis of Rn (1, 0), (0, 1) are orthogonal unit vectors in R 2. (1, 0, 0), (0, 1, 0), (0, 0, 1) are orthogonal unit vectors in R 3. We call a set of unit pairwise orthogonal vectors an orthonormal set. The standard basis for Rn, {(1, 0, …, 0), (0, 1, 0, …, 0), …, (0, …, 0, 1)} is an orthonormal set. 隨堂作業： 16(d) Ch 1_66

Example 3 (a) Let w be a vector in Rn. Let W be the set of vectors that are orthogonal to w. Show that W is a subspace of Rn. (b) Find a basis for the subset W of vectors in R 3 that are orthogonal to w=(1, 3, 1). Give the dimension and a geometrical description of W. Solution (a) Let u, v W. Since u w and v w, we have u w=0 and v w=0. (u+v) w = u w + v w = 0 u+v w If c is a scalar, c(u w) = cu w = 0 u+v W cu w cu W W is a subspace of Rn. (b) Let (a, b, c) W and (a, b, c) w, then (a, b, c) (1, 3, 1)=0 a+3 b+c=0 W is the set {(a, b, -a - 3 b) | a, b R} Since (a, b, -a - 3 b) = a(1, 0, -1) + b(0, 1, -3). It is clear that {(1, 0, -1), (0, 1, -3)} is a basis for W dim(W)=2 Ch 1_67

Example 3 (continue) W is the plane in R 3 defined by (1, 0, -1) and (0, 1, -3). Figure 1. 19 隨堂作業： 20 Ch 1_68

Distance between Points The distance between x=(x 1, x 2) and y =(y 1, y 2) is Generalize this expression to Rn. Def. Let x=(x 1, x 2, …, xn) and y=(y 1, y 2, …, yn) be two points in Rn. The distance between x and y is denoted d(x, y) and is defined by Note: We can also write this distance as follows. x-y x y Example 4 Determine the distance between the points x = (1, – 2 , 3, 0) and y = (4, 0, – 3, 5) in R 4. Solution Ch 1_69

Example 5 Prove that the distance in Rn has the following symmetric property: d(x, y)=d(y, x) for any x, y Rn. Solution Let Ch 1_70

Theorem 1. 5 The Cauchy-Schwartz Inequality. If u and v are vectors in Rn then Here u v. denoted the absolute value (絕對值) of the number Ch 1_71

Theorem 1. 6 Let u and v be vectors in Rn. (a) Triangle Inequality (三角不等式): ||u + v|| ||u|| + ||v||. (a) Pythagorean theorem (畢氏定理): If u‧v = 0 then ||u + v||2 = ||u||2 + ||v||2. Figure 1. 21 Ch 1_72

Homework Exercise 1. 5: 2, 6, 10, 13, 16, 20, 33 (1. 6節跳過) Ch 1_73