Linear Spaces Row and Columns Spaces From D

Linear Spaces Row and Columns Spaces From: D. A. Harville, Matrix Algebra from a Statistician’s Perspective, Springer. Chapter 4

Definitions - I • Column Space of mxn matrix A ≡ set of all m-dimensional column vectors that can be expressed as linear combinations of columns of A • Row Space of mxn matrix A ≡ set of all n-dimensional row vectors that can be expressed as linear combinations of rows of A

Definitions - II • Linear Spaces: A non-empty set V of matrices of the same dimensions is a linear space if: 1) For every A in V, and every B in V, A+B is in V 2) For every A in V, and every scalar k, k. A is in V 3) A 1, …, Ak in V and scalars x 1, …, xk x 1 A 1+…+xk. Ak is in V • Examples • • • Column space of any mxn matrix is a linear space of mx 1 vectors Set containing all mxn matrices is a linear space Set of all nxn symmetric matrices is a linear space (sums and scalar multiples) All linear spaces contain the null matrix of correct dimension {0} is a linear space with only the null matrix included

Notation • C(A) ≡ Column space of the matrix A • R(A) ≡ Row space of the matrix A • Rm n ≡ Linear space of all mxn matrices • Rn ≡ Linear space of all nx 1 column vectors (1 xn row vectors) • R(In) = Rn ≡ Linear space of all 1 xn row vectors • C(In) = Rn ≡ Linear space of all nx 1 column vectors • x S x is an element of S • x S x is not an element of S

2 Lemmas Involving Row/Column Spaces and Linear Spaces Lemma 4. 1. 1. For any matrix A, y C(A) if and only if y’ R(A’) y C(A) y = Ax for some x y’ = (Ax)’ = x’A’ y’ R(A’) y’ = x’A’ for some x (y’)’ = (x’A’)’ = Ax y C(A) Lemma 4. 1. 2. B ≡ mxn V ≡ Linear space of mxn matrices Then for any A V, A+B V iff B V If A V and B V, then A+B V by definition of a linear space If A V and A+B V then k 1 A + k 2(A+B) V by definition of a linear space. Let k 1 = -1, k 2 = 1 k 1 A + k 2(A+B) = B and thus B V

Subspaces • Subset U of a linear space V is termed a subspace of V if it is a linear space. • Trivial Cases: (1) The null set {0} and (2) the entire set V • Column space C(A) of mxn matrix A is a subspace of R m (set of all mdimensional vectors) • Row space R(A) of mxn matrix A is a subspace of R n (set of all ndimensional vectors) • For any 2 subsets, S and T of a given set (say 2 subspaces of R mxn), S is contained in T if all elements of S are elements of T (S T) • If S T and T S, then S = T

Results Involving Subspaces Lemma 4. 2. 1. A ≡ mxn Then for any subspaces U of R m and V of R n : C (A) U iff every column of A is in U, R (A) V iff every row of A is in V where A = [a 1 … an] If C (A) U then all ai U (Let xi = (0, …, 0, 1, 0, …, 0)’) If ai U i=1, …, n then y C (A) y = Ax y = x 1 a 1 + … + xnan and since U is linear space, y U Lemma 4. 2. 2. A ≡ mxn B ≡ mxp C ≡ qxn : C (B) C (A) iff there exists nxp F such that B = AF R (C) R (A) iff there exists qxm L such that C = LA Corollary 4. 2. 3. For any A ≡ mxn F ≡ nxp L ≡ qxm : C (AF) C (A) and R (LA) R (A) Corollary 4. 2. 4. A ≡ mxn E ≡ nxk F ≡ nxp L ≡ qxm T ≡ sxm : (1) If C (E) C (F) then C (AE) C (AF) and if C (E) = C (F) then C (AE) = C (AF) (2) If R (L) R (T) then R (LA) R (TA) and if R (L) = R (T) then R (LA) = R (TA) Lemma 4. 2. 5. A ≡ mxn B ≡ mxp : (1) C (A) C (B) iff R (A’) R (B’) (2) C (A) = C (B) iff R (A’) = R (B’) (1) A = BF A’ = (BF)’ = F’B’

Bases • Span of a finite set of matrices of common dimensions: • Finite nonempty set {A 1, …, Ak}: set of all matrices being linear combinations of {A 1, …, Ak} • Empty set: {0} (unique basis – see below for definition of basis) • Span of a finite set S of matrices written as sp(S) which is a linear space • sp({A 1, …, Ak}) ≡ sp(A 1, …, Ak) ≡ span of the set {A 1, …, Ak} • Finite set S of matrices in linear space V spans V if sp(S) = V • Basis for V is a finite linearly independent set of matrices in V that spans V • C(A) for A ≡ mxn is spanned by the set of its n columns. If lin. indep. basis • R(A) for A ≡ mxn is spanned by the set of its m rows. If lin. indep. basis • If the columns or rows of A are not linearly independent, they are not a basis

Natural Bases for R m n and Linear Space of nxn Symmetric Matrices

Results Involving a Basis - I Lemma 4. 3. 1. A 1, …, Ap and B 1, …, Bq in linear space V If {A 1, …, Ap} spans V, then so does {A 1, …, Ap, B 1, …, Bq} If {A 1, …, Ap, B 1, …, Bq} spans V and if B 1, …, Bq are linear functions of A 1, …, Ap then {A 1, …, Ap} spans V Theorem 4. 3. 2. V ≡ linear space spanned by a set of r matrices. Let S ≡ set of k lin. indep. Matrices in V Then k ≤ r and if k = r, then S is a basis for V Corollary 4. 3. 3. The number of matrices in a linearly independent set of m n matrices cannot exceed mn. Number of matrices in a linearly independent set of n n symmetric matrices cannot exceed n+n(n-1)/2 = n(n+1)/2 Theorem 4. 3. 4. Every linear space of m n matrices has a basis Lemma 4. 3. 5. Matrix A in linear space V can be written as a unique linear combination of matrices in any particular basis {A 1, …, Ak} with unique coefficients x 1, …, xk and A = x 1 A 1+…+xk. Ak

Results Involving a Basis - II Theorem 4. 3. 6. Any two bases of the same linear space V have the same number of matrices. The number of matrices in the basis is its dimension: dim(V) Theorem 4. 3. 7. If linear space V is spanned by set of r matrices, then dim(V) ≤ r. If there is set of k linearly independent matrices in V, dim(V) ≥ k Theorem 4. 3. 8. If U is a subspace of linear space V, dim(U) ≤ dim(V) Theorem 4. 3. 9. Any set of r linearly independent matrices in r-dimensional linear space V is a basis for V Theorem 4. 3. 10. U, V ≡ linear spaces of m n matrices with U V and dim(U) = dim(V), then U = V. Aside: dim(R m n) = mn dim(V) ≤ mn Theorem 4. 3. 11. Any set S that spans a linear space V of m n matrices contains a subset that is basis for V. The number of matrices in the subset is dim(V) Theorem 4. 3. 12. For any set S of r lin. indep. matrices in k-dimensional V, there exists a basis for V containing the r matrices in S and k-r additional linearly independent matrices

Rank of a Matrix • Row Rank – Dimension of the row space of A (number of lin. indep. Rows) • Column Rank – Dimension of the column space of A Theorem 4. 4. 1. For any matrix A, row rank = column rank Theorem 4. 4. 2. Nonnull A ≡ mxn with row rank = r, column rank = c. Then B ≡ m c and L ≡ c n such that A = BL. Similarly, K ≡ m r and T ≡ r n such that A = KT. Theorem 4. 4. 2. Basis for C(A) has c vectors: {b 1, …, bc} B=[b 1…bc]. C(B) = sp(b 1, …, bc) = C(A) L ≡ c n such that A = BL Basis for R(A) has r vectors: {t 1’, …, tr’} T=[t 1…tr]’. R(T) = sp(t 1’, …, tr’) = R(A) K ≡ m r such that A = KT Theorem 4. 4. 1. Assume A ≡ nonnull (A = 0 r = c = 0) A = BL A = KT R(A) R(L) and C(A) C(K) R(L) spanned by the c rows of L C(K) spanned by the r columns of K r ≤ dim(R(L)) ≤ c and c ≤ dim(C(K)) ≤ r r = c

Results on Dimensions/Ranks Lemma 4. 4. 3. For any A ≡ m n: rank(A) ≤ m, rank(A) ≤ n Theorem 4. 4. 4. A ≡ m n, B ≡ m p, C ≡ q n: [Theorem 4. 3. 8. ] If C(B) C(A) then rank(B) ≤ rank(A). If R(C) R(A) then rank(C) ≤ rank(A) Corollary 4. 4. 5. A ≡ m n, F ≡ n p: rank(AF) ≤ rank(A) rank(AF) ≤ rank(F) [Corollary 4. 2. 3. ] Theorem 4. 4. 6. A ≡ m n, B ≡ m p, C ≡ q n: [Theorem 4. 3. 10. ] If C(B) C(A) and rank(B) = rank(A) then C(B) = C(A). If R(C) R(A) and rank(C) = rank(A) then R(C) = R(A) Corollary 4. 4. 7. A ≡ m n, F ≡ n p: If rank(AF) = rank(A) then C(AF) = C(A). If rank(AF) = rank(F) then R(AF) = R(F)

Nonsingular, Full Row Rank, and Full Column Rank Matrices • Among m n matrices, maximum rank = min(m, n) • Mininimum rank is 0, only for the null matrix 0 • A ≡ m n is said to be full row rank if rank(A) = m • A ≡ m n is said to be full column rank if rank(A) = n • A ≡ n n is said to be nonsingular if rank(A) = n (full row and column rank) Theorem 4. 4. 8. A ≡ m n nonnull matrix of rank r. Then B ≡ m r and T ≡ r n such that A = BT For any B ≡ m r and T ≡ r n s. t. A = BT, rank(B) = rank(T) = r B ≡ full column rank and T ≡ full row rank By Theorem 4. 4. 2. : B ≡ m r and T ≡ r n such that A = BT By their dimensions: rank(B) ≤ r and rank(T) ≤ r (Lemma 4. 4. 3). By definition: rank(A) = rank(BT) = r By Corollary 4. 4. 5. : r = rank(BT) ≤ rank(B) ≤ r, r = rank(BT) ≤ rank(T) ≤ r rank(B) = rank(T) = r

Decomposition of Rectangular Matrix

Definitions and Results Involving Ranks Theorem 4. 4. 10. A ≡ m n with rank(A) = r A has r linearly independent rows and columns The r r submatrix Ar made up of the r lin. Indep. rows and the r lin. indep. columns is nonsingular. Any of the remaining rows or columns are linearly dependent of those of Ar. There is no submatrix of A with rank > r Corollary 4. 4. 11. Any n n symmetric matrix of rank r has an r r principal submatrix that is nonsingular Useful Equalities 1) rank(A’) = rank(A) (r = max # of linearly independent rows and columns of matrix) 2) For any nonzero scalar k, rank(k. A) = rank(A) (No linear dependencies changed) 3) rank(-A) =rank(A) (special case of 2))

Partitioned Matrices and Sums of Matrices - I

Partitioned Matrices and Sums of Matrices - II

Partitioned Matrices and Sums of Matrices - III

Partitioned Matrices and Sums of Matrices - IV