Outline n Low Rank Matrix 1 Why Low

  • Slides: 14
Download presentation
Outline n Low Rank Matrix 1

Outline n Low Rank Matrix 1

Why Low Rank? n Many Objects are Sparse in Some Domain of Linear Transform

Why Low Rank? n Many Objects are Sparse in Some Domain of Linear Transform n Many image coefficients negligible in the DCT domain n Narrow interference sparse in the frequency domain n Sparse in One Domain Low Rank in another Linear Transform Domain 2

Low Rank Related Optimization n Low Rank Approximation Problem to a Matrix under L

Low Rank Related Optimization n Low Rank Approximation Problem to a Matrix under L 2 -Norm Criterion min. X ||X - Z||2 s. t. rank(X) ≤ r n Solution: X = UΣVH U = [u 1 u 2 … ur], V = [v 1 v 2 … vr] Σ = diag(σ1, σ2, …, σr) the largest r singular values of Z 3

Low Rank Related Optimization n Correlation Matrix Approximation Problem to a Matrix under L

Low Rank Related Optimization n Correlation Matrix Approximation Problem to a Matrix under L 2 -Norm Criterion min. X ||X - G||2 s. t. Xii = 1, 2, …, N; X ≥ 0; rank(X) ≤ r. n Solution: left for homework discussion 4

Low Rank Related Optimization n Matrix Completion Problem min. X rank(X) s. t. Xij

Low Rank Related Optimization n Matrix Completion Problem min. X rank(X) s. t. Xij = Mij, (i, j) in Ω is a subset of {1, 2, …, p}*{1, 2, …, q} n Approximation form min. X rank(X) s. t. |Xij - Mij | < ε, (i, j) in Ω 5

Rank Minimization Problem formulation min. X rank(X), s. t. X belongs to C for

Rank Minimization Problem formulation min. X rank(X), s. t. X belongs to C for some convex set C n Rank of matrix [X]m*n, Rank(X) = r n Number of independent rows/columns of X n Number of nonzero singular values n Smallest number r, such that X = FG, [F]m*r, [G]r*n 6

Rank Minimization Problem Solution n Rank Minimization Problem n The L 0 -norm of

Rank Minimization Problem Solution n Rank Minimization Problem n The L 0 -norm of eigenvalues n Non-convex optimization problem n Heuristic Solution to Rank Minimization Problem n Nuclear norm: L 0 -norm approximation of eigenvalues n Log-det minimization approximation n Matrix factorization, alternating algorithm 7

Nuclear Norm Heuristic n Rank Minimization Nuclear Norm Minimization min. X |X|*, s. t.

Nuclear Norm Heuristic n Rank Minimization Nuclear Norm Minimization min. X |X|*, s. t. X in C n The Number of Non-zero Singular Values σi The Summation of All Singular Values σi n Rank(X) = ||singular(X)||0 n Nuclear Norm = ||singular(X)||1 8

Log-det Heuristic n Rank minimization Log-det minimization min. X log det(X + δI), s.

Log-det Heuristic n Rank minimization Log-det minimization min. X log det(X + δI), s. t. X in C n The summation of |σi| for all singular values σi n Rank(X) = ||singular(X)||0 n Geometric average of ||singular(X) + δI|| 9

Matrix Factorization n Write X = FG, [F]m*r, [G]r*n n Find a rank-r matrix

Matrix Factorization n Write X = FG, [F]m*r, [G]r*n n Find a rank-r matrix from the set C n Randomly Initialize Matrices n Alternating the following two sub-problems n For X in C and [G]r*n, solve min ||X-F(k-1)G||2 n For X in C and [F]m*r, solve min ||X-FG(k)||2 n Record, e(k) = ||X(k) – F(k)G(k)|| 10

Background Extraction Problem n Need to Separate the Background from the Foreground from a

Background Extraction Problem n Need to Separate the Background from the Foreground from a Video Stream n Background component: static within a short period n Foreground component, a small fraction of nonzero n Framework: n Treat each frame as a vector n Put all vectors together as a matrix n Matrix contains two components, background component + foreground component 11

Background Extraction Problem n Background component (X): static within a short period, n low

Background Extraction Problem n Background component (X): static within a short period, n low rank of matrix X n Foreground component (E), a small fraction of space n sparse component E n Entire Signal Y Written as follows Y=X+E Y is low rank, E is sparse 12

Background Extraction Problem formulation min. X rank(X) + γ||vec(E)||0 s. t. Y = X

Background Extraction Problem formulation min. X rank(X) + γ||vec(E)||0 s. t. Y = X + E n Nuclear norm heuristic min. X ||X||* + γ||vec(E)||1 s. t. Y = X + E n Conclusion: for low rank X and sparse enough E, exact recovery happens with high probability 13

More Discussions n How to Solve the Following Problems n Low rank approximation problem

More Discussions n How to Solve the Following Problems n Low rank approximation problem n Correlation matrix approximation problem n Matrix completion problem n Rank minimization problem n Homework of this Week n A comprehensive overview of the solution to the above problems 14