Nonnegative Matrix Factorization via Rankone Downdate Nonnegative Matrix

2 by 1965 560 by 1965 20 by 28 560 by 2 -2. 19

Power method • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 1

Example • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 2 while

Naive approach to NMF using this observation 1 2 3 4 5 6 for

Modified power iteration 1 2 while not converged 3 4 5 6 7 8

Modified power iteration: Demo A= Rank-1 submatrix

Modified power iteration: Demo v: 0. 14 0. 07 0. 64 0. 41 0.

Modified power iteration: Demo v: 0. 0 0. 64 0. 41 0. 55 Rank-1

Modified power iteration: Demo v: u: 0. 16 0. 21 0. 22 0. 44

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20

Modified power iteration: Demo Anew = Rank-1 submatrix

Rank-one Downdata (R 1 D) 1 2 3 4 5 end for

Experimental results • Information retrieval task – TDT corpus (pilot study, v. 1. 3,

Experimental results • For topic: “OJ Simpson trial” (simpson, defense, judge, case, jury, trial)

Slides: 50

Download presentation

Nonnegative Matrix Factorization via Rank-one Downdate

Nonnegative Matrix Factorization

2 by 1965 560 by 1965 20 by 28 560 by 2 -2. 19 -3. 19 -0. 02 1. 02 2 by 1

Singular Value Decomposition (SVD)

History

History (Algorithms)

First observation

Power method • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 1 2 while not converged 3 4 5 6 end

Example • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 2 while not converged 3 4 5 6 end

Naive approach to NMF using this observation 1 2 3 4 5 6 for all end for set Without step 5, this will simply compute the SVD (Jordan's algorithm, Camille Jordan 1874. )

Second observation

Modified power iteration 1 2 while not converged 3 4 5 6 7 8 end

Modified power iteration: Demo A= Rank-1 submatrix

Modified power iteration: Demo v: 0. 14 0. 07 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: 0. 0 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 16 0. 21 0. 22 0. 44 0. 74 0. 20 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 60 0. 28 0. 59 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 60 0. 28 0. 59 Rank-1 submatrix Rank-1 Zero-out! submatrix

Modified power iteration: Demo Anew = Rank-1 submatrix

Rank-one Downdata (R 1 D) 1 2 3 4 5 end for

Objective function

Approx. Rank. One. Submatrix(A)

Rank-one Downdata (R 1 D)

A simple model for text

Generating a corpus in the model

Theorem about text

LSI

R 1 D

Experimental results • Information retrieval task – TDT corpus (pilot study, v. 1. 3, 1997): news articles Identified topics (first few columns of ): Topic: president, clinton, house, white Topic: bosnian, serbs, bosnia, serb, nato, sarajevo, air, bihac Topic: haiti, military, aristide, haitian, troops, port, invasion, … Topic: simpson, defense, judge, case, jury, trial, angeles, los, court, … Topic: bill, today, senate, republicans, house, congress, republican, …

Experimental results • For topic: “OJ Simpson trial” (simpson, defense, judge, case, jury, trial) Document 1 Document 2 Document 3 Document 4 Allen Simpson Judge Gloves Kaelin Judge Defense Case Police Defence Statements Prosecution Defense Marcus Opening Defense Chicago Court Jury Roger Testimony Witness Legal Put Night Testify Yesterday Fit Knapsack Nicole Prosecution Today Cnn gloves Marc Problems Blood los jury Witness

Theorem about images

Experimental results

LSI

NMF-DIV

R 1 D

LSI

NMF_DIV

NMF_SC

R 1 D