Nonnegative Matrix Factorization Recent algorithms extensions and available
Non-negative Matrix Factorization Recent algorithms, extensions and available software Atina Dunlap Brooks (adbrook 2@stat. ncsu. edu) North Carolina State University 1
Recent Algorithms o o Lee & Seung’s multiplicative updates are easy to understand to implement Can be very slow to converge n o ALS can speed things up Convergence theory is not particularly strong n Most NMF methods do not have robust convergence, but work well in practice 2
Projected Gradient Descent Method o Chih-Jen Lin (2007) Bound-constrained optimization o Projected Gradient o 3
Projected Gradient Descent Method o o Can be applied to both the multiplicative updates and the ALS solution Generally, greatest speed was achieved with the projected gradient combined with ALS 4
Fast Non-Negative Matrix Approximation o o Kim, Sra & Dhillon (2007) Employs Newton-type methods to solve NMF Uses curvature information vs. gradient descent approach Provide an exact method (good accuracy, but still slow) and a very fast inexact method 5
References for Algorithm Comparisons o o Algorithms and Applications for Approximate Nonnegative Matrix Factorization by Berry, Browne, Langville, Pauca & Plemmons (2006) Optimality, Computation, and Interpretations of Nonnegative Matrix Factorizations by Chu, Diele, Plemmons & Ragni (2004) 6
Extensions o o o Tri-Factorization Semi-NMF Convex-NMF Non-negative Tensor Factorization Inferential Robust Matrix Factorization 7
Orthogonal Tri-factorization o o Ding, Li, Peng & Park (2006) Requiring orthogonality introduces uniqueness and improves clustering interpretations A = WSH, o where WTW=I and HTH=I W gives row clusters while H gives column clusters 8
Semi-NMF o o o Ding, Li & Jordan (2006) Allows A and W to contain negative values, but H is restricted to nonnegative Provides more flexibility (negative entries) and a clustering which is usually better than k-means 9
Convex-NMF o o Ding, Li & Jordan (2006) Restricts W to be convex combinations of the columns of A Ensures meaningful cluster centroids W and H tend to be sparse 10
Non-Negative Tensor Factorization o o Uses n-way arrays instead of the 2 dimensional arrays used by NMF Presentations during the workshop by Michael Berry and Bob Plemmons 11
Inferential Robust Matrix Factorization o o o Fogel, Young, Hawkins & Ledirac (2007) Uses the same method for robustness as Liu et al. (2003) for robust SVD Paul Fogel will be presenting on an application 12
Software - Matlab o Matlab Code n Patrik Hoyer o o n http: //www. cs. helsinki. fi/u/phoyer/ Includes Lee & Seung’s multiplicative updates and Hoyer’s sparseness Chih-Jen Lin o o http: //www. csie. ntu. edu. tw/~cjlin/nmf/ Includes projected gradient descent applied to multiplicative updates and ALS 13
Software o C code – nnmf() n Simon Sheperd o o o http: //www. simonshepherd. supanet. com/ nnmf. htm Very fast algorithm (as of 2004) JMP script - ir. MF n Paul Fogel o o http: //www. niss. org/ir. MF/ Inferential Robust Matrix Factorization 14
Thank You 15
- Slides: 15