Sparse Inverse Covariance Estimation with Graphical LASSO J

Sparse Inverse Covariance Estimation with Graphical LASSO J. Friedman, T. Hastie, R. Tibshirani Biostatistics, 2008 Presented by Minhua Chen 1

• • • Outline Motivation Mathematical Model Mathematical Tools Graphical LASSO Related papers 2

Motivation (M. Choi, V. Chandrasekaran and A. S. Willsky, 2009) (O. Banerjee, L. Ghaoui, 3 and A. d’Aspremont, 2008)

Mathematical Model • The optimization problem is concave (M. Yuan and Y. Lin, 2007). • Various optimization algorithms have been proposed (M. Yuan and Y. Lin, 2007; O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008; N. Meinshausen and P. Buhlmann, 2006). • The Graphical LASSO algorithm, built on a previous paper (O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008) , is widely used due to its computational efficiency. • It transforms the above optimization to LASSO regressions. 4

Mathematical Tools (1) • Subgradient (J. Tropp, 2006) Example 1: Example 2: 5

Mathematical Tools (2) • Matrix inversion identity: • The above equations reveal the relationship between the inverse covariance matrix and the covariance matrix. 6

Graphical LASSO (1) 7

Graphical LASSO (2) 8

Graphical LASSO (3) 9

Graphical LASSO (4) Ground Truth Inferred 10

Related papers: • N. Stadler and P. Buhlmann, Missing Values: Sparse Inverse Covariance Estimation and an Extension to Sparse Regression Proposed a Miss. GLasso algorithm to impute the missing data and infer the inverse covariance matrix simultaneously. • O. Banerjee, L. El Ghaoui and A. d’Aspremont, Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data Used a constrained quadratic programming algorithm (COVSEL) to solve the same optimization problem as Graphical LASSO. • N. Meinshausen and P. Buhlmann, High-Dimensional Graphs and Variable Selection with the Lasso Proposed a neighborhood selection method to approximate the Gaussian Graph. 11