Proximal Algorithms A MODERN CONVEX OPTIMIZATION ALGORITHMS Overview

Proximal Algorithms A MODERN CONVEX OPTIMIZATION ALGORITHMS

Overview l. Introduction l. Interpretations of proximal operator l. Proximal algorithms l. Summary l. Q & A

Optimize smooth function

Subgradient

Subgradient descent

Proximal operator Evaluating a proximal operator at various points

Interpretations of proximal operator

Fixed point

Fixed point Ⅱ

Moreau envelope

Moreau envelope Ⅱ

Tikhonovregularized Newton update

Gradient flow

Gradient flow Ⅱ Ø Forward Euler discretization Ø Backward Euler discretization

Proximal gradient method

Problem min

Proximal gradient method

Interpretation: fixed point

Interpretation: gradient flow

ADMM

Separable sum

Parallel and Distributed

Experiment - LASSO Comparing algorithms for solving the lasso

Experiment – Matrix decomposition Comparing CVX and ADMM for solving a matrix decomposition problem

Summary & Resource Boyd et. al: Much like Newton’s method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, largescale, or distributed versions of these problems. Resource: http: //stanford. edu/~boyd/admm. html

Q&A