CSE 203 B Convex Optimization Chapter 5 Duality

CSE 203 B Convex Optimization: Chapter 5 Duality CK Cheng Dept. of Computer Science and Engineering University of California, San Diego 1

Chapter 5 Duality • Primal and Dual Problem – Primal Problem – Lagrangian Function – Lagrange Dual Problem • Examples (Primal Dual Conversion Procedure) – – Linear Programming Quadratic Programming Conjugate Functions (Duality) Entropy Maximization • Interpretation (Duality) – – Saddle-Point Interpretation Geometric Interpretation Slater’s Condition Shadow-Price Interpretation • KKT Conditions (Optimality Conditions) • Sensitivity (Shadow-Price) • Generalized Inequalities 2

Duality 3

Duality 4

Example: Linear Programming 5

Example: Linear Programming • 6

Example: Linear Programming or 7

Example: Quadratic Programming 8

Example: Quadratic Program 9

Example: Quadratic Program (nonconvex prob. ) 10

Example: Discrete Problem 11

Example: Discrete Problem 12

Examples: Conjugate Function Conjugate function 13

Examples: Entropy Maximization 14

Examples: Minimum Volume Covering Ellipsoid 15

Interpretation: Saddle-point 16

Interpretation: Saddle-point 17

Interpretation: Saddle-point 18

Interpretation: Saddle-point 19

Interpretation: Saddle-point 20

Geometric Interpretation 21

Duality via Separating Hyperplane Separating hyperplane: Example 22

Lagrange dual problem relint: relative interior of set D any norm 23

Shadow Price Interpretation: Food vs. Vitamin Primal Flour protein powder Veg. vitamins A, B, D, E, K Fruits minerals Dual Lagrangian 24

Shadow Price Interpretation: Spring Energy & Force 25

KKT (Karush-Kuhn-Tucker) Conditions 26

KKT (Karush-Kuhn-Tucker) Conditions 1. Primal, Lagrangian, and Dual Necessary condition for local optimality Sufficient when the problem is convex & satisfy regularity conditions (Slater condition) 27

Sensitivity Perturbed Problem Unperturbed Problem 28

Generalized Inequalities Primal Lagrangian Lagrange Dual 29

Generalized Inequality: KKT Conditions 30

Generalized Inequalities: SOCP Primal Lagrangian Lagrange Dual 31

Generalized Inequalities: Semidefinite Program Primal Lagrangian Lagrange Dual 32

Chapter 5 Summary • Primal and Dual Problem – Primal Problem – Lagrangian Function – Lagrange Dual Problem • Examples (Primal Dual Conversion Procedure) – – Linear Programming Quadratic Programming Conjugate Functions (Duality) Entropy Maximization • Interpretation (Duality) – – Saddle-Point Interpretation Geometric Interpretation Slater’s Condition Shadow-Price Interpretation • KKT Conditions (Optimality Conditions) • Sensitivity (Shadow-Price) • Generalized Inequalities 33

Chapter 5 Summary • Duality provides a lower bound of the problem even the primal may not be convex. • KKT conditions convert the minimization problem into equations. • Lagrange multipliers provide the sensitivity of the constraints. • Generalized inequality broadens the scope of convex optimization. 34