CS 5321 Numerical Optimization 19 9152020 InteriorPoint Methods

  • Slides: 11
Download presentation
CS 5321 Numerical Optimization 19 9/15/2020 Interior-Point Methods for Nonlinear Programming (Barrier Methods) 1

CS 5321 Numerical Optimization 19 9/15/2020 Interior-Point Methods for Nonlinear Programming (Barrier Methods) 1

Problem formulation min f (x) x s. t. c. E (x) = 0 c.

Problem formulation min f (x) x s. t. c. E (x) = 0 c. I (x) ¡ s = 0 s¸ 0 l Ø Ø Ø c. E is a vector of ci, i E c. I is a vector of ci, i I s is the slack variables Add barrier functions for inequality constraints. Xm min f (x) ¡ ¹ x ln si i= 1 s. t. c. E (x) = 0 c. I (x) ¡ s = 0 9/15/2020 Ø is the barrier parameter, which may be reduced iteratively. 2

KKT conditions l The Lagrangian of the Xm original problem is L (x; s;

KKT conditions l The Lagrangian of the Xm original problem is L (x; s; y; z) = f (x) ¡ ¹ ln si ¡ y. T c. E (x) ¡ z. T (c. I (x) ¡ s) i= 1 l l l y and z are the Lagrangian multipliers. Let AE and AI denote the Jacobian of c. E and c. I. The KKT condition of the barrier function is T T = 9/15/2020 r f (x) ¡ A E (x)y ¡ A I (x)z 0 Sz ¡ ¹ e = 0 c. E (x) = 0 c. I (x) ¡ s = 0 3

Line-search algorithm l 2 6 6 4 r Solve KKT conditions by the Newton’s

Line-search algorithm l 2 6 6 4 r Solve KKT conditions by the Newton’s method l 2 L xx 0 AE AI l 32 The primal-dual system 0 Z 0 ¡ I ¡ A TE 0 0 0 3 2 3 px r f ¡ A ET y ¡ A TI z ¡ A TI 6 7 S 7 Sz ¡ ¹ e 7 6 ps 7 = ¡ 6 7 4 5 0 5 4 py 5 c. E pz 0 c. I ¡ s Update (x, s, y, z)+=( xpx, sps, ypy, zpz) and k+1 l The fraction to the boundary rule: for (0, 1) = ®s maxf ® 2 (0; 1] : s + ®ps ¸ (1 ¡ ¿)sg ®z = maxf ® 2 (0; 1] : z + ®pz ¸ (1 ¡ ¿)zg 9/15/2020 4

Solving the primal-dual system l Let =S− 1 Z and rewrite the primal dual

Solving the primal-dual system l Let =S− 1 Z and rewrite the primal dual system as l l The matrix become symmetric. Eliminate ps and reform the problem 9/15/2020 5

Avoid singularity Singularity may caused by (some elements goes to ) or ill-conditionness of

Avoid singularity Singularity may caused by (some elements goes to ) or ill-conditionness of xx 2 f and AI. l 2 1. 3 and modified Use projected Hessian T T 2. Add diagonal shift G 6 0 6 4 A E AI 9/15/2020 0 T 0 ¡ I AE 0 0 0 AI ¡ I 7 7 0 5 0 6

Barrier param and step length Barrier parameters { k} need converge to zero. Static

Barrier param and step length Barrier parameters { k} need converge to zero. Static method k+1= k k for k (0, 1) Adaptive methods s. Tk zk l 1. 2. a. l In the linear programming, (chap 14) ¹ k+ 1 = ¾ m Use merit function to decide whether a step is Xm productive and should be accepted. (chap 18) = Á(x; s) f (x) ¡ ¹ ln si + º kc. E k + º kc. I ¡ sk i= 1 9/15/2020 7

Trust-region SQP method Two differences from the line search approach l 1. 2. l

Trust-region SQP method Two differences from the line search approach l 1. 2. l 9/15/2020 Solve primal (x, s) and dual problem (y, z) alternately Use scaling S− 1 on ps. The primal problem 8

Solving the dual problem · ¸ l l Define solve A^ = A^T ·

Solving the dual problem · ¸ l l Define solve A^ = A^T · AE AI y z 0 ¡ S. The dual problem is to ¸ · ¸ = r f (x) ¡ ¹e · least-square ¸ · Using the method y z l 9/15/2020 = ( A^A^T ) ¡ 1 A^ r f (x) ¡ ¹e ¸ The solution cannot guarantee z to be positive. replaced by a small positive number if zi 0 zi ! min(10¡ 3 ; ¹ =si ) 9

Scaling l Let ps' = S− 1 ps. The problem can be rewritten as

Scaling l Let ps' = S− 1 ps. The problem can be rewritten as l Parameter r and r. I can be computed by solving r = A (x)v. E + c (x); r = A (x)v ¡ Sv + c (x) ¡ s E E x E I I x s I min kr E k 22 + kr I k 22 v s. t. k(vx ; vs )k · 0: 8¢ ; vs ¸ V (¿=2)e 9/15/2020 10

Convergence theory l Theorem 19. 1: Let {xk} be the iterative points of the

Convergence theory l Theorem 19. 1: Let {xk} be the iterative points of the basic algorithm and { k} 0. Suppose f and c are continuously differentiable. The all limit points x* of {xk} are feasible. Moreover, if any x* satisfies LICQ, the KKT conditions are satisfied. 9/15/2020 11