Lecture 9 Nonlinear Programming Models Topics Convex sets

Lecture 9 – Nonlinear Programming Models Topics • Convex sets and convex programming • First-order optimality conditions • Examples • Problem classes

General NLP Minimize f(x) s. t. gi(x) ( , , =) bi, i = 1, …, m x = (x 1, …, xn)T is the n-dimensional vector of decision variables f (x) is the objective function gi(x) are the constraint functions bi are fixed known constants

Convex Sets Definition: A set S n is convex if every point on the line segment connecting any two points x 1, x 2 S is also in S. Mathematically, this is equivalent to x 0 = lx 1 + (1–l)x 2 S for all l such 0 ≤ l ≤ 1. x 1 x 2

(Nonconvex) Feasible Region S = {(x 1, x 2) : (0. 5 x 1 – 0. 6)x 2 ≤ 1 2(x 1)2 + 3(x 2)2 ≥ 27; x 1, x 2 ≥ 0}

Convex Sets and Optimization Let S = { x n : gi(x) bi, i = 1, …, m } Fact: If gi(x) is a convex function for each i = 1, …, m then S is a convex set. Convex Programming Theorem: Let x n and let f (x) be a convex function defined over a convex constraint set S. If a finite solution exists to the problem Minimize { f (x) : x S } then all local optima are global optima. If f (x) is strictly convex, the optimum is unique.

Convex Programming Min f (x 1, …, xn) s. t. gi(x 1, …, xn) bi i = 1, …, m x 1 0, …, xn 0 is a convex program if f is convex and each gi is convex. Max f (x 1, …, xn) s. t. gi(x 1, …, xn) bi i = 1, …, m x 1 0, …, xn 0 is a convex program if f is concave and each gi is convex.

Linearly Constrained Convex Function with Unique Global Maximum Maximize f (x) = (x 1 – 2)2 + (x 2 – 2)2 subject to – 3 x 1 – 2 x 2 ≤ – 6 –x 1 + x 2 ≤ 3 x 1 + x 2 ≤ 7 2 x 1 – 3 x 2 ≤ 4

(Nonconvex) Optimization Problem

First-Order Optimality Conditions Minimize { f (x) : gi(x) bi, i = 1, …, m } Lagrangian: Optimality conditions • Stationarity: • Complementarity: migi(x) = 0, i = 1, …, m • Feasibility: gi(x) bi, i = 1, …, m • Nonnegativity: mi 0, i = 1, …, m

Importance of Convex Programs Commercial optimization software cannot guarantee that a solution is globally optimal to a nonconvex program. NLP algorithms try to find a point where the gradient of the Lagrangian function is zero – a stationary point – and complementary slackness holds. Given L(x, m) = f(x) + m(g(x) – b) we want L(x, m) = f(x) + m g(x) = 0 m(g(x) – b) = 0 g(x) – b ≤ 0, m 0 For a convex program, all local solutions are global optima.

Example: Cylinder Design We want to build a cylinder (with a top and a bottom) of maximum volume such that its surface area is no more than s units. Max V(r, h) = r 2 h s. t. 2 r 2 + 2 rh = s r h r 0, h 0 There a number of ways to approach this problem. One way is to solve the surface area constraint for h and substitute the result into the objective function.

Solution by Substitution s 2 r 2 h= 2 r d. V = 0 dr Volume = V = r 2 s 1/2 r = ( ) h= 6 3/2 s V = r 2 h = 2 ( ) 6 s 2 rs r 3 [ ] = 2 2 r s s 1/2 r = 2( ) 2 r 6 s 1/2 r = ( ) 6 s 1/2 ) h = 2( 6 Is this a global optimal solution?

Test for Convexity d. V(r) s rs 3 r dr = 2 3 r 2 V(r ) = 2 d 2 V(r ) dr 2 = 6 r d 2 V 0 for all r 0 dr 2 Thus V(r ) is concave on r 0 so the solution is a global maximum.

Advertising (with Diminishing Returns) • A company wants to advertise in two regions. • The marketing department says that if $x 1 is spent in region 1, sales volume will be 6(x 1)1/2. • If $x 2 is spent in region 2, sales volume will be 4(x 2)1/2. • The advertising budget is $100. Model: Max f (x) = 6(x 1)1/2 + 4(x 2)1/2 s. t. x 1 + x 2 100, x 1 0, x 2 0 Solution: x 1* = 69. 2, x 2* = 30. 8, f (x*) = 72. 1 Is this a global optimum?

Excel Add-in Solution

Portfolio Selection with Risky Assets (Markowitz) • Suppose that we may invest in (up to) n stocks. • Investors worry about (1) expected gain (2) risk. Let rj = random variable associated with return on stock j mj = expected return on stock j sjj = variance of return for stock j We are also concerned with the covariance terms: sij = cov(ri, rj) If sij > 0 then returns on i and j are positively correlated. If sij < 0 returns are negatively correlated.

Decision Variables: xj = # of shares of stock j purchased n R(x) = mjxj Expected return of the portfolio: j =1 n Variance (measure of risk): n V(x) = sijxixj Example: i =1 j =1 If x 1 = x 2 = 1, we get V(x) = s 11 x 1 x 1 + s 12 x 1 x 2 + s 21 x 2 x 1 + s 22 x 2 x 1 = 1 + ( 1) + 1 = 0 Thus we can construct a “risk-free” portfolio (from variance point of view) if we can find stocks “fully” negatively correlated.

If , then buying stock 2 is just like buying additional shares of stock 1. Nonlinear optimization models … Let pj = price of stock j b = our total budget b = risk-aversion factor (when b = 0 risk is not a factor) Consider 3 different models: 1) Max f (x) = R(x) – b. V(x) n s. t. pj xj b, xj 0, j = 1, …, n j =1 where b 0 is determined by the decision maker

2) Max f (x) = R(x) s. t. n V(x) a, pjxj b, xj 0, j = 1, …, n j =1 where a 0 is determined by the investor. Smaller 2) values of a represent greater risk aversion. 3) Min s. t. f (x) = V(x) n R(x) g, pj xj b, xj 0, j = 1, …, n j =1 where g 0 is the desired rate of return (minimum expectation) is selected by the investor.

Hanging Chain with Rigid Links 10 ft 1 ft x y each link What is equilibrium shape of chain? Decision variables: Let (xj, yj), j = 1, …, n, be the incremental horizontal and vertical displacement of each link, where n 10. Constraints: xj 2 + yj 2 = 1, j = 1, …, n, each link has length 1 x 1 + x 2 + • • • + xn = 10, net horizontal displacement y 1 + y 2 + • • • + yn = 0, net vertical displacement

Objective: Minimize chain’s potential energy Assuming that the center of the mass of each link is at the center of the link. This is equivalent to minimizing 1 y + (y + 1 1 2 1 1 y 2) + (y 1 + y 2 + y 3) + 2 2 • • • + (y 1 + y 2 + • • • + yn-1 + 1 1 = (n 1 + 2 ]y 1 + (n 2 + 2 )y 2 1 + (n 3 + 2 )y 3 + • • • + 3 1 2 yn-1 + 2 yn Summary n Min (n j + ½)yj j =1 s. t. 1 2 yn) xj 2 + yj 2 = 1, j = 1, …, n x 1 + x 2 + • • • + xn = 10 y 1 + y 2 + • • • + yn = 0

Is a local optimum guaranteed to be a global optimum? No! Constraints xj 2 + yj 2 = 1 for all j yield a nonconvex feasible region so there may be several local optima. Consider a chain with 4 links: These solutions are both local minima.

Direct Current Network Problem: Determine the current flows I 1, I 2, …, I 7 so that the total content is minimized Content: G(I) = I 0 0 v(i)di for I ≥ 0 and G(I) = I v(i)di for I < 0

Solution Approach Electrical Engineering: Use Kirchoff’s laws to find currents when power source is given. Operations Research: Optimize performance measure in network taking flow balance into account. Linear resistor: Voltage, v(I ) = IR 2 Content function, G(I ) = I R/2 Battery: Voltage, v(I ) = –E Content function, G(I ) = –EI

Network Flow Model Network diagram: Minimize Z = – 100 I 1 + 5 I 22 + 5 I 32 + 10 I 42 + 10 I 52 subject to I 1 – I 2 = 0, I 2 – I 3 – I 4 = 0, I 5 – I 6 = 0, I 5 + I 7 = 0, I 3 + I 6 – I 7 = 0, –I 1 – I 6 = 0 Solution: I 1 = I 2 = 50/9, I 3 = 40/9, I 4 = I 5 = 10/9, I 6 = – 50/9, I 7 = – 10/9

NLP Problem Classes • Constrained vs. unconstrained • Convex programming problem • Quadratic programming problem f (x) = a + c. Tx + ½ x. TQx, Q 0 • Separable programming problem f (x) = j=1, n fj(xj) • Geometric programming problem g(x) = t=1, T ct. Pt(x), Pt(x) = (x 1 at 1). . . (xnatn), xj > 0 • Equality constrained problems

What You Should Know About Nonlinear Programming • How to identify a convex program. • How to write out the first-order optimality conditions. • The difference between a local and global solution. • How to classify problems.