Chapter 7 Nonlinear Optimization Models Introduction In many

Chapter 7 Nonlinear Optimization Models

Introduction • In many complex optimization problems, the objective and/or the constraints are nonlinear functions of the decision variables. Such optimization problems are called nonlinear programming (NLP) problems. In this chapter, we discuss a variety of interesting problems with inherent nonlinearities, from product pricing to portfolio optimization to rating sports teams. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Introduction continued • A model can become nonlinear for several reasons, including the following: – There are nonconstant returns to scale, which means that the effect of some input on some output is nonlinear. – In pricing models, where the goal is to maximize revenue (or profit), revenue is price multiplied by quantity sold, and price is typically the decision variable. Because quantity sold is related to price through a demand function, revenue is really price multiplied by a function of price, and this product is a nonlinear function of price. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Introduction continued – Analysts often try to find the model that best fits observed data. To measure the goodness of the fit, they typically sum the squared differences between the observed values and the model’s predicted values. Then they attempt to minimize this sum of squared differences. The squaring introduces nonlinearity. – In one of the most used financial models, the portfolio optimization model, financial analysts try to invest in various securities to achieve high return and low risk. The risk is typically measured as the variance (or standard deviation) of the portfolio, and it is inherently a nonlinear function of the decision variables (the investment amounts). Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Introduction continued • As these examples illustrate, nonlinear models are common in the real world. In fact, it is probably more accurate to state that truly linear models are hard to find. • The real world often behaves in a nonlinear manner, so when you model a problem with LP, you are typically approximating reality. • By allowing nonlinearities in your models, you can often create more realistic models. Unfortunately, this comes at a price - nonlinear optimization models are more difficult to solve. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Basic ideas of nonlinear optimization • When you solve an LP problem with Solver, you are guaranteed that the Solver solution is optimal. • When you solve an NLP problem, however, Solver sometimes obtains a suboptimal solution. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Basic ideas of nonlinear optimization continued • For the figure graphed below, points A and C are called local maxima because the function is larger at A and C than at nearby points. • However, only point A maximizes the function; it is called the global maximum. • The problem is that Solver can get stuck near point C, concluding that C maximizes the function. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Convex and concave functions • Solver is guaranteed to solve certain types of NLPs correctly. • To describe these NLPs, we need to define convex and concave functions. – A function of one variable is convex in a region if its slope (rate of change) in that region is always nondecreasing. Equivalently, a function of one variable is convex if a line drawn connecting two points on the curve never lies below the curve. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Convex and concave function continued • In contrast, the function is concave if its slope is always nonincreasing, or equivalently, if a line connecting two points on the curve never lies above the curve. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Convex and concave functions continued • It can be shown that the sum of convex functions is convex and the sum of concave functions is concave. • If you multiply any convex function by a positive constant, the result is still convex, and if you multiply any concave function by a positive constant, the result is still concave. • However, if you multiply a convex function by a negative constant, the result is concave, and if you multiply a concave function by a negative constant, the result is convex. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Problems that solvers always solve correctly • In some situations, if certain conditions hold, Solver is guaranteed to find the global optimum. • Conditions for maximization problems: both conditions below have to be true. – The objective function is concave or the logarithm of the objective function is concave, and – The constraints are linear. • Conditions for minimization problems: – The objective function is convex, and – The constraints are linear. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

When assumptions do not hold • There are many problems for which the conditions outlined previously do not hold or cannot be verified. • Because there is then some doubt whether Solver’s solution is the optimal solution, the best strategy is to 1. Try several possible starting values for the changing cells, 2. Run Solver from each of these, and 3. Take the best solution Solver finds. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

When assumptions do not hold continued • In general, if you try several starting combinations for the changing cells and Solver obtains the same optimal solution in all cases, you can be fairly confident - but still not absolutely sure - that you have found the optimal solution to the NLP. • On the other hand, if you try different starting values for the changing cells and obtain several different solutions, then all you can do is keep the best solution you have found and hope that it is indeed optimal. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Multistart option • There is a welcome new feature in Solver for Excel 2010, the Multistart option. • Because it is difficult to know where to start, the Multistart option provides an automatic way of starting from a number of starting solutions. • It selects several starting solutions automatically, runs the GRG nonlinear algorithm from each, and reports the best solution it finds. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Multistart option continued • To use the Multistart option, select the GRG Nonlinear method in the Solver dialog box, click on Options and then on the GRG Nonlinear tab. You can then check the Use Multistart box, as shown here. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Pricing models • Setting prices on products and services is becoming a critical decision for many companies. • A good example is pricing hotel rooms and airline tickets. To many airline customers, ticket pricing appears to be madness on the part of the airlines (how can it cost less to fly thousands of miles to London than to fly a couple of hundred miles within the United States? ), but there is a method to the madness. • In this section, we examine several pricing problems that can be modeled as NLPs. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Multiple product purchases • Many products create add-ons to other products. • For example, if you own a men’s clothing store, you should recognize that when a person buys a suit, he often buys a shirt or a tie. • Failure to take this into account causes you to price your suits too high—and lose potential sales of shirts and ties. • Example 7. 3 illustrates the idea. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Peak-load and off-peak demands • In many situations, there are peak-load and offpeak demands for a product. • In such a situation, it might be optimal for a producer to charge a larger price for peak-load service than for off-peak service. • Example 7. 4 illustrates this situation. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Advertising response and selection models • In Chapter 4, we discussed an advertising allocation model (Example 4. 1), where the problem was basically to decide how many ads to place on various television shows to reach the required number of viewers. • One assumption of that model was that the “advertising response” - that is, the number of exposures - is linear in the number of ads. This means that if one ad gains, say, one million exposures, then 10 ads will gain 10 million exposures. • This is a questionable assumption at best. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Advertising response and selection models continued • More likely, there is a decreasing marginal effect at work, where each extra ad gains fewer exposures than the previous ad. • In fact, there might even be a saturation effect, where there is an upper limit on the number of exposures possible and, after sufficiently many ads, this saturation level is reached. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Advertising response and selection models continued • In this section, we look at two related examples. • In the first example, a company uses historical data to estimate its advertising response function - the number of exposures it gains from a given number of ads. This is a nonlinear optimization model. • This type of advertising response function is used in the second example to solve a nonlinear version of the advertising selection problem from Chapter 4. Because the advertising response functions are nonlinear, the advertising selection problem is also nonlinear. • This model is demonstrated by Example 7. 5 Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Advertising selection model • Now that you know how a company can estimate the advertising response function for any type of ad to any group of customers, you can use this type of response function in an advertising selection model. • This model is shown in Example 7. 6 Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Facility location models • Suppose you need to find a location for a facility such as a warehouse, a tool crib in a factory, or a fire station. • Your goal is to locate the facility to minimize the total distance that must be traveled to provide required services. • Facility location problems such as these can usually be set up as NLP models. • Example 7. 7 is typical. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Models for rating sports teams • Sports fans always wonder which team is best in a given sport. Was USC, LSU, or Oklahoma number one during the 2003 NCAA football season? • You might be surprised to learn that Solver can be used to rate sports teams. • We illustrate one method for doing this in example 7. 8 Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Methodology for rating sports teams • We first need to explain the methodology used to rate teams. Suppose that a team plays at home against another team. Then our prediction for the point spread of the game (home team score minus visitor team score) is Predicted point spread = Home team rating - Visitor team rating + Home team advantage • The home team advantage is the number of points extra for the home team because of the psychological (or physical) advantage of playing on its home field. Football experts claim that this home team advantage in the NFL is about 3 points. However, we will estimate it, as well as the ratings. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Methodology for rating sports teams continued • We define the prediction error to be Prediction error = Actual point spread - Predicted point spread • We determine ratings that minimize the sum of squared prediction errors. To get a unique answer to the problem, we need to “normalize” the ratings - that is, fix the average rating at some nominal value. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Portfolio optimization models • Given a set of investments, how do financial analysts determine the portfolio that has the lowest risk and yields a high expected return? • This question was answered by Harry Markowitz in the 1950 s. For his work on this and other investment topics, he received the Nobel Prize in economics in 1990. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Portfolio optimization models continued • The ideas discussed in this section are the basis for most methods of asset allocation used by Wall Street firms. • Asset allocation models are used, for example, to determine the percentage of assets to invest in stocks, gold, and Treasury bills. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Portfolio selection models • Most investors have two objectives in forming portfolios: to obtain a large expected return and to obtain a small variance (to minimize risk). • The problem is inherently nonlinear because variance is a nonlinear function of the investment amounts. • The most common way of handling this two-objective problem is to require a minimal expected return and then minimize the variance subject to the constraint on the expected return. • Example 7. 9 illustrates how to accomplish this in Excel. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock • For financial analysts, it is important to be able to predict the return on a stock from the return on the market, that is, on a market index such as the S&P 500 index. • Here, the return on an investment over a time period is the percentage change in its value over the time period. • There is a variable called beta (β), which is never known but can only be estimated. It measures the responsiveness of a stock’s return to changes in the market return. – The returns on stocks with large positive or negative betas are highly sensitive to the business cycle. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock continued • Sharpe’s capital asset pricing model (CAPM) implies that stocks with large beta values are riskier and therefore must yield higher returns than those with small beta values. • This implies that if you can estimate beta values more accurately than people on Wall Street, you can better identify overvalued and undervalued stocks and make a lot of money. • There are four possible criteria for choosing the unknown estimates of beta. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock continued • Criterion 1: Sum of squared errors (Least Squares) Here the objective is to minimize the sum of the squared errors over all observations, the same criterion used elsewhere in this chapter. The sum of the squared errors is a convex function of the estimates a and b, so Solver is guaranteed to find the (unique) estimates of α and β that minimize the sum of squared errors. The main problem with the least squares criterion is that outliers, points for which the error in Equation (7. 12) is especially large, exert a disproportionate influence on the estimates of α and β. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock continued • Criterion 2: Weighted sum of squared errors Criterion 1 gives equal weights to older and more recent observations. It seems reasonable that more recent observations have more to say about the beta of a stock, at least for future predictions, than older observations. To incorporate this idea, a smaller weight is attached to the squared errors for older observations. Although this method usually leads to more accurate predictions of the future than least squares, the least squares method has many desirable statistical properties that weighted least squares estimates do not possess. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock continued • Criterion 3: Sum of absolute errors (SAE) Instead of minimizing the sum of the squared errors, it makes sense to minimize the sum of the absolute errors for all observations. This is often called the sum of absolute errors (SAE) approach. This method has the advantage of not being greatly affected by outliers. Unfortunately, less is known about the statistical properties of SAE estimates. Another drawback to SAE is that there can be more than one combination of a and b that minimizes SAE. However, SAE estimates have the advantage that they can be obtained with linear programming. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Estimating the beta of a stock continued • Criterion 4: Minimax A final possibility is to minimize the maximum absolute error over all observations. This method might be appropriate for a highly risk-averse decision maker. This minimax criterion can also be implemented using LP. • Example 7. 10 illustrates how Solver can be used to obtain estimates of α and β for these four criteria. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Conclusion • A large number of real-world problems can be approximated well by linear models. • However, many problems are also inherently nonlinear. • We have illustrated several such problems in this chapter, including the important class of portfolio selection problems where the risk, usually measured by portfolio variance, is a nonlinear function of the decision variables. Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Summary of key management science terms Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Summary of key management science terms continued Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

Summary of key Excel terms Winston/Albright Practical Management Science, 4 e South-Western/Cengage Learning © 2012 Thomson/South-Western 2007 ©

End of Chapter 7