Optimization Approaches in Network Design ECECSC 777 Telecommunications

Optimization Approaches in Network Design ECE/CSC 777: Telecommunications Network Design Fall, 2013, Rudra Dutta

Algorithms Information representation by bits l Algorithms as manipulators of bits l Abstract manipulations of abstract data types l – Not specific to machine architecture etc. Copyright Rudra Dutta, NCSU, Fall, 2013 2

Asymptotic Complexity l Complexity - number of elemental steps required What is a step? – Worst case, average case? – Naturally, problem instances of bigger “size” can reasonably take more steps l Asymptotic - beyond a certain “size” l f, g: N N f is O(g) iff N, c: f(n) c g(n) n N g is (f) f is (g) iff f is O(g) and (g) Copyright Rudra Dutta, NCSU, Fall, 2013 3

Turing Machines l Turing machine is quite convenient Concept of “steps” and “space” are obvious – Difference in scale with practical computers – l Reads problem on tape, ends with solution Non-determinism does not increase power – Multiple tapes do not increase power – l Acceptors and enumerators (of languages) Finite-state control Copyright Rudra Dutta, NCSU, Fall, 2013 4

Classes and Reductions Some problems are equivalent to others l Others are “at least as difficult as” l Concept of reduction l – l More explanatory names: “equivalence” or “transformability” Different kinds of reduction Generic versus specific – Turing versus many-to-one – l Reductions allow equivalence classes to be built – But finding first complete problem (generic) could be difficult Copyright Rudra Dutta, NCSU, Fall, 2013 5

Equivalence Classes Copyright Rudra Dutta, NCSU, Fall, 2013 6

Cook’s Theorem l Satisfiability is NP-Complete A collection of Boolean clauses – Problem: Decide whether truth assignment satisfying all the clauses exists – l Given an instance of a decision problem in NP, an instance of Satisfiability can be produced in polynomial time, which is satisfiable iff the original instance is a “yes” instance Copyright Rudra Dutta, NCSU, Fall, 2013 7

Many-to-One Reductions First show problem at hand is in NP l Pick a known NPC problem to reduce from l All instances of the problem must be amenable to reduction – If not, danger of picking tractable subset – l Define a function to transform instance into instance of target Must be polynomial procedure – Need not reach all instances of target – Hence many-to-one, not one-to-one or onto – l Special case: restriction – Of problem space, not solution method Copyright Rudra Dutta, NCSU, Fall, 2013 8

Decision and Optimization Most problems can be stated in either of two versions l Generally interchangable l Proof of computational complexity class with one is often taken as proof for another – Complexity proofs attempted for decision versions – l Decision - quantified yes-no question – l “Can you finish your shopping within x dollars? ” Optimization - best-of-class question – “How few dollars can you finish your shopping within? ” Copyright Rudra Dutta, NCSU, Fall, 2013 9

Well Known NPC Problems l Three-satisfiability Each clause has exactly three literals – 1 in 3 Sat – NAE 3 Sat – l l l Max. Cut Partition Exact Cover by 3 -sets Chromatic number Knapsack Bin-packing Copyright Rudra Dutta, NCSU, Fall, 2013 10

An Example Knapsack: Given a set of elements, a positive integer size and value for each, a positive integer size upper bound and value lower bound, does a subset meeting the bounds exist? l Partition: (NPC) Given a set of elements, each with a positive integer size such that the sum of all sizes is an even number, can the set by partitioned into two subsets such that the sum of elements in each subset is equal? l Proof by Restriction l In Knapsack, allow only instances where the sum of all sizes is a multiple of 2, size equals value for each element, and bounds are each equal to half the sum of sizes – Reduces to Partition, so Knapsack must be at least as difficult – Copyright Rudra Dutta, NCSU, Fall, 2013 11

Approximations Practically, very close may be as good as there l “Close” is defined as fraction of or difference from the optimal objective value l – Must be constant An approximation to an NPC problem, which is in P, is quite valuable l May be possible to customize the algorithm to obtain any desired approximation ratio l – l Polynomial Time Approximation Scheme However, many problems are NP-approximable Copyright Rudra Dutta, NCSU, Fall, 2013 12

References l M. R. Garey, D. S. Johnson, Computers and Intractability: A Guide to the Theory of NPCompleteness, W. H. Freeman and Company, 1979 Copyright Rudra Dutta, NCSU, Fall, 2013 13

Networking Context Network topology information often translates directly to graphs l Network flows often translate even more easily to traffic conditions l – Note that this is truer of more static traffic When problems are tractable, posing them as graph problems might be the complete design l For intractable problems, we try to draw parallels and perhaps obtain approximate solutions l Copyright Rudra Dutta, NCSU, Fall, 2013 14

Capacitated Flow Maximization l l l Concept of “commodity flow” Network modeled as a digraph with capacitated links Designated “source” nodes produce commodity Designated “sink” nodes consume Problem: to maximize commodity flow (or to decide whether a given flow can be obtained) Copyright Rudra Dutta, NCSU, Fall, 2013 15

Network Flow - Graph Term l Remember: In graph theory terms, A network is a directed graph – A flow is a function f(a) which assigns some number to every arc B – l A D Positive and negative total flow – E F Convention - outgoing flow is positive At each node, the total flow on all arcs equals: l sum of magnitudes of flow along incoming arcs l - sum of magnitudes of flow along outgoing arcs C G l arc (a) flow f(a) AB 4 FG 4 BC 2 DC x 3 0 AD 1 DG 0 If positive, node sources some flow, etc. BD 2 CE 2 Multiple commodities must be represented by multiple distinct flow functions AF 7 GE 4 FD 3 DE 1 – l Copyright Rudra Dutta, NCSU, Fall, 2013 16

Single-Source Single-Destination l Sufficient to represent (solve) multi-s&d Decision version - set fictitious arc capacities to desired flow – Optimization version - set them to infinity (very large) – 7 B 3 5 8 VS Copyright Rudra Dutta, NCSU, Fall, 2013 F 6 D E 3 1 4 3 7 A C 5 2 G 9 5 VD 17

Min-cut max-flow theorem A cut of a graph G(V, E) is a partitioning of the nodes of the graph into two disjoint sets l A cutset is the set of edges that go from one to the other l The capacity of a cut is the sum of the capacities on the edges of the cutset l Regardless of routing, the maximum flow between two nodes is fixed by the minimum capacity of any cut of the graph between those two nodes l Some combination of edges (in parallel) will always act as the bottleneck – There exists a routing that achieves this, others may result in less – Copyright Rudra Dutta, NCSU, Fall, 2013 18

Flow Augmentation Algorithm l Max-flow Min-cut theorem The maximal amount of flow is equal to the capacity of a minimal cut – Formalization of the concept of bottleneck – Equivalent: no augmenting path can be found in the residual network l Residual - update capacity by subtracting flow l – May create positive capacity links out of zero capacity links (reverse flow) Augmenting path - path with positive capacity l Iterate to get maximum flow l Copyright Rudra Dutta, NCSU, Fall, 2013 19

Flow Augmentation Example 3 B 7 B A C 5 , 3 3, 3 7, 3 Copyright Rudra Dutta, NCSU, Fall, 2013 3 3 -3=0 D 7 3 7 C 2 0 3 A 0 -(-3)=3 7 B 0 3, 3 D 0 -(-3)=3 A C 5 -3=2 0 -(-3)=3 D 7, 3 7 B 3 -3=0 X 0 3 7 3 C X 2 5 X 0 3 A 3 D 3 3 20

Minimum Cost 1 -Commodity Flow l More practical problem Every link has cost as well as capacity – Posed with integer cost, supply/demand, capacity – Total supply equals total demand – l Question: can a given flow to/from s-/d-nodes be achieved within a given cost bound? – l Feasibility can be checked using max-flow approach Polynomial problem Several known polynomial time algorithms – Capacity scaling algorithm, e. g. – Simpler version: successive shortest path algorithm – l – Not strictly polynomial Refer to excerpt from Ahuja, Magnanti, Orlin Copyright Rudra Dutta, NCSU, Fall, 2013 21

Successive Shortest Path Mass balance: amounts of incoming of outgoing flow balance to total source or sink requirement of a node (zero if it is neither source nor sink) l Pseudoflow: flow function which obeys nonnegativity and capacity, but not mass balance l – Imbalance of a node: difference in flow in and out, corrected by supply/demand Potential: a value maintained for each node l Keep a pseudoflow and a potential vector, both initially zero l Initially all imbalances are equal to supply l Copyright Rudra Dutta, NCSU, Fall, 2013 22

Successive Shortest Path Choose a node k with excess and a node l with deficit l Calculate shortest path distances from k to all other nodes l Use edge costs corrected by node potential differences as costs – Update node potentials, reducing by shortest path cost from k – l Choose a particular path from k to l – l Augment the pseudoflow as much as possible, not exceeding capacity of any link or imbalance of k or l Repeat until no imbalance exists Copyright Rudra Dutta, NCSU, Fall, 2013 23

Multi-Commodity Flow Capacitated, cost-weighted digraph, as before l Now there are distinct “commodities” l – l Producers and consumers for each commodity must be matched, cannot be interchanged In particular, every node may generate a unique commodity for every other node – N x (N-1) distinct commodities Question: can a given set of commodity flows to/from s-/d-nodes be achieved within a given cost bound? l Significantly more difficult l In fact, NP-complete l Optimization version often of practical interest l Copyright Rudra Dutta, NCSU, Fall, 2013 24

Trying your hand l Refer to in-class exercise l First try Traveling Salesman Problem (TSP) l Then try Multi-commodity Flow (MCF) l If you cannot solve by hand, Sketch an algorithm – Need not be possible to execute by hand – Need not execute in reasonable time (obviously) – Must be possible to verify correctness (simply) – Copyright Rudra Dutta, NCSU, Fall, 2013 25

Optimization

Optimization Another name for minimization or maximization l Usually used when quantity (function) is l Built of complex combinations of variables – Does not vary smoothly like “normal” functions – l Conceptual difficulty in describing the problem Basic entities may be many, of different kinds – Consider the multi-commodity flow problem – Or the simpler version we attempted in class – Contrast with, say, Traveling Salesman Problem – Copyright Rudra Dutta, NCSU, Fall, 2013 27

Elements l Parameters Given input values, constant for a given instance – The list of weights and values in Knapsack, e. g. – l Variables Required output values in solution – List of “yes/no” answers for each object – l Constraints Reflection of characteristics of real-life problem – “Knapsack cannot hold more than 50 lb and remain intact” – l Objective Definition of quantity to be maximized or minimized – Total value of objects tagged “yes” in answer – Copyright Rudra Dutta, NCSU, Fall, 2013 28

Static Optimization Distinct from dynamic or stochastic optimization l Mathematical programming problems l Minimize F(x) – Subject to x S – l l l x is the variable vector S (together with parameters) represents constraints F() is the objective function S is a (feasible) solution set or optimization space If S is empty the problem is infeasible We are often interested in the specific class of ((Mixed) Integer ) Linear Programming problems Copyright Rudra Dutta, NCSU, Fall, 2013 29

Solution as Variable Vector l Say there are two variables x 1 and x 2 (say, max delay on line, avg delay on line) – “Unknowns”, like algebra – Can consider generally as variable vector x = {x 1, x 2} – A particular “solution” consists of assigning a particular value to each variable l A “solution” is a point in the space defined by the variables l Sometimes called state space – Part of the space might be “out of bounds” due to meaning of variables (e. g. x 2 cannot exceed x 1) – Copyright Rudra Dutta, NCSU, Fall, 2013 30

Feasible Region Much of the ease or difficulty of a problem lies in the shape of the feasible region l If some regular traversal of the solution space exists for which objective function exhibits simple increase or decrease, easy to solve l Difficulty arises when no such traversal exists or is apparent l Copyright Rudra Dutta, NCSU, Fall, 2013 Objective x 2 x 1 31

Convexity S is convex if for all points x, y in S, the whole segment joining x and y belongs to S (is “above”) l A function f on S is a convex function in a similar sense l Concavity, strictly convex (concave) l If F and S are both convex, such problems have nice properties and are easier to solve l Discrete analogue to convexity l Pareto optimal - for multiple objectives Fi, a solution such that no other exists l – At least as good for all, and better for at least one objective Copyright Rudra Dutta, NCSU, Fall, 2013 32

Gradient and Differentiability l Gradient - generalization of derivative Vector of partial (directional) derivatives – – l If a point is a local minimum, gradient is zero at that point – l For convex functions, an interior point which is a local minimum is also a global minimum Hessian - square matrix of second order partial derivatives – If Hessian is PSD at an interior gradient zero point, that point is locally optimal Copyright Rudra Dutta, NCSU, Fall, 2013 33

Interior and Border Optima l Often local minima are attained at border points, not interior points Must be characterized by other means – Karush-Kuhn-Tucker Conditions – Copyright Rudra Dutta, NCSU, Fall, 2013 34

Further Concepts l l l Perturbation - sensitivity Gradient descent - numerical methods Duality Subgradient Maximization Relaxation Copyright Rudra Dutta, NCSU, Fall, 2013 35

Relaxation l Removing some constraints - relaxing Makes the solution space larger – But may make the problem “easier” – Provides bound on optimal objective of original – l Removing integer constraints – l LP is possible to solve, so beneficial if linear system In general, if problem P is relaxed to Q by removing some constraints, – Feasible region of Q is superset of that of P All solutions to P are also solutions of Q l Optimal solution of P is a solution of Q l But may not be optimal l – Consider edge-disjoint and vertex-disjoint paths Copyright Rudra Dutta, NCSU, Fall, 2013 36

Lagrangian relaxation l A special type of relaxation involving “Lagrangian Multipliers” – l Often turns out to be useful in practice Solving the dual of the given problem Convert one or more or all the constraints into “violation costs” in objective – For given multipliers, often easy to solve – Now optimize new cost get values for multipliers – Results regarding the “duality gap”, “complementary slack” – Copyright Rudra Dutta, NCSU, Fall, 2013 37

Duality and Lagrangian Primal: Lagrangian Function: Dual Problem: Copyright Rudra Dutta, NCSU, Fall, 2013 38

Example: Knapsack l Consider the knapsack instance {(8, 16), (9, 20), (5, 12), (4, 10)} (value, weight) – Weight bound 42, optimization version – l Optimization formulation: Maximize 8 x 1 + 9 x 2 + 5 x 3 + 4 x 4 – S. t. 16 x 1 + 20 x 2 + 12 x 3 + 10 x 4 ≤ 42 – xi {0, 1} i – Not obvious (in general NPC, as we know) l But remove the inequality constraint l Now trivial, set all xi = 1 – Clearly a bound, but not a very good one – Next? – Copyright Rudra Dutta, NCSU, Fall, 2013 39

Example: Knapsack (contd. ) l Now consider the following problem: to maximize: 8 x 1 + 9 x 2 + 5 x 3 + 4 x 4 + l (42 - 16 x 1 - 20 x 2 - 12 x 3 - 10 x 4) – Each xi still required to be from {0, 1} – l For any given l, this is likely to be much easier In general, eliminate constraints one by one into “violation cost” – Unique approach: eliminate all constraints except bounds – Specifically, all inequality constraints – Reduced to equality problem: hence “Lagrangian” – l Now the overall problem is one of choosing l so as to minimize this cost – l Likely to be amenable to numerical search Example adapted from CMU webpage, Michael Trick Copyright Rudra Dutta, NCSU, Fall, 2013 40

(Integer) Linear Programs l Linear - all constraints can be posed as only linear combinations of variables – For long complexity was open - thought NPI l – Comparatively recently resolved to be polynomial Simplex is exponential but usually runs faster If the polyhedron of feasible region has extreme points, and an optimal exists, an extreme optimal exists l Strictly polynomial methods (elliptical, interior points) exist l l Integer - all (pure) or some (mixed) variables are constrained to be integers Jump in practical difficulty level - NPC – Branch-and-bound, dynamic programing, … – l Relaxation of integer constraints is an obvious way to solve practically Need to round off solutions – But if integer constraints are binary, not a good method – In general binary problems are harder and “best” formulations are not easy to characterize – Copyright Rudra Dutta, NCSU, Fall, 2013 41

Formulating LP Problems l A steel company must decide how to allocate production time on a rolling mill. The mill takes unfinished slabs of steel as input and can produce either of two products: bands and coils. Bands roll off the mill at 200 Tons/hr, generate $25/Ton profit, and at most 6000 Tons per week can be produced. The same figures for coils are 140 Tons/hr, $30/Ton profit, 4000 Tons/wk. If 40 hours of production time are available, what should be produced to maximize profit? l Verbal model l Put the objective and constraints into words – For constraints, use the form XB number of tons of bands produced. – XC number of tons of coils produced. – – {a verbal description of the LHS} {a relationship} {an RHS constant} Define the Decision Variables l Construct the Symbolic Model Maximize: total profit Subject to: total number of production hours 40 tons of bands produced 6, 000 tons of coils produced 4, 000 Example adapted from “AMPL”, by Kernighan et al, 1993 Copyright Rudra Dutta, NCSU, Fall, 2013 42

Solving LP Problems l Graphical Solution Method Coils Constraints 6000 Hours Profit 220 K 192 K 4000 Optimal solution 2000 Feasible region 2000 Bands 0 0 2000 Copyright Rudra Dutta, NCSU, Fall, 2013 4000 6000 8000 120 K Bands 0 0 2000 4000 6000 8000 43

Solving LP Problems l 4 Possible Outcomes Unique Optimal Solution No Feasible Solution Copyright Rudra Dutta, NCSU, Fall, 2013 Alternate Optimal Solutions Unbounded Optimal Solution 44

Solving LP Problems l Simplex method Efficient algorithm to solve LP problems by performing matrix operations on the LP Tableau – Developed by George Dantzig (1947) – Can be used to solve small LP problems by hand – l AMPL and CPLEX AMPL: modeling language (and software) for designing large and complex LP/IP problems (now use OPL) – CPLEX: software package (“solver”) to solve large and complex LP/IP problems – l Sub-Optimal Algorithms (Heuristics) Simulated annealing – Genetic algorithms – Tabu search – Many others, often very specific to the type of problem. – Copyright Rudra Dutta, NCSU, Fall, 2013 45

Integer Programming l Convert Example to Integer Program Assume that orders for bands and coils are placed (and filled) in 1, 000 s of pounds only. – Although feasible region is greatly reduced, problem becomes much more difficult. – l New Symbolic Model – Let the new decision variables be the number of 1000 pound “units” or orders of bands and coils. Maximiz e: Subject to: integer Copyright Rudra Dutta, NCSU, Fall, 2013 46

Integer Programming l Graphical Solution Method Coils 6 $185 K Optimal integer solution ($185 K) 4 2 Feasible integer solutions Bands 0 0 Copyright Rudra Dutta, NCSU, Fall, 2013 2 4 6 8 47

Integer Programming Copyright Rudra Dutta, NCSU, Fall, 2013 48

Solving MIP Problems l Branch-and-Bound Procedure – – – – The solution space consists of a tree of LP subproblems, in which each integer variable is either fixed or its integrality constraint is “relaxed. ” The root node of the tree is the LP relaxation of the problem, i. e. all integer variables are relaxed. The relaxation can result in an all integer solution, or a fractional solution (some decision variables are non-integer). If the solution of the relaxation has fractional-valued integer variables, a fractional variable is selected for branching and two new subproblems are generated, each with more restrictive bounds on the branching variable. The subproblems can result in an all integer solution, an infeasible problem or another fractional solution. If the solution is fractional, the process is repeated. Branches are “fathomed” if the subproblem is infeasible, the objective value is worse than the current best integer solution or the subproblem gives an integer solution. Copyright Rudra Dutta, NCSU, Fall, 2013 49

Branch and Bound Tree Bounds Solution 0<=XB<=6 Obj. = 192 K 0 0<=XC<=4 XB = 6. 00 XC = 1. 40 1 3 2 Bounds Solution 0<=XB<=6 Obj. = 180 K 0<=XC<=1 XB = 6. 00 XC = 1. 00 1 2 4 Copyright Rudra Dutta, NCSU, Fall, 2013 Solution *Infeasible 7 6 4 Bounds 6<=XB<=6 6 2<=XC<=4 3 5 Bounds Solution 0<=XB<=5 Obj. = 185 K 2<=XC<=2 XB = 5. 00 XC = 2. 00 10 9 Bounds Solution 0<=XB<=5 Obj. = 189 K 2<=XC<=4 XB = 5. 00 XC = 2. 10 Solution Bounds 0<=XB<=6 Obj. = 189 K 2<=XC<=4 XB = 5. 14 XC = 2. 00 8 5 Bounds Solution 0<=XB<=5 Obj. = 183 K 3<=XC<=4 XB = 3. 00 XC = 3. 71 50

Network Context l Many networking problems are linear programs Commodity flow problems – Objective function sometimes poses problems – Integer constraints often appear – We are interested not only in optimal value, but also actual solution l Practically, solving ILP is not a good approach l However, domain specific knowledge may allow specific relaxation or other approach l Copyright Rudra Dutta, NCSU, Fall, 2013 51

Multi-Commodity Flow Formulation l Parameters – – – l n : number of nodes i A : set of all links (i, j) uij : bitrate of link cij : cost per bit on link bkl : traffic demand from node k to node l j Variables – l l k xijkl : traffic from k to l using link from i to j Goal: minimize total cost Source: Bertsimas and Tsitsilkis Copyright Rudra Dutta, NCSU, Fall, 2013 52

Multi-Commodity Flow Formulation i Copyright Rudra Dutta, NCSU, Fall, 2013 n : number of nodes A : set of all links (i, j) uij : bitrate of link cij : cost per bit on link bkl : traffic demand k to l xkl : k to l traffic using link i to j ij 53

Node-Link and Link-Path Two ways of formulating the same thing l Node-link - nodes, links are entities of interest l Focus on traffic demand to and from nodes, on links – Bridging variable - demand between nodes on links – Like previous example – l Link-path - links, paths are entities of interest Demand is still between nodes – For each given demand node pair, list all paths – Assign variable to path traffic flow (implicitly identifies demand) – For each link, sum up path flow variables, and constrain with capacities – Copyright Rudra Dutta, NCSU, Fall, 2013 54

Telephone Networks Generally heirarchical structure l Various routing schemes l Traffic measured in Erlangs l – l l/m Appropriate for blocking study Regional Center Sectional Centers Primary Centers Toll Centers End Offices Copyright Rudra Dutta, NCSU, Fall, 2013 55

Overflow Routing may be fixed – “load sharing” l May allow calls to find alternate paths l General fixed alternate – Fixed heirarchical – Dynamic non-heirarchical – l B E F Dimensioning problem Under given routing – Find size of each trunk – Target blocking – A O Copyright Rudra Dutta, NCSU, Fall, 2013 C D 56