CSCECE 791 B Survivable Networks Spring 2008 Optimization

CSC/ECE 791 B - Survivable Networks Spring 2008 Optimization Approaches in Network Design Copyright Rudra Dutta, NCSU, Spring, 2009

• Network problems often represented as graph problems • Most “problems” are NOT “solvable” • Others are merely hard to solve • This includes many practically interesting problems • Sometimes solutions are not just costly, but. . . • Difficult or impossible to express • Optimization techniques allow modeling such problems • Sometimes provide “good” if not “perfect” solutions • Bottom-line: part of the language of complex network design, must understand Copyright Rudra Dutta, NCSU, Spring, 2009

Complexity Theory, NP-Completeness Copyright Rudra Dutta, NCSU, Spring, 2009

Algorithms Information representation by bits l Algorithms as manipulators of bits l Abstract manipulations of abstract data types l – Not specific to machine architecture etc. Copyright Rudra Dutta, NCSU, Spring, 2009 4

Asymptotic Complexity l Complexity - number of elemental steps required What is a step? – Worst case, average case? – Naturally, problem instances of bigger “size” can reasonably take more steps l Asymptotic - beyond a certain “size” f, g: N N f is O(g) iff N, c: f(n) c g(n) n N g is (f) f is (g) iff f is O(g) and (g) l Copyright Rudra Dutta, NCSU, Spring, 2009 5

Languages Alphabet - a finite set of symbols (S) l String - finite ordered list of symbols from S l – l The empty string is denoted by e Universe of strings S* – All strings of length k - Sk Concatenation, suffix, prefix, substring l Language - a subset of S* l Characteristic function c. L l Examples on S = {0, 1} l L = {00, 01, 10, 11} = S 2 – L = {x ends with a single 0 after one or more 1’s} – Copyright Rudra Dutta, NCSU, Spring, 2009 6

Infinite Sets and Computability Function, domain, co-domain, range l Surjective (onto) l – l Range = co-domain Injective (one-to-one) – Distinct elements mapped to distinct elements Bijective: both l Bijection allows comparison of infinite set sizes l |N| = |E| = |O| = 0 , countable infinity – Q = N x N, 0. 0 = 0 – Diagonalization argument : R is not countable – 2 N is not countable - most functions not computable – Copyright Rudra Dutta, NCSU, Spring, 2009 7

Automata l Finite Automata or FSM States, transition function – FSM’s as language acceptors – l Determinism NDFA: accept if an accepting sequence exists – Magic – DFA’s in parallel (not inherently different from DFA) – 1 0, 1 0 0 0, 1 1 1 0, 1 Copyright Rudra Dutta, NCSU, Spring, 2009 8

Computational Models l A language (or function) is a problem – l Regular expressions and regular languages – l Program computes desired quantity for each instance of a problem - decision or optimization Shown to be equivalent to finite automata The Halting Problem – A diagonalization argument on programs Assume a program P( , ) exists to predict if a given program Q stops correctly on some input R l Create G( ) which stops on E only if P returns “no” when Q=R = E l Now call G (G) l l Universal models of computation – Several now known, (polynomial) equivalent Copyright Rudra Dutta, NCSU, Spring, 2009 9

Turing Machines l Turing machine is quite convenient Concept of “steps” and “space” are obvious – Difference in scale with practical computers – l Reads problem on tape, ends with solution Non-determinism does not increase power – Multiple tapes do not increase power – l Acceptors and enumerators Finite-state control Copyright Rudra Dutta, NCSU, Spring, 2009 10

Further on Computability Recursive and R. E. sets l Rice’s Theorem l Function classes l Reductions, classes of reduction – Hardness, easiness, completeness – All partial functions total partial recursive not total primitive recursive Copyright Rudra Dutta, NCSU, Spring, 2009 11

Classes and Reductions Some problems are equivalent to others l Others are “at least as difficult as” l Concept of reduction l – l More explanatory names: “equivalence” or “transformability” Different kinds of reduction Generic versus specific – Turing versus many-to-one – l Reductions allow equivalence classes to be built – But finding first complete problem (generic) could be difficult Copyright Rudra Dutta, NCSU, Spring, 2009 12

Equivalence Classes Copyright Rudra Dutta, NCSU, Spring, 2009 13

Cook’s Theorem l Satisfiability is NP-Complete A collection of Boolean clauses – Problem: Decide whether truth assignment satisfying all the clauses exists – l Given an instance of a decision problem in NP, an instance of Satisfiability can be produced in polynomial time, which is satisfiable iff the original instance is a “yes” instance Copyright Rudra Dutta, NCSU, Spring, 2009 14

Many-to-One Reductions First show problem at hand is in NP l Pick a known NPC problem to reduce from l All instances of the problem must be amenable to reduction – If not, danger of picking tractable subset – l Define a function to transform instance into instance of target Must be polynomial procedure – Need not reach all instances of target – Hence many-to-one, not one-to-one or onto – l Special case: restriction – Of problem space, not solution method Copyright Rudra Dutta, NCSU, Spring, 2009 15

Decision and Optimization Most problems can be stated in either of two versions l Generally interchangable l Proof of computational complexity class with one is often taken as proof for another – Complexity proofs attempted for decision versions – l Decision - quantified yes-no question – l “Can you finish your shopping within x dollars? ” Optimization - best-of-class question – “How few dollars can you finish your shopping within? ” Copyright Rudra Dutta, NCSU, Spring, 2009 16

Well Known NPC Problems l Three-satisfiability Each clause has exactly three literals – 1 in 3 Sat – NAE 3 Sat – l l l Max. Cut Partition Exact Cover by 3 -sets Chromatic number Knapsack Bin-packing Copyright Rudra Dutta, NCSU, Spring, 2009 17

An Example Knapsack: Given a set of elements, a positive integer size and value for each, a positive integer size upper bound and value lower bound, does a subset meeting the bounds exist? l Partition: Given a set of elements, each with a positive integer size such that the sum of all sizes is an even number, can the set by partitioned into two subsets such that the sum of elements in each subset is equal? l Restriction: In Knapsack, allow only instances where the sum of all sizes is a multiple of 2, size equals value for each element, and bounds are each equal to half the sum of sizes l Copyright Rudra Dutta, NCSU, Spring, 2009 18

Approximations Practically, very close may be as good as there l “Close” is defined as fraction of or difference from the optimal objective value l – Must be constant An approximation to an NPC problem, which is in P, is quite valuable l May be possible to customize the algorithm to obtain any desired approximation ratio l – l Polynomial Time Approximation Scheme However, many problems are NP-approximable Copyright Rudra Dutta, NCSU, Spring, 2009 19

Networking Context Many network design problems are complex l It is important to know whether optimal solutions may be tractably attempted l If not, approximations/heuristics may be attempted l NPC reduction method may provide an insight into profitable heuristic methods l Copyright Rudra Dutta, NCSU, Spring, 2009 20

References l M. R. Garey, D. S. Johnson, Computers and Intractability: A Guide to the Theory of NPCompleteness, W. H. Freeman and Company, 1979 – Problem numbers later in the slides refer to the list of problems in this book Copyright Rudra Dutta, NCSU, Spring, 2009 21

Graph Algorithms Copyright Rudra Dutta, NCSU, Spring, 2009

Graphs Common useful abstraction of topology l Nodes, arcs (including loops and multiple arcs) l – G = {V(G), E(G)} Simple graphs, finite graphs l Directed and undirected edges l Subgraphs, complement of a graph l Representation l – l Adjacency list, incidence matrix Useful for modeling many problems Copyright Rudra Dutta, NCSU, Spring, 2009 23

Common Concepts l l l Paths, cycles (Pn and Cn) (walks, trails) Connectedness Cliques - Completely connected graphs (Kn) Independent Set - pairwise nonadjacent set Bipartite graphs - V(G) is the union of two disjoint independent sets Isomorphism – l Bijection between V(G) and V(H) that preserves edges and non-edges Labeling or coloring Copyright Rudra Dutta, NCSU, Spring, 2009 24

Trees l Connected acyclic graphs – l Root (arbitrary designation) – l Parent, child, leaf, depth Special cases – l “Forest” if not connected Paths, Stars, Spiders An n-vertex graph G: Is connected and has no cycles – Is connected and has n-1 edges – Has n-1 edges and no cycles – Has exactly one path between node u and node v – l Spanning tree Copyright Rudra Dutta, NCSU, Spring, 2009 25

Graph Problems l Path related problems – l Finding shortest paths, k-shortest paths, Hamiltonian Tree related problems Tree traversal problems – Finding specific tree subgraphs (spanning tree) – l Matching and Cover problems Maximal and maximum matchings – Vertex cover, edge cover – Dominating set – Flow problems l Coloring problems l Copyright Rudra Dutta, NCSU, Spring, 2009 26

Flows l Network - a directed graph with capacity function on edges – Designated source and sink nodes Flow - a function from E(G) to N or R l Feasible flow l Total flow out of source equals total flow into sink – Flow is balanced at all other nodes – Flow does not exceed capacity on any edge – l Source/sink cut Partition nodes so that source and sink in two parts – All edges from source part to sink part – Maximum flows, integral flows l (More details later) l Copyright Rudra Dutta, NCSU, Spring, 2009 27

Clustering Finding strongly connected subgraphs l Weighted edges l Progressively increase size of clusters by adding nodes l New node to be added has to satisfy connectedness criteria w. r. t. current subgraph – Weight of links between node and subgraph should exceed weight of links to nodes outside subgraph – “connectedness” score of subgraph should increase by inducting this node – Copyright Rudra Dutta, NCSU, Spring, 2009 28

Complexity of Graph Algorithms Many interesting graph problems are NPC, many others are in P l Often, a “NPC-cliff” exists, at small value of critical parameters (2, 3) l Very often jump is from low order polynomial to NPC l Copyright Rudra Dutta, NCSU, Spring, 2009 29

Path Problems l Shortest path problem - well known, in P Dijkstra’s algorithm, Bellman-Ford, Floyd-Warshall – Can be modified to allow negative edge weights – Meaningless under negative cycles – K-th shortest path - NPH [ND 31] l Longest path problem - NPC [ND 29] l Hamiltonian - NPC l – l Path or circuit [GT 37, GT 39] Traveling Salesman - NPC [ND 22] – Is there a tour less than a certain cost? Copyright Rudra Dutta, NCSU, Spring, 2009 30

Flow Problems Concept of commodity - that which flows l Maximum network flow (single commodity) l Ford-Fulkerson labeling algorithm – Find flow-augmenting paths – l Path over links such that either flow < capacity, or negative Redirecting flow can increase flow – Verify maximum by finding capacity of minimum cut – l m-commodity integral flow - NPC [ND 38, ND 39] – l Even for m = 2 Minimum edge-cost flow - NPC [ND 32] Copyright Rudra Dutta, NCSU, Spring, 2009 31

Matching Problems Minimum maximal matching - NPC [GT 10] l Maximum bipartite matching l – l Weighted bipartite matching – l Augmenting path algorithm Hungarian algorithm Partition into perfect matchings - NPC [GT 16] – Even for no more than 2 parts Copyright Rudra Dutta, NCSU, Spring, 2009 32

Coloring Problems Coloring - an intuitive use, basically labeling l Vertex, Edge, and Face (for planar) coloring l “coloring” usually refers to vertex coloring – k-coloring - using no more than k colors – l Chromatic number – – – l Can graph G be properly colored using no more than k colors? NPC, [GT 4] Polynomial for k = 2 Efficient algorithms also known for many families Approximated to O(n(logn) − 3(loglogn)2) Many other problems Equitable coloring - use each color nearly equally – Exact coloring - every pair of colors identifies edge – Copyright Rudra Dutta, NCSU, Spring, 2009 33

Networking Context Network topology information often translates directly to graphs l Network flows often translate even more easily to traffic conditions l – Note that this is more true of static traffic When problems are tractable, posing them as graph problems might be the complete design l For intractable problems, we try to draw parallels and perhaps obtain approximate solutions l Copyright Rudra Dutta, NCSU, Spring, 2009 34

Capacitated Flow Maximization l l l Concept of “commodity flow” Network modeled as a digraph with capacitated links Designated “source” nodes produce commodity Designated “sink” nodes consume Problem: to maximize commodity flow (or to decide whether a given flow can be obtained) Copyright Rudra Dutta, NCSU, Spring, 2009 35

Network Flow - Graph Term l Remember: In graph theory terms, A network is a directed graph – A flow is a function f(a) which assigns some number to every arc B – l A D Positive and negative total flow – E F Convention - outgoing flow is positive At each node, the total flow on all arcs equals: l sum of magnitudes of flow along incoming arcs l - sum of magnitudes of flow along outgoing arcs C G l arc (a) flow f(a) AB 4 FG 4 BC 2 DC x 3 0 AD 1 DG 0 If positive, node sources some flow, etc. BD 2 CE 2 Multiple commodities must be represented by multiple distinct flow functions AF 7 GE 4 FD 3 DE 1 – l Copyright Rudra Dutta, NCSU, Spring, 2009 36

Single-Source Single-Destination l Sufficient to represent (solve) multi-s&d Decision version - set fictitious arc capacities to desired flow – Optimization version - set them to infinity (very large) – 7 B 3 5 8 VS Copyright Rudra Dutta, NCSU, Spring, 2009 F 6 D E 3 1 4 3 7 A C 5 2 G 9 5 VD 37

Min-cut max-flow theorem l A cut of a graph G(V, E) is a partitioning of the nodes of the graph into two disjoint sets l A cutset is the set of edges l The capacity of a cut is the sum of the weights on the edges of the cutset. l The min-cut max-flow theorem is that : or. . . l More intuitively: “regardless of the routing details, the maximum flow between two nodes is set by the minimum capacity of any cut of the between those two nodes” or…“some combination of spans in parallel will always act as the bottleneck” Copyright Rudra Dutta, NCSU, Spring, 2009 38

Illustration 20 B C 6 6 A 6 D 26 6 12 F 20 8 E 20 32 26 the “min-cut” = 12 Hence --> no routing solution can provide more than 12 units of flow between A -F (i. e. , this is the “max-flow”) Copyright Rudra Dutta, NCSU, Spring, 2009 39

All Distinct Routes 7 B 2 7 C 4 F 3 9 4 1 7 15 3 6 A A E 9 etc. H C 18 6 2 G B B 6 11 E B H (own route parent) ABEH discovered G C D F G 17 5 3 D E 12 F F C 9 10 4 7 B (own parent) C H F B D G D B D H C 15 D route ABFH D 16 8 H G C B 13 (own depth parent) D 14 G depth limit depth Example of first phase of DFS to find all distinct routes from A - H subject to hop-limit of 3 The set (or subsets of) of “all distinct routes” is often required in mesh network capacity design formulations. - very different from set of disjoint or k-shortest paths: |route-set| = O(2 d. H) Copyright Rudra Dutta, NCSU, Spring, 2009 40

Flow Augmentation Algorithm l Max-flow Min-cut theorem The maximal amount of flow is equal to the capacity of a minimal cut – Formalization of the concept of bottleneck – Equivalent: no augmenting path can be found in the residual network l Residual - update capacity by subtracting flow l – May create positive capacity links out of zero capacity links (reverse flow) Augmenting path - path with positive capacity l Iterate to get maximum flow l Copyright Rudra Dutta, NCSU, Spring, 2009 41

Flow Augmentation Example 3 B 7 B A C 5 , 3 3, 3 7, 3 Copyright Rudra Dutta, NCSU, Spring, 2009 3 3 -3=0 D 7 3 7 C 2 0 3 A 0 -(-3)=3 7 B 0 3, 3 D 0 -(-3)=3 A C 5 -3=2 0 -(-3)=3 D 7, 3 7 B 3 -3=0 X 0 3 7 3 C X 2 5 X 0 3 A 3 D 3 3 42

Minimum Cost 1 -Commodity Flow l More practical problem Every link has cost as well as capacity – Posed with integer cost, supply/demand, capacity – Total supply equals total demand – l Question: can a given flow to/from s-/d-nodes be achieved within a given cost bound? – l Feasibility can be checked using max-flow approach Polynomial problem Several known polynomial time algorithms – Capacity scaling algorithm, e. g. – Simpler version: successive shortest path algorithm – l – Not strictly polynomial Refer to excerpt from Ahuja, Magnanti, Orlin Copyright Rudra Dutta, NCSU, Spring, 2009 43

Successive Shortest Path Mass balance: amounts of incoming of outgoing flow balance to total source or sink requirement of a node (zero if it is neither source nor sink) l Pseudoflow: flow function which obeys nonnegativity and capacity, but not mass balance l – Imbalance of a node: difference in flow in and out, corrected by supply/demand Potential: a value maintained for each node l Keep a pseudoflow and a potential vector, both initially zero l Initially all imbalances are equal to supply l Copyright Rudra Dutta, NCSU, Spring, 2009 44

Successive Shortest Path Choose a node k with excess and a node l with deficit l Calculate shortest path distances from k to all other nodes l Use edge costs corrected by node potential differences as costs – Update node potentials, reducing by shortest path cost from k – l Choose a particular path from k to l – l Augment the pseudoflow as much as possible, not exceeding capacity of any link or imbalance of k or l Repeat until no imbalance exists Copyright Rudra Dutta, NCSU, Spring, 2009 45

Multi-Commodity Flow Capacitated, cost-weighted digraph, as before l Now there are distinct “commodities” l – l Producers and consumers for each commodity must be matched, cannot be interchanged In particular, every node may generate a unique commodity for every other node – N x (N-1) distinct commodities Question: can a given set of commodity flows to/from s-/d-nodes be achieved within a given cost bound? l Significantly more difficult l In fact, NP-complete l Optimization version often of practical interest l Copyright Rudra Dutta, NCSU, Spring, 2009 46

Trying your hand l Refer to in-class exercise l First try Traveling Salesman Problem (TSP) l Then try Multi-commodity Flow (MCF) l If you cannot solve by hand, Sketch an algorithm – Need not be possible to execute by hand – Need not execute in reasonable time (obviously) – Must be possible to verify correctness (simply) – Copyright Rudra Dutta, NCSU, Spring, 2009 47

Optimization Copyright Rudra Dutta, NCSU, Spring, 2009

Optimization Another name for minimization or maximization l Usually used when quantity (function) is l Built of complex combinations of variables – Does not vary smoothly like “normal” functions – l Conceptual difficulty in describing the problem Basic entities may be many, of different kinds – Consider the multi-commodity flow problem – Or the simpler version we attempted in class – Contrast with, say, Traveling Salesman Problem – Copyright Rudra Dutta, NCSU, Spring, 2009 49

Elements l Parameters Given input values, constant for a given instance – The list of weights and values in Knapsack, e. g. – l Variables Required output values in solution – List of “yes/no” answers for each object – l Constraints Reflection of characteristics of real-life problem – “Knapsack cannot hold more than 50 lb and remain intact” – l Objective Definition of quantity to be maximized or minimized – Total value of objects tagged “yes” in answer – Copyright Rudra Dutta, NCSU, Spring, 2009 50

Static Optimization Distinct from dynamic or stochastic optimization l Mathematical programming problems l Minimize F(x) – Subject to x S – l l l x is the variable vector S represents constraints (with parameters) F() is the objective function S is a (feasible) solution set or optimization space If S is empty the problem is infeasible We are often interested in the specific class of ((Mixed) Integer ) Linear Programming problems Copyright Rudra Dutta, NCSU, Spring, 2009 51

Feasible Region Much of the ease or difficulty of a problem lies in the shape of the feasible region l If some regular traversal of the solution space exists for which objective function exhibits simple increase or decrease, easy to solve l Difficulty arises when no such traversal exists or is apparent l Copyright Rudra Dutta, NCSU, Spring, 2009 52

Convexity l S is convex if for all points x, y S, the whole segment joining x and y belongs to S – l {ax + (1 - a) y : 0 a 1} S A function f on S is a convex function if x, y S, a [0, 1] – f (ax + (1 - a) y) af (x) + (1 - a) f (y) – Concavity, strictly convex (concave) l If F and S are both convex, such problems have nice properties and are easier to solve l Discrete analogue to convexity l Pareto optimal - for multiple objectives Fi, a solution such that no other exists l – At least as good for all, and better for at least one objective Copyright Rudra Dutta, NCSU, Spring, 2009 53

Gradient and Differentiability l Gradient - generalization of derivative Vector of partial (directional) derivatives – – l If a point is a local minimum, gradient is zero at that point – l For convex functions, an interior point which is a local minimum is also a global minimum Hessian - square matrix of second order partial derivatives – If Hessian is PSD at an interior gradient zero point, that point is locally optimal Copyright Rudra Dutta, NCSU, Spring, 2009 54

Interior and Border Optima Copyright Rudra Dutta, NCSU, Spring, 2009 55

Karush-Kuhn-Tucker Conditions l Often local minima are attained at border points, not interior points – Must be characterized by other means l Let S be defined by (all hi and gj differentiable): l Then multipliers m (unconstrained sign) and l (nonnegative) exist such that: Copyright Rudra Dutta, NCSU, Spring, 2009 56

Further Concepts l l l Perturbation - sensitivity Gradient descent - numerical methods Duality Subgradient Maximization Relaxation Copyright Rudra Dutta, NCSU, Spring, 2009 57

Relaxation l Removing some constraints - relaxing Makes the solution space larger – But may make the problem “easier” – Provides bound on optimal objective of original – l Removing integer constraints – l LP is possible to solve, so beneficial if linear system In general, if problem P is relaxed to Q by removing some constraints, – Feasible region of Q is superset of that of P All solutions to P are also solutions of Q l Optimal solution of P is a solution of Q l But may not be optimal l – Consider edge-disjoint and vertex-disjoint paths Copyright Rudra Dutta, NCSU, Spring, 2009 58

Lagrangian relaxation l A special type of relaxation involving “Lagrangian Multipliers” – l Often turns out to be useful in practice Solving the dual of the given problem Convert one or more or all the constraints into “violation costs” in objective – For given multipliers, often easy to solve – Now optimize new cost get values for multipliers – Results regarding the “duality gap”, “complementary slack” – Copyright Rudra Dutta, NCSU, Spring, 2009 59

Duality and Lagrangian Primal: Lagrangian Function: Dual Problem: Copyright Rudra Dutta, NCSU, Spring, 2009 60

Example: Knapsack l Consider the knapsack instance {(8, 16), (9, 20), (5, 12), (4, 10)} (value, weight) – Weight bound 42, optimization version – l Optimization formulation: Maximize 8 x 1 + 9 x 2 + 5 x 3 + 4 x 4 – S. t. 16 x 1 + 20 x 2 + 12 x 3 + 10 x 4 ≤ 42 – xi {0, 1} i – Not obvious (in general NPC, as we know) l But remove the inequality constraint l Now trivial, set all xi = 1 – Clearly a bound, but not a very good one – Next? – Copyright Rudra Dutta, NCSU, Spring, 2009 61

Example: Knapsack (contd. ) l Now consider the following problem: to maximize: 8 x 1 + 9 x 2 + 5 x 3 + 4 x 4 + l (42 - 16 x 1 - 20 x 2 - 12 x 3 - 10 x 4) – Each xi still required to be from {0, 1} – l For any given l, this is likely to be much easier In general, eliminate constraints one by one into “violation cost” – Unique approach: eliminate all constraints except bounds – Specifically, all inequality constraints – Reduced to equality problem: hence “Lagrangian” – l Now the overall problem is one of choosing l so as to minimize this cost – l Likely to be amenable to numerical search Example adapted from CMU webpage, Michael Trick Copyright Rudra Dutta, NCSU, Spring, 2009 62

(Integer) Linear Programs l Linear - all constraints can be posed as only linear combinations of variables – For long complexity was open - thought NPI l – Comparatively recently resolved to be polynomial Simplex is exponential but usually runs faster If the polyhedron of feasible region has extreme points, and an optimal exists, an extreme optimal exists l Strictly polynomial methods (elliptical, interior points) exist l l Integer - all (pure) or some (mixed) variables are constrained to be integers Jump in practical difficulty level - NPC – Branch-and-bound, dynamic programing, … – l Relaxation of integer constraints is an obvious way to solve practically Need to round off solutions – But if integer constraints are binary, not a good method – In general binary problems are harder and “best” formulations are not easy to characterize – Copyright Rudra Dutta, NCSU, Spring, 2009 63

Formulating LP Problems l A steel company must decide how to allocate production time on a rolling mill. The mill takes unfinished slabs of steel as input and can produce either of two products: bands and coils. Bands roll off the mill at 200 Tons/hr, generate $25/Ton profit, and at most 6000 Tons per week can be produced. The same figures for coils are 140 Tons/hr, $30/Ton profit, 4000 Tons/wk. If 40 hours of production time are available, what should be produced to maximize profit? l Verbal model l Put the objective and constraints into words – For constraints, use the form XB number of tons of bands produced. – XC number of tons of coils produced. – – {a verbal description of the LHS} {a relationship} {an RHS constant} Define the Decision Variables l Construct the Symbolic Model Maximize: total profit Subject to: total number of production hours 40 tons of bands produced 6, 000 tons of coils produced 4, 000 Example adapted from “AMPL”, by Kernighan et al, 1993 Copyright Rudra Dutta, NCSU, Spring, 2009 64

Solving LP Problems l Graphical Solution Method Coils Constraints 6000 Hours Profit 220 K 192 K 4000 Optimal solution 2000 Feasible region 2000 Bands 0 0 2000 Copyright Rudra Dutta, NCSU, Spring, 2009 4000 6000 8000 120 K Bands 0 0 2000 4000 6000 8000 65

Solving LP Problems l 4 Possible Outcomes Unique Optimal Solution No Feasible Solution Copyright Rudra Dutta, NCSU, Spring, 2009 Alternate Optimal Solutions Unbounded Optimal Solution 66

Solving LP Problems l Simplex method Efficient algorithm to solve LP problems by performing matrix operations on the LP Tableau – Developed by George Dantzig (1947) – Can be used to solve small LP problems by hand – l AMPL and CPLEX AMPL: modeling language (and software) for designing large and complex LP/IP problems (now use OPL) – CPLEX: software package (“solver”) to solve large and complex LP/IP problems – l Sub-Optimal Algorithms (Heuristics) Simulated annealing – Genetic algorithms – Tabu search – Many others, often very specific to the type of problem. – Copyright Rudra Dutta, NCSU, Spring, 2009 67

Integer Programming l Convert Example to Integer Program Assume that orders for bands and coils are placed (and filled) in 1, 000 s of pounds only. – Although feasible region is greatly reduced, problem becomes much more difficult. – l New Symbolic Model – Let the new decision variables be the number of 1000 pound “units” or orders of bands and coils. Maximiz e: Subject to: integer Copyright Rudra Dutta, NCSU, Spring, 2009 68

Integer Programming l Graphical Solution Method Coils 6 $185 K Optimal integer solution ($185 K) 4 2 Feasible integer solutions Bands 0 0 Copyright Rudra Dutta, NCSU, Spring, 2009 2 4 6 8 69

Solving MIP Problems l Branch-and-Bound Procedure – – – – The solution space consists of a tree of LP subproblems, in which each integer variable is either fixed or its integrality constraint is “relaxed. ” The root node of the tree is the LP relaxation of the problem, i. e. all integer variables are relaxed. The relaxation can result in an all integer solution, or a fractional solution (some decision variables are non-integer). If the solution of the relaxation has fractional-valued integer variables, a fractional variable is selected for branching and two new subproblems are generated, each with more restrictive bounds on the branching variable. The subproblems can result in an all integer solution, an infeasible problem or another fractional solution. If the solution is fractional, the process is repeated. Branches are “fathomed” if the subproblem is infeasible, the objective value is worse than the current best integer solution or the subproblem gives an integer solution. Copyright Rudra Dutta, NCSU, Spring, 2009 70

Branch and Bound Tree Bounds Solution 0<=XB<=6 Obj. = 192 K 0 0<=XC<=4 XB = 6. 00 XC = 1. 40 1 3 2 Bounds Solution 0<=XB<=6 Obj. = 180 K 0<=XC<=1 XB = 6. 00 XC = 1. 00 1 2 4 Copyright Rudra Dutta, NCSU, Spring, 2009 Solution *Infeasible 7 6 4 Bounds 6<=XB<=6 6 2<=XC<=4 3 5 Bounds Solution 0<=XB<=5 Obj. = 185 K 2<=XC<=2 XB = 5. 00 XC = 2. 00 10 9 Bounds Solution 0<=XB<=5 Obj. = 189 K 2<=XC<=4 XB = 5. 00 XC = 2. 10 Solution Bounds 0<=XB<=6 Obj. = 189 K 2<=XC<=4 XB = 5. 14 XC = 2. 00 8 5 Bounds Solution 0<=XB<=5 Obj. = 183 K 3<=XC<=4 XB = 3. 00 XC = 3. 71 71

Network Context l Many networking problems are linear programs Commodity flow problems – Objective function sometimes poses problems – Integer constraints often appear – We are interested not only in optimal value, but also actual solution l Practically, solving ILP is not a good approach l However, domain specific knowledge may allow specific relaxation or other approach l Copyright Rudra Dutta, NCSU, Spring, 2009 72

Multi-Commodity Flow Formulation l Parameters – – – l n : number of nodes i A : set of all links (i, j) uij : bitrate of link cij : cost per bit on link bkl : traffic demand from node k to node l j Variables – l l k xklij : traffic from k to l using link from i to j Goal: minimize total cost Source: Bertsimas and Tsitsilkis Copyright Rudra Dutta, NCSU, Spring, 2009 73

Multi-Commodity Flow Formulation i Copyright Rudra Dutta, NCSU, Spring, 2009 74

Node-Link and Link-Path Two ways of formulating the same thing l Node-link - nodes, links are entities of interest l Focus on traffic demand to and from nodes, on links – Bridging variable - demand between nodes on links – Like previous example – l Link-path - links, paths are entities of interest Demand is still between nodes – For each given demand node pair, list all paths – Assign variable to path traffic flow (implicitly identifies demand) – For each link, sum up path flow variables, and constrain with capacities – Copyright Rudra Dutta, NCSU, Spring, 2009 75

Programming Assignment (Implement the flow augmentation algorithm for capacitated single-commodity flow maximization) l Implement the successive shortest path algorithm for minimum-cost capacitated single commodity flow l Realize the ILP formulation for minimum-cost capacitated multi commodity flow in CPLEX l – Use to solve single commodity flow problem instances only (simplify formulation first) Copyright Rudra Dutta, NCSU, Spring, 2009 76