Chapter 4 Basic Graph Algorithms and Computational Complexity

Chapter 4: Basic Graph Algorithms and Computational Complexity Massoud Pedram Dept. of EE University of Southern California

Outline n n n n M. Pedram Analysis of Algorithms Graph Algorithms Dynamic Programming Mathematical Programming Stochastic Search Greedy Algorithms Divide and Conquer Floorplanning

Analysis of Algorithms n Big-O notation n n Small-o notation n n O(f(n)) means that as n , the execution time t is at most cf(n) for some constant c o(f(n)) means that as n , the execution time t is at least cf(n) for some constant c Theta notation n (f(n)) means that as n , the execution time t is at most c 1 f(n) and at least c 2 f(n) for some constants c 1 and c 2 where n = input size M. Pedram

Big-O Notation n Express the execution time as a function of the input size n. Since only the growth rate matters, we can ignore the multiplicative constants and the lower order terms, e. g. , 3 n 2+6 n+2. 7 is O(n 2) n 1. 1 + 100000 n is O(n 1. 1) n 1. 1 is O(n 1. 1) M. Pedram

Effect of Multiplicative Constant Run time 800 f(n)=n 2 600 400 f(n)=10 n 200 0 M. Pedram 10 20 25 n

Growth Rate of Some Functions O(log n), O(log 2 n), O(n 0. 5), O(n log n), O(n 1. 5), O(n log 2 n), O(n 2), O(n 3), O(n 4) Polynomial Functions O(nlog n), O(2 n), O(3 n), O(4 n), O(n!) Exponential Functions M. Pedram

Exponential Functions n Exponential functions increase rapidly, e. g. , 2 n will double whenever n is increased by 1 n 10 20 30 40 50 60 M. Pedram 2 n 103 106 109 1012 1015 1018 1 ms x 2 n 0. 001 s 1 s 16. 7 mins 11. 6 days 31. 7 years 31710 years

NP-Complete n n n M. Pedram The class NP-Complete is a set of problems that, we believe, has no polynomial time algorithms Therefore, they are hard problems If a problem is NP-complete, there is no hope to solve it efficiently

Solution Type for NP-Complete Problems n n M. Pedram Exponential time algorithms Special case algorithms Approximation algorithms Heuristic algorithms

Graph Algorithms

Basic Graph Definitions n n A graph G(V, E) is a collection of vertices in V connected by a set of edges E Dual of a graph G is obtained by replacing each face of G with one vertex and connecting two such vertices by an edge if the two corresponding faces share a common edge in G A B A D C M. Pedram Graph G B D C Dual Graph G

Types of Graph Commonly Used in VLSI CAD n n n M. Pedram Interval graphs Permutation graphs Comparability graphs Vertical and horizontal constraint graphs Neighborhood graphs and rectangular dualization

Interval Graphs A C B E D G F A C B E F G D V = Set of intervals E = {(vi, vj) | vi and vj intersect} M. Pedram

Permutation Graphs A B C D E B E D A E C A B D C V = Set of lines E = {(vi, vj) | vi and vj intersect} M. Pedram

Comparability Graphs n Orientable Property: Each edge can be assigned a one-way direction in such a way that the resulting graph G(V, F) satisfies the following property: n n An undirected graph which is transitively orientable is called a comparability graph ? Orientable graph M. Pedram Non-orientable graph

Horizontal Constraint Graphs A C D A B w. A w. B w. A C B D w. C Weighted directed graph: V = Set of modules E = {(vi, vj) | vj on the right of vi} weight(vi, vj) = width of module vi M. Pedram

Vertical Constraint Graphs A C D A h. A B B h. B D C Weighted directed graph: V = Set of modules E = {(vi, vj) | vj above vi} weight(vi, vj) = height of module vi M. Pedram

Neighborhood Graphs A B C D A B D C V = Set of modules E = {(vi, vj) | vj and vi share an edge} M. Pedram

Rectangular Dualization Rectangular dualization does the inverse operation of the neighborhood graph construction A B A D C B D C Each rectangle represents a vertex vi and vj share an edge exactly if e(vi, vj) exists Some graphs have no rectangular duals M. Pedram

Basic Graph Algorithms Commonly Used in VLSI CAD n n n M. Pedram Minimum Spanning Tree (MST) Problems Steiner Minimum Tree (SMT) Problems Rectilinear SMT Problems Partitioning Problems Network Flow Problems

Minimum Spanning Tree (MST) Problem: Given a connected undirected weighted graph G(V, E), construct a minimum weighted tree T connecting all vertices in V Note: A graph is connected exactly if there is at least one path for any pair of vertices M. Pedram

Minimum Spanning Tree (MST) n Kruskal’s algorithm: n n Prim’s algorithm : n n M. Pedram O(m log n) Can be improved to O(n log n) A greedy algorithm Note: m = no. of edges; n = no. of vertices

Kruskal’s Algorithm n n n M. Pedram Sort the edges T= Consider the edges in ascending order of their weights until T contains n-1 edges: n Add an edge e to T exactly if adding e to T does not create any cycle in T

Kruskal’s Algorithm Example e e M. Pedram

Kruskal’s Algorithm Example (Cont’d) e e M. Pedram

Kruskal’s Algorithm Example (Cont’d) M. Pedram

Prim’s Algorithm n n n M. Pedram T= S = {v} for an arbitrary vertex v Repeat until S contains all the vertices: n Add the lightest edge e(v 1, v 2) to T where v 1 S and v 2 (V-S). Add v 2 to S

Prim’s algorithm Example M. Pedram

Prim’s algorithm Example (Cont’d) M. Pedram

Steiner Minimum Tree (SMT) Problem: Given an undirected weighted graph G(V, E) and a demand set D V, find a minimum cost tree T which spans a set of vertices V’ V such that D V’ Elements of V’-D, which have a degree larger than 2 in T, are called the Steiner points M. Pedram

Steiner Minimum Tree (SMT) n n n M. Pedram When D = V, SMT = MST When |D| = 2, SMT = Single Pair Shortest Path Problem The SMT problem is NP-Complete

Rectilinear Steiner Tree (RST) n A Steiner tree whose edges are constrained to be vertical or horizontal Steiner point Demand point M. Pedram

Rectilinear Steiner Minimum Tree (RSMT) n n M. Pedram Similar to the SMT problem, except that the tree T is a minimum cost rectilinear Steiner tree This problem is NP-Complete but we can get an approximate solution to this problem by making use of the MST

Approximate Solution to RSMT n n Construct a MST T Obtain a RST T’ from T by connecting the vertices in T using vertical and horizontal edges There can be many solutions or MST M. Pedram a better one

Approximate Solution to RSMT n M. Pedram It is proved that: WMST 1. 5 WRSMT Let W be the weight of the Steiner tree obtained by this method, i. e. , by “rectilinearizing” the MST What is the smallest b such that: W b. WMST

Other Graph Problems n n n M. Pedram Minimum Clique Covering Problem Maximum Independent Set Problem Minimum Coloring Problem Maximum Clique Problem Network Flow Problem

Minimum Clique Covering n n n M. Pedram A clique of a graph G(V, E) is a set of vertices V’ V such that every pair of vertices in V’ are joined by an edge The clique cover number, k, of a graph G(V, E) is the minimum no. of cliques needed to cover all of the vertices of G The problem of finding the clique cover number is NP-complete for general graphs

Maximum Independent Set (MIS) n n M. Pedram An independent set of a graph G(V, E) is a set of vertices V’ V such that no pair of vertices in V’ are joined by an edge The MIS problem is to find the independence (stability) number, a, of a graph, i. e. , the size of the largest independent set in the graph This problem is NP-complete for general graphs What is the relationship between the clique cover number and the independence number? Answer:

Graph Problems in Interval Graphs n n What is the meaning of a clique cover in an interval graph? What is the meaning of a MIS in an interval graph? The clique covering and MIS in an interval graph can be found in O(n log n) time where n is the no. of intervals. How? Answer for MIS: n n M. Pedram Sort the 2 n points in ascending order Scan list from left to right until encounter a right endpoint Output the interval having this right endpoint as a member of MIS and delete all intervals containing this point Repeat until done

Minimum Chromatic Number n n M. Pedram The chromatic number, c, of a graph G(V, E) is the minimum no. of colors needed to color the vertices such that every pair of vertices joined by an edge are colored differently The problem of finding the chromatic number is NP-complete for general graphs

Maximum Clique n n n M. Pedram The maximum clique problem is to find the clique number, w, of a graph, i. e. , the size of the largest clique in the graph This problem is NP-complete for general graphs What is the relationship between the chromatic number and the clique number? Answer:

Graph Problems in Interval Graphs (Cont’d) n n n What is the meaning of the chromatic number and the clique number in an interval graph? The chromatic number and the clique number of an interval graph can be found in O(n log n) time. How? Answer for maximum clique {simple O(n 2) algorithm} n n n A = Sort intervals (I); cliq=0; max-cliq=0; for i=1 to 2 n do n if A[i]=Left then cliq++; n n n M. Pedram if cliq > max-cliq then max-cliq=cliq; else cliq--; Return max-cliq;

Perfect Graphs n A graph G(V, E) is called perfect of all of its induced sub-graphs satisfy the following two properties: n n M. Pedram P 1: P 2: Interval graphs, permutation graphs and orientable graphs are examples of the perfect graphs; Cycle of odd length > 3 is not a perfect graph The four key graph problems can be solved in polynomial time for the class of perfect graphs

Partitioning Problem: Given an undirected graph G(V, E), partition V into two sets V 1 and V 2 of equal sizes such that the number of edges |E 1| between V 1 and V 2 is minimized E 1 is called the cut M. Pedram

Partitioning Problem n More general versions of this problem: n n n M. Pedram Specify the sizes of V 1 and V 2 Partition V into k sets where k 2 All these problems are NP-complete

Network Flow Problem: Given a weighted directed graph G(V, E). One vertex s V is designated as the source and one vertex t V as the sink. Find the maximum flow from s to t such that the flow along each edge does not exceed the capacity (weight) of the edge M. Pedram

Network Flow Problem n For example: a 10 s n n M. Pedram 4 5 7 3 c b 5 2 7 6 d t 10 What is the maximum flow in this network? (see next slide) Network flow problem can be solved in polynomial time

Network Flow Problem n The maximum flow is 9 9/10 s 2/5 7 n 4/4 c 2/2 b 5 3/7 6 d t 6/10 A well known theorem: n M. Pedram a 3/3 Maximum Flow = Minimum Cut

Network Flow Problem n The minimum cut is 9 10 s M. Pedram a 4 c b 5 5 7 n 3 2 7 6 d t 10 Notice that we only count the edges going from s to t

Network Flow Problem n M. Pedram Unfortunately, we cannot use this network flow method to solve the partitioning problem. Why?

Dynamic Programming

Dynamic Programming n M. Pedram In dynamic programming, an array A[1. . n] (can be of higher dimension) of data is defined and stored. The computation of A[k] is dependent on the values of A[j] where j < k, so we can compute their values one by one, i. e. , A[1], then A[2], then A[3],

Dynamic Programming For example: s t How many shortest multi-bend routes can there be from s to t? M. Pedram

Dynamic Programming We define an array A[1. . 8, 1. . 6] where A[x, y] is the number of routes from s to (x, y): x s 1 1 1 2 3 1 y 1 1 1 A[1, k] = 1 A[j, 1] = 1 A[j, k] = A[j-1, k] + A[j, k-1] 3 1 1 1 t How about if we only allow 1 -bend? M. Pedram

Dynamic Programming in Maze Routing Criteria: Follow Shortest Manhattan Distance Always route towards t with minimum distance s A[1, k] = 1 A[j, 1] = 1 A[j, k] = A[j-1, k] + A[j, k-1] t M. Pedram

Dynamic Programming in Maze Routing Allow detours, i. e. , allow routing away from t (routes do not follow the shortest Manhattan distance detour s t detour M. Pedram If at most 2 detours are allowed, what is the DP for this problem? A[1, k] =? A[j, 1] = ? A[j, k] =?

Mathematical Programming n In mathematical programming, the problem is expressed as an objective function and a set of constraints. The followings are common problems that are solvable in polynomial time: n n n M. Pedram Linear Programming Quadratic Programming Geometric Programming

Mathematical Programming n For Example: 2 1 3 M. Pedram Let A 1, A 2 and A 3 be the area of the three modules What should be their dimensions (w 1, w 2, and w 3) in order to minimize the size of the bounding rectangle?

Mathematical Programming Objective: Minimize W H 2 1 3 M. Pedram Constraints: W w 1 + w 2 W w 3 H A 3/w 3 + A 1/w 1 H A 3/w 3 + A 2/w 2

Linear Programming (LP) n n n Both the objective and constraint functions are linear LP can be solved optimally Many solvers are available that can efficiently solve problems with a large number of variables n M. Pedram http: //www-fp. mcs. anl. gov/otc/Guide/Software. Guide/Categories/linearprog. html

Integer Linear Programming (ILP) n n n M. Pedram ILP is a variant of LP in which all of the variables are integer ILP is NP-complete LP that contains both integer and real variables is called mixed integer linear programming

Stochastic Search n n M. Pedram Local Search Tabu Search Genetic Algorithm Simulated Annealing

Local Search n n M. Pedram Local search is a simple searching algorithm that always move “downhill” in the solution space The notion of neighborhood N(f) of a feasible solution f is used. Given f, N(f) is a set of feasible solutions that are “close” to f based on some criteria

Local Search Algorithm: Initialize a feasible solution f Do G {g | g N(f) and cost(g) < cost(f)} If (G ), assign f as one element in G while (G ) Return f M. Pedram

Local Search n n n M. Pedram It is called first improvement if we always assign f with the first element found with a lower cost in the do-whileloop It is called steepest descent if we always assign f with the element of the lowest cost in the do-while-loop The biggest problem of local search is getting stuck in a local minimum

Tabu Search n n n M. Pedram Tabu search allows “uphill” moves Given a neighborhood set N(f) of a feasible solution f, the principle of tabu search is to move to the cheapest element g N(f) even when cost(g) > cost(f) A Tabu list containing the k last visited solutions is maintained to avoid circular search pattern of length k

Tabu Search Algorithm: Initialize a feasible solution f best f, Q an empty queue Do G {g | g N(f) and g Q} If (G ) Assign f as the cheapest element in G If |Q| < k, enqueue(Q, f); else dequeue(Q), enqueue(Q, f) If cost(best) > cost(f), best f while (G ) and not stop() Return best M. Pedram

Genetic Algorithms (GA) n n n M. Pedram Instead of keeping just one current solution, GA keeps track of a set P of feasible solutions, called population In each iteration, the current population Pi is replaced by the next one Pi+1 To generate a feasible solution h Pi+1, two solutions f and g are selected from Pi and h is generated such that it inherits properties from both parents f and g. This is called crossover

Genetic Algorithms (GA) n n M. Pedram A lower cost solution in a population will have a higher chance of being selected as the parent Mutation may occur to avoid getting stuck in a local minimum

Genetic Algorithms (GA) Algorithm: Initialize P with k feasible solutions Do new. P For (i = 1; i k; i++} Select two low cost solutions from P as f and g h crossover(f, g) Add h to new. P P new. P while not stop() Return the best solution in P M. Pedram

Markov Chain n n Let G(V, E) be a directed graph with V={S 1, S 2, …. . Sn} Let C: V Z be a vertex-weighting function We write Ci for C(Si) Let P: E [0, 1] be an edge-weighting function (notation: Pij=P(Si, Sj) such that: for all Si V n n M. Pedram N(Si) is the set of next states of Si We call (G, P) (finite) time-homogenous Markov chain Pij is called the conditional probability of edge(Si, Sj) Assume Pij>0, if Pij=0, simply delete edge(Si, Sj) from E

Irredundant Markov Chain n n n Stationary distribution n n M. Pedram A Markov chain (G, P)is irreducible if G is strongly connected P=[pij] is the matrix whose <i, j> entry is pij i is the probability of being in state i A state distribution of a Markov Chain (G, P) is stationary if

Markov Chain Example 1 1/3 2 1 3 M. Pedram 1/2 2/3 1/2

Simulated Annealing (SA) n n M. Pedram Simulated annealing is a powerful technique to provide high quality solutions to some difficult combinatorial problems It keeps a variable Temperature (T) which determines the behavior of the annealing process. This variable T is initialized to a very large value at the beginning, and will be gradually decreased (cooled down).

Simulated Annealing Algorithm: Initialize T and a feasible solution f While (T a threshold) • Make a slight modification to f to get g • Check if g is better than f, i. e. , cost(g) cost(f)? • If yes, accept g, i. e. , f g; else, compute p as e-k(cost(g)-cost(f))/T where k is a positive constant, and then, accept g with probability p • Update T M. Pedram

Basic Ingredients of Simulated Annealing n n Solution Space Neighboring Structure Cost Function Annealing Schedule n M. Pedram Moves are selected randomly, and the probability that a move is accepted is proportional to system’s current temp

Simulated Annealing 500 modules This is a floorplanning result obtained by simulated annealing M. Pedram

Floorplanning

Hierarchical Design n Several blocks after partitioning: n Need to: n n Put the blocks together. Design each block. Which step to go first? M. Pedram

Hierarchical Design n n M. Pedram How to put the blocks together without knowing their shapes and the positions of the I/O pins? If we design the blocks first, those blocks may not be able to form a tight packing

Floorplanning The floorplanning problem is to plan the positions and shapes of the modules at the beginning of the design cycle to optimize the circuit performance: n n n M. Pedram chip area total wire length delay of critical path routability others, e. g. , noise, heat dissipation, etc.

Floorplanning v. s. Placement n n Both determines block positions to optimize the circuit performance. Floorplanning: n n Placement: n M. Pedram Details like shapes of blocks, I/O pin positions, etc. are not yet fixed (blocks with flexible shape are called soft blocks) Details like module shapes and I/O pin positions are fixed (blocks with no flexibility in shape are called hard blocks)

Floorplanning Problem n Input: n n n Output: n n Coordinates (xi, yi), width wi and height hi for each block such that hi wi = Ai and ri hi/wi si Objective: n M. Pedram n Blocks with areas A 1, . . . , An Bounds ri and si on the aspect ratio of block Bi To optimize the circuit performance

Bounds on Aspect Ratios If there is no bound on the aspect ratios, we can surely pack very tightly: However we don’t want to layout blocks as long strips, so we require ri hi/wi si for each i M. Pedram

Bounds on Aspect Ratios n n M. Pedram We can also allow several shapes for each block: For hard blocks, the orientations can be changed:

Objective Function A commonly used objective function is a weighted sum of area and wirelength: cost = a. A + b. L where A is the total area of the packing, L is the total wire length, and a and b are constants M. Pedram

Wire length Estimation n Exact wire length of each net is not known until routing is done In floorplanning, even pin positions are not known yet Some possible wire length estimations: n n M. Pedram Center-to-center estimation Half-perimeter estimation

Dead space n Dead space is the space that is wasted: Dead space n n M. Pedram Minimizing area is the same as minimizing dead space Dead space percentage is computed as (A - i. Ai) / A 100%

Mixed Integer Linear Program n A mathematical program such that: n n M. Pedram The objective is a linear function All constraints are linear functions Some variables are real numbers and some are integers, i. e. , “mixed integer” It is almost like a linear program, except that some variables are integers

Problem Formulation n Minimize the packing area: n n Y Need to have constraints so that blocks do not overlap Associate each block Bi with 4 variables: n n M. Pedram Assume that one dimension W is fixed Minimize the other dimension Y xi and yi: coordinates of its lower left corner wi and hi: width and height W

Non-overlapping Constraints n For two non-overlapping blocks Bi and Bj, at least one of the following four linear constraints must be satisfied: hi (xi, yi) M. Pedram Bi wi hj (xj, yj) Bj wj

Integer Variables n n M. Pedram Use integer (0 or 1) variables xij and yij: xij=0 and yij =0 if (1) is true xij=0 and yij =1 if (2) is true xij=1 and yij =0 if (3) is true xij=1 and yij =1 if (4) is true Let W and H be upper bounds on the total width and height. Non-overlapping constraints:

Formulation M. Pedram

Formulation with Hard Blocks n M. Pedram If the blocks can be rotated, use a 0 -1 integer variable zi for each block Bi s. t. zi = 0 if Bi is in the original orientation and zi = 1 if Bi is rotated 90 o

Formulation with Soft Blocks n n M. Pedram If Bi is a soft block, wihi=Ai. But this constraint is quadratic! Linearized by taking the first two terms of the Taylor expression of hi=Ai/wi at wimax (max. width of block Bi) hi =himin+li(wimax-wi) where himin =Ai/wimax and li=Ai/wimax 2

Formulation with Soft Blocks M. Pedram n If Bi is soft and Bj is hard: n If both Bi and Bj are soft:

Solving Linear Program n n Linear Programming (LP) can be solved by classical optimization techniques in polynomial time. Mixed Integer LP (MILP) is NP-Complete. n M. Pedram The run time of the best known algorithm is exponential to the no. of variables and equations

Complexity n For a problem with n blocks, and for the simplest case, i. e. , all blocks are hard: n n M. Pedram 4 n continuous variables (xi, yi, wi, hi) n(n-1) integer variables (xij, yij) 2 n 2 linear constraints Practically, this method can only solve small size problems.

Successive Augmentation n A classical greedy approach to keep the problem size small: repeatedly pick a small subset of blocks to formulate a MILP, solve it together with the previously picked blocks with fixed locations and shapes: Next subset Y M. Pedram Partial Solution