Approximation algorithms Algorithms and Networks 20152016 Hans L

Approximation algorithms Algorithms and Networks 2015/2016 Hans L. Bodlaender Johan M. M. van Rooij

What to do if a problem is NP-complete? g We have already seen many options to deal with NP- complete problems. g FTP, special cases, exact exponential-time algorithms, heuristics. g In other courses: local search, ILP, constraint programming, … g Approximation algorithms are one of these options. g An approximation algorithm is a heuristic with a performance guarantee. g We consider polynomial-time approximation algorithms. g Non-optimal solutions, but with some performance guarantee compared to the optimal solution. g Also useful as a starting point for other approaches: g Local search, branch and bound. 2

Different forms of approximation algorithms (outline of lecture) Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. Ø |OPT – ALG| · c 2. APX: Constant-factor approximation. Ø Approximation ratio: ALG/OPT · c for minimisation problems. Ø Approximation ratio: OPT/ALG · c for maximisation problems. 3. f(n)-APX : Approximation by a factor of f(n). Ø f(n) depends only one the size of the input. 4. PTAS: Polynomial-time approximation scheme. Ø Approximation ratio 1+² for any ² > 0, while the algorithm runs in polynomial time for any fixed ². 5. FPTAS: Fully polynomial-time approximation scheme. Ø Approximation ratio 1+² for any ² > 0, while the algorithm runs in time polynomial in n and 1/². 3

Absolute constant difference g Algorithms that run in worst-case polynomial time, with: g |OPT – ALG| · c g Example: Planar graph colouring. g Algorithm that tests 1 and 2 -colourability and if no solution found always outputs 4. g Algorithm has an error of at most 1. g These examples are rare. g Not so difficult to show: g TSP cannot be approximated by a constant factor unless P NP. 4

Constant-factor approximation g Approximation ratio: g ALG/OPT · c for minimisation problems. g OPT/ALG · c for maximisation problems. g Ratio always bigger or equal to 1. g Class of problems with a constant factor approximation algorithm: APX. g Notions of APX-completeness also exist. g Examples of constant-factor approximation algorithms from earlier lectures: g 2 -approximations: MST based algorithms for metric TSP. (TSP with triangle inequality). g 1. 5 -approximation: Christofides algorithm for metric TSP. 5

Approximation for vertex cover Approximation algorithm for vertex cover: 1. Let E’ = E, C = ; 2. While E’ ; 1. Let {u, v} be any edge from E’ 2. C = C [ {u, v} 3. Remove every edge incident to u of v from E’. 3. Return C g Runs in polynomial time. g Returns a vertex cover. g How good is this vertex cover? 6

2 -approximation for vertex cover Theorem: g The algorithm on the previous slide is a 2 -approximation. Proof: g Let A be the set of edges which endpoints we picked. g OPT ¸ |A|, because every edge in A must be covered. g ALG = 2|A| · 2 OPT, hence ALG/OPT · 2. 7

Approximation by a factor of f(n) g Approximation ratio of f(n). g Approximation ratio depends on the size of the input (and can be very bad) Set Cover g Given: finite set U and a family F of subsets of U. g Question: Does there exists a subset C of F of size at most k such that [S 2 C S = U? Greedy algorithm: g While U ; : g Select S 2 F that maximises |S Å U| g Add S to C and let U : = U S 8

An (ln(n)+1)-approximation algorithm for se cover Theorem: g The greedy set cover algorithm is an (ln(n)+1)-approximation. Proof: g The algorithm runs in polynomial time and returns a set cover. g Let Si be the ith set from F picked by the algorithm. g We assign cost cx to elements from x: g Let x be firstly covered by set Si while running the algorithm. g And, let Xi = Si (S 1 [ S 2 [. . . [ Si-1) g Define: cx = 1 / |Xi|. g Now we have: g. . . 9

Proof of the (ln(n)+1)-approximation algorithm for set cover g The nth Harmonic number H(n) is defined as: g From mathematics (bounding using integrals): g On the next two slides, we will prove that for any S 2 F: g From this we derive our approximation ratio: g Where C* is the optimal set cover. 10

Bounding the cost of any set S g What remains is to prove that for any S 2 F: g Remind: g If x firstly covered by Si, then cx = 1 / |Xi| g Where: Xi = Si (S 1 [ S 2 [. . . [ Si-1) g For any S 2 F, let ui be the number of uncovered items in S when the algorithm selects Si. g Take the smallest value k s. t. uk = 0. g With the last equality due to the selection by the algorithm. g Then. . . 11

Bounding the cost of any set S g To prove: g Last slide: g Thus: g Remind: g Q. E. D. 12

PTAS g A Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a real value e>0. g For each fixed e>0, the algorithm g Uses polynomial time g Is an (1+e)-approximation algorithm 13

FPTAS g A Fully Polynomial Time Approximation Scheme is an algorithm, that gets two inputs: the “usual” input X, and a real value e>0, and g For each fixed e>0, the algorithm g Is an (1+e)-approximation algorithm g The algorithm uses time that is polynomial in the size of X and 1/e. 14

Example: Knapsack g Given: n items each with a positive integer weight w(i) and integer value v(i) (1 ≤ i ≤ n), and an integer B. g Question: select items with total weight at most B such that the total value is as large as possible. 15

Dynamic Programming for Knapsack g Let P be the maximum value of any item. g We can solve the problem in O(n 2 P) time with dynamic programming: g Tabulate M(i, Q): minimum total weight of a subset of items 1, …, i with total value Q for Q at most n. P g M(0, 0) = 0 g M(0, Q) = ∞, if Q>0 g M(i+1, Q) = min{ M(i, Q), M(i, Q-v(i+1)) + w(i+1) } g This algorithm is clearly correct and runs in the given time. 16

Scaling for Knapsack g Take input for knapsack g Throw away all items that have weight larger than B (they g g 17 are never used) Let c be some constant Build new input: do not change weights, but set new values v’(i) = ë v(i) / c û Solve scaled instance with DP optimally Output this solution: approximates solution for original instance

The Question is…. g How do we set c, such that g Approximation ratio good enough g Polynomial time 18

Approximation ratio g Consider optimal solution Y for original problem, value OPT g Value in scaled instance: at least OPT/c – n g At most n items, for each v(i)/c - ë v(i)/c û <1 g So, DP finds a solution of value at least OPT/c –n for scaled problem g So, value of approximate solution for original instance is at least c*(OPT/c –n) = OPT - nc 19

Setting c g Set c = e. P/(2 n) g This is an FPTAS g Running time: g Largest value of an item is at most P/c = P / (e. P/(2 n)) = 2 n/e. g Running time is O(n 2 * 2 n/e) = O(n 3/e) g Approximation: … next 20

e-approximation g Note that each item is a solution (we removed items with g g 21 weight more than B). So OPT ≥ P. Algorithm gives solution of value at least: OPT – nc = OPT – n(e P / (2 n) ) = OPT – e/2 P OPT / (OPT – e/2 P)≤ OPT / (OPT – e/2 OPT) = 1/(1 -e/2) ≤ 1+e QED.

2 -approximation for minimum weight vertex cover g Minimum weight vertex cover: g Vertex cover where each vertex has a weight. g We look for the minimum weight vertex cover. g 2 -approximation for vertex cover no longer works. g We may select very heavy vertices using that algorithm. g Consider the following ILP: g It’s LP relaxation is ILP with the last constraint replaced by: g 0 · x(v) · 1 22

2 -approximation algorithm for minimum weight vertex cover Algorithm: g Compute the optimal solution to the LP relaxation. g Output all v with x(v) ¸ ½. g Algorithm runs in polynomial time. g Linear programming can be solved in polynomial time. g Not by the simplex algorithm!! g Ellipsoid method / interior point methods. g Algorithm returns a vertex cover. g For every edge, sum of incident vertices at least 1. g Hence at least one of the vertex variables at least ½. 23

Proof of 2 -approximation algorithm for minimum weight vertex cover g Let z* be the solution to the LP. g Because any vertex cover is a solution to the LP we have: g Also, we can bound ALG in terms of z*: g Hence: g QED 24

Conclusion Qualities of polynomial-time approximation algorithms: 1. Absolute constant difference. Ø Planar graph colouring. 2. APX: Constant-factor approximation. Ø TSP with triangle inequality. Ø Vertex cover. 3. f(n)-APX : Approximation by a factor of f(n). Ø Set cover. 4. PTAS: Polynomial-time approximation scheme. 5. FPTAS: Fully polynomial-time approximation scheme. Ø Scaling for Knapsack. 25