Approximation Algorithms Fabrizio Grandoni IDSIA fabrizioidsia ch About

Approximation Algorithms Fabrizio Grandoni IDSIA fabrizio@idsia. ch

About these Lectures The material of these lectures is (mostly) taken from: • Vazirani V. V. , Approximation Algorithms, Springer • Williamson D. P. and Shmoys D. B. , The Design of Approximation Algorithms, Cambridge University Press • Lau L. C. , Ravi R. , and Singh M. , Iterative Methods in Combinatorial Optimization, Cambridge University Press • “Exe” indicates exercises that we will discuss in class • “Hmw” indicates homeworks that you have to solve on your own, and whose written solutions you have to email me within proper deadlines. Your final mark will be based on homeworks. You can ask me questions, and you can cooperate with your class-mates (but too similar solutions will be penalized. . . )

Algorithms and Complexity

Problems and Algorithms • A (computational) problem is a binary relation P Ix. O between a set I of input instances and set O of output instances, where there exists at least one (i, o) P for each i I • An algorithm for P is a procedure that, given i I, computes with a finite (expected) number of elementary operations (on a proper machine. . . ) an o O such that (i, o) P • We often compare different algorithms for the same problem in terms of the worst-case (i. e. maximum) number of elementary operations T(n) needed to solve a problem of size n • In order to simplify the worst-case analysis, we simply show that T(n)=O(f(n)) where O-notation suppresses constant factors and considers only large enough n (asymptotic analysis) • An algorithm is efficient if T(n)=O(nc) for some constant c>0

NP-Completeness • A decision problems is a problem where the output is true or false • The class NP is the subset of decision problems such that there exists a polynomial-time procedure so that the answer is true iff there exists a polynomial-size certificate that, given as input to the procedure, shows that the answer is correct • The class P is the subset of NP problems that can be solved in polynomial time • In the class NP there exists a special class of NP-complete problems: an efficient algorithm for any NP-complete problem would imply an efficient algorithm for all problems in NP • If any such algorithm exists, then P=NP. Most researchers believe that P≠NP • Therefore, if a problem is NP-complete, there is not much hope to solve it efficiently. . .

NP-Hardness and Optimization • A problem is NP-hard if an efficient algorithm for it implies P=NP • If a problem is NP-hard, there is not much hope that we can solve it efficiently. . . Def: an optimization problem is a problem P where we are given a cost function c: O Q and a set of feasible solutions F(i) for each i I. A valid output for i is any f F(i) that maximizes/minimizes c(f) In these lectures we will focus on NP-hard optimization problems

Approximation Algorithms

Approximation Algorithms Prb: How can we deal with NP-hard optimization problems? Idea: maybe we can compute efficiently a feasible solution whose cost is “close” to the optimal cost! Def: An a-approximation algorithm APX for an optimization problem P, a: N R+, is a polynomial-time algorithm that computes a feasible solution APX(i) on every input i with optimal solution OPT(i) such that: • a(n) ≥ maxi I, |i|=n{c(APX(i))/c(OPT(i))} for a minimization problem • a(n) ≥ maxi I, |i|=n{c(OPT(i))/c(APX(i))} for a maximization problem • Our goal is to minimize a(n)≥ 1 (approximation factor/ratio) • Sometimes we replace |i| with some natural size parameter (e. g. number n of nodes of a graph) • In some cases b: =supn a(n)=O(1), and we say that the approximation factor is b

Some Known Approximation Factors Problem Best-Known Approximation factor Knapsack 1+ε, for any constant ε>0 Minimum Vertex Cover 2 Minimum Set Cover O(log n), n number of elements in the universe Maximum Independent Set O(n), n number of nodes in the graph Polynomial-Time Approximation Scheme (PTAS)

Hardness of Approximation Prb: Is it always possible to get a PTAS like for Knapsack? Sometimes an a-apx implies that NP-complete problems can be solved in poly-time (i. e. , even approximating the solution is NP-hard!) Problem Approximation factor Knapsack 1+ε Minimum vertex cover 2 Minimum set cover O(log n) Ω(log n) Maximum independent set O(n) Ω(n 1 -ε) for any constant ε>0 Travelling salesman problem Inapproximability 1. 36 no finite apx There is a gap

Examples of Approximation Algorithms

How to Design an Approximation Algorithm? Any approximation algorithm involves 2 main ingredients: • Designing a good algorithm • Finding good lower/upper bounds on the cost of the optimum solution for a minimization/maximization problem approximation factor minimization problem LB OPT APX Cost UB Cost approximation factor maximization problem APX OPT

Steiner Tree

Steiner Tree

Steiner Tree Def (Steiner tree): given an undirected graph G=(V, E), with edge costs c(e)≥ 0, and a collection of terminals R V, find a tree T (Steiner tree) containing R of minimum cost c(T): = e Tc(e) 1 6 5 2 1 1 Steiner node 2 3 non-terminal 2 4 3 7 Rem: The special case R=V is the (minimum) spanning tree problem, which can be solved in polynomial time (while the general case is NP-hard)

The Spanning Tree Heuristic Idea: compute a spanning tree among the terminals (terminal spanning tree), but in the metric closure of the graph 1. Compute a minimum spanning tree T of Gc[R], where Gc is the metric closure of G 2. Replace each edge of T with the corresponding shortest path in G, remove any cycle, and return the resulting tree APX 1 6 5 2 1 1 3 2 2 4 3 7 2 3 2 5 3 2

Eulerian Tours Rem: Euler, solving the problem of the Seven Bridges of Königsberg, defined the family of Eulerian (multi-)graphs. This is one of the first results in Graph Theory Def: A Eulerian tour of a undirected (multi)graph G is a circuit that includes all the edges of G exactly once Thr [Euler 1753]: A (multi-)graph is Eulerian, i. e. , contains an Eulerian tour, iff it is connected and all nodes have even degree

The Spanning Tree Heuristic Thr: the minimum spanning tree heuristic is a 2 -apx for Steiner tree 1. 2. Compute a minimum spanning tree T of Gc[R], where Gc is the metric closure of G Replace each edge of T with the corresponding shortest path in G, remove any cycle, and return the resulting tree APX • Take OPT, duplicate its edges and compute an Euler tour C • Shortcut duplicated nodes and all Steiner nodes in C, then remove one edge: let T be the resulting path (spanning all terminals) • c(APX) ≤ c(T) ≤ c(C) = 2 c(OPT)

The Spanning Tree Heuristic Thr: the minimum spanning tree heuristic is a 2 -apx for Steiner tree 1. 2. Compute a minimum spanning tree T of Gc[R], where Gc is the metric closure of G Replace each edge of T with the corresponding shortest path in G, remove any cycle, and return the resulting tree APX Hmw: show an approximation lower bound of 2 for the spanning tree heuristic

Bin Packing

Bin Packing ?

Bin Packing Def (bin packing): Given a multi-set of items a 1, . . . , an (0, 1], partition them in the minimum number of subsets (bins) so that the sum of the values in each bin is at most 1

First Fit Idea: pack items one by one, opening a new bin only if needed 1. Open one bin 2. For i=1, . . . , n a. Place ai in the first open bin where it fits, if any b. Otherwise open a new bin and place ai there

First Fit Thr: First Fit is a 2 -apx for Bin Packing 1. 2. Open one bin For i=1, . . . , n a. Place ai in the first open bin where it fits, if any b. Otherwise open a new bin and place ai there • Let OPT and APX be the number of bins in the optimal and approximate solution, respectively • At least APX-1 bins must be more than half-full: otherwise, there would be 2 bins whose total items size is at most one, and First Fit would have opened ≤APX-1 bins • Then iai > (APX-1)/2 • Clearly OPT ≥ iai (Exe: Why? ) • Altogether APX < 2 iai+1 ≤ 2 OPT+1, i. e. APX ≤ 2 OPT Rem: in [Dosa, Sgall’ 13] it is shown that the approximation factor of First Fit is 17/11, and this is tight

Hardness of Bin Packing Thr: There is no 3/2 -ε approximation for Bin Packing unless P=NP Hmw: prove the above claim Hint: show that such apx would imply a poly-time algorithm to solve the following NP-complete Partition problem Def (partition): Given a multiset of numbers a 1, . . . , an≥ 0 whose sum is A, determine whethere exists a sub-multiset of numbers of sum exactly A/2