CSCI 256 Data Structures and Algorithm Analysis Lecture

CSCI 256 Data Structures and Algorithm Analysis Lecture 25 Some slides by Kevin Wayne copyright 2005, Pearson Addison Wesley all rights reserved, and some by Iker Gondra

The "First" NP-Complete Problem • Theorem: CIRCUIT-SAT is NP-complete. [Cook 1971, Levin 1973]. Pf: (sketch) Need to take any NP problem X and show: X p CIRCUIT-SAT First we remark: – Any algorithm that takes a fixed number of bits n as input and produces a yes/no answer can be represented by a circuit of AND, OR and NOT gates which is satisfiable iff answer is yes Moreover, if algorithm takes poly-time, then circuit is of poly-size (we won’t go into the transformation from algorithm to circuit – but any algorithm implemented on physical computers can be reduced to their operations on the /, V, and ¬ gates; need to fix the number of input bits as a circuit is structurally hardcoded)

The "First" NP-Complete Problem – Let X be a problem in NP. It has a poly-time certifier C(s, t). To determine whether s is in X, need to know if there exists a certificate t of length p(|s|) such that C(s, t) = yes – View C(s, t) as an algorithm on n =(|s| + p(|s|) bits (input string s, certificate t) and convert it into a polysize circuit K • first |s| bits are hard-coded with s • remaining p(|s|) bits represent bits of t – Circuit K is satisfiable iff C(s, t) = yes

Establishing NP-Completeness • Remark: Once we establish first "natural" NP-complete problem, others fall like dominoes • Recipe to establish NP-completeness of problem Y – Step 1. Show that Y is in NP – Step 2. Choose an NP-complete problem X – Step 3. Prove that X p Y • Justification: If X is an NP-complete problem, and Y is a problem in NP with the property that X P Y then Y is NP -complete • Pf: Let W be any problem in NP. Then W P X P Y – By transitivity, W P Y – Hence Y is NP-complete

Worked Example: Does graph G have an independent set of size 2? ? We already showed that this problem is in NP – We show an instance of the problem can be solved by constructing an equivalent instance of CIRCUIT-SAT A graph G of n nodes can be specified by (n Choose 2) bits -- for each pair of nodes there is a bit specifying if there is an edge between them; thus s is of size (n Choose 2) The certificate t can be specified by n bits -- for each node we need a bit saying whether this node belongs to the independent set. The certifier needs to check (1) at least 2 of the bits of t are set to 1 (represents 2 nodes) and (2) no two bits in t are both set to 1 if they form the ends of an edge - this is determined by the bits in s – (represents the independence of the two nodes)

Worked Example • Ex: Construction below creates a circuit K whose inputs can be set so that K outputs true iff graph G has an independent set of size 2 independent set? both endpoints of some edge have been chosen? independent set of size 2? u v w G = (V, E), n = 3 set of size 2? u-v u-w v-w u v w 1 0 1 ? ? ? hard-coded inputs (graph description) n inputs (nodes in independent set)

Worked Example • More explanation: • RHS needs to find values of u, v and w so that at least 2 of them are set to 1 – i. e. , it checks that at least 2 nodes are selected; • LHS checks that we haven’t selected both ends of any edge • Before the negation symbol, if there are 2 independent nodes the formula evaluates to false. • Solution: u = 1, v = 0, w = 1.

• Indeed we can conclude that • Independent Set p CIRCUIT-SAT • But this doesn’t tell us about the NPCompleteness of Independent Set

Recall: Establishing NP-Completeness • Remark: Once we establish first "natural" NP-complete problem, others fall like dominoes • Recipe to establish NP-completeness of problem Y – Step 1. Show that Y is in NP – Step 2. Choose an NP-complete problem X – Step 3. Prove that X p Y • Justification: If X is an NP-complete problem, and Y is a problem in NP with the property that X P Y then Y is NP -complete • Pf: Let W be any problem in NP. Then W P X P Y – By transitivity, W P Y – Hence Y is NP-complete

Prove: 3 -SAT is NP-Complete What do we have to do? (1) Show 3 -SAT is in NP (2) Show for any X in NP, X P 3 -SAT We know (1), that 3 -SAT is in NP already, as we can verify in polynomial time that a proposed truth assignment satisfies a set of clauses To establish (2) we will show that CIRCUIT-SAT P 3 -SAT; then we can conclude (see slide 4) that for any X in NP, X P 3 -SAT So summarizing here is the proof:

3 -SAT is NP-Complete • Theorem: 3 -SAT is NP-complete • Pf: Suffices to show that CIRCUIT-SAT P 3 -SAT since 3 -SAT is in NP and CIRCUIT-SAT is NP-complete. • Given an arbitrary instance of CIRCUIT-SAT, we will first construct an equivalent instance of SAT in which each clause has at most 3 variables and then construct and equivalent instance in which each clause has exactly 3 variables --- thus getting an instance of 3 -SAT

3 -SAT is NP-Complete: prove: CIRCUIT-SAT P 3 -SAT Let K be a circuit – Create a 3 -SAT variable xi for each node v of the circuit to encode the value that the circuit holds at that node – below the circuit has 6 nodes so there are 6 variables – Encode requirement that circuit computes correct values at each node with clauses of the SAT problem * • x 2 = x 3 add 2 clauses: • x 1 = x 4 x 5 add 3 clauses: • x 0 = x 1 x 2 add 3 clauses: output x 0 – Hard-coded input values and output value • x 5 = 0 add 1 clause: • x 0 = 1 add 1 clause: * eg: (x 2 = 1 iff – x 3 = 1) iff (x 2 v x 3) / (- x 2 v –x 3) = 1 x 1 x 2 x 5 0 ? x 4 ? x 3

K satisfiable iff the SAT instance is satisfiable • Can show that if the given circuit K is satisfiable then a satisfying assignment to the circuit input nodes can be propagated to create values at all nodes of K which satisfies the SAT instance we created; also if the SAT instance we created from K is satisfiable, using the values which satisfy of the variables the SAT instance as K’s input, will force K to evaluate to 1 (because the SAT clauses ensure that the values assigned to all nodes of K are the same the circuit computes for these nodes). • Final step: turn clauses of length < 3 into clauses of equivalent truth values of length exactly 3 (see text) yielding a instance of 3 -SAT • Thus CIRCUIT-SAT P 3 -SAT Since 3 -SAT is in NP by transitivity we can conclude for any NP problem X, X P 3 -SAT

Independent Set is NP-Complete • Indeed we can show that we can transform an instance of 3 -SAT into an instance of Independent Set (see text page 460 -461) • Thus there is a poly reduction from 3 -SAT to Independent set: 3 -SAT p Independent Set • Hence Independent Set is NP-Complete • What does this tell you about the Vertex Cover problem? ? ? – Recall we showed: Independent Set p Vertex Cover

Hint on assignment 7!! • Problems 2, 3 and 6: each involves a reduction of either Vertex Cover or Independent Set to the problem!! • Recall you need to give the transformation of the known problem (Vertex Cover or Independent Set) to the assignment problem and show the “if and only if relationship” of the answers to the known problem and the problem on the assignment.

NP-Completeness • Observation: All problems below are NP-complete and polynomial reduce to one another! by definition of NP-completeness CIRCUIT-SAT 3 -SAT to ces SET u d re ENT D AT 3 -S EPEN D N I INDEPENDENT SET DIR-HAM-CYCLE GRAPH 3 -COLOR SUBSET-SUM VERTEX COVER HAM-CYCLE PLANAR 3 -COLOR SCHEDULING SET COVER TSP

Some NP-Complete Problems • Six basic genres of NP-complete problems and paradigmatic examples – Packing problems: SET-PACKING, INDEPENDENT SET – Covering problems: SET-COVER, VERTEX-COVER – Constraint satisfaction problems: SAT, 3 -SAT – Sequencing problems: HAMILTONIAN-CYCLE, TSP – Partitioning problems: 3 D-MATCHING 3 -COLOR – Numerical problems: SUBSET-SUM, KNAPSACK • Practice: Most NP problems are either known to be in P or are NP-complete

Extent and Impact of NP-Completeness • Extent of NP-completeness [Papadimitriou 1995] – Prime intellectual export of CS to other disciplines – 6, 000 citations per year (title, abstract, keywords) • more than "compiler", "operating system", "database" – Broad applicability and classification power – "Captures vast domains of computational, scientific, mathematical endeavors, and seems to roughly delimit what mathematicians and scientists had been aspiring to compute feasibly. “ • NP-completeness can guide scientific inquiry

More Hard Computational Problems • Aerospace engineering: optimal mesh partitioning for finite elements • Biology: protein folding Chemical engineering: heat exchanger network synthesis • Civil engineering: equilibrium of urban traffic flow • Economics: computation of arbitrage in financial markets with friction • Electrical engineering: VLSI layout • Environmental engineering: optimal placement of contaminant sensors • Financial engineering: find minimum risk portfolio of given return • Game theory: find Nash equilibrium that maximizes social welfare • Genomics: phylogeny reconstruction • Mechanical engineering: structure of turbulence in sheared flows • Medicine: reconstructing 3 -D shape from biplane angiocardiogram • Operations research: optimal resource allocation • Physics: partition function of 3 -D Ising model in statistical mechanics • Politics: Shapley-Shubik voting power Pop culture: Minesweeper consistency. • Statistics: optimal experimental design

Traveling Salesperson Problem • TSP: Given a set of n cities and a pairwise distance function d(u, v), is there a tour of length D? All 13, 509 cities in US with a population of at least 500 Reference: http: //www. tsp. gatech. edu

Traveling Salesperson Problem • TSP: given a set of n cities and a pairwise distance function d(u, v), is there a tour of length D? • HAM-CYCLE: given a graph G = (V, E), does there exists a simple cycle that contains every node in V? • Claim: HAM-CYCLE P TSP • Pf: – Given instance G = (V, E) of HAM-CYCLE, |V| = n, create n cities with distance function – TSP instance has tour of length n iff G has Hamiltonian cycle

Travelling Salesman problem • It can be shown that the HAM-CYCLE problem is NP-complete; using the above we can conclude that the TSP is NP-complete

Map Coloring • Given a map, can it be colored using 3 colors so that no adjacent regions have the same color? YES instance

Graph Coloring • Each region is represented by a node in a graph; if 2 regions have a common boundary represent this by an edge between them. So we wish to assign colors to the nodes so that no two nodes have the same color if there is an edge between them

Graph Coloring • We seek to assign a color to each node of a graph G so that if (u, v) is an edge, then u and v are assigned different colors; and the goal is to do this while using a small set of colors • A k-coloring of G is a function f: V {1, 2, …, k} so that for every edge (u, v), we have f(u) f(v). If G has a k-coloring, we say that it is a kcolorable graph – When is a graph 2 -colorable? • Given a graph G and a bound k, does G have a k-coloring?

Graph Coloring - complexity • A graph is 2 colorable iff it is bipartite – so not so hard to determine if a graph is 2 colorable ( recall: O(m+n) ) • However the case of 3 colors is already a very hard problem – we can show it is NP-complete! • Interesting point: the 4 color conjecture for maps in the plane – outstanding problem for over a century; resolved in 1976 by Appel and Haken; structure of the proof was induction on the number of regions but the induction step involved nearly 2000 complicated cases and had to be carried out by a computer; the problem of finding a reasonably short, human readable proof still remains open

Applications: Scheduling • Example: schedule final exams and, being very considerate, avoid having a student do more than one exam a day • What is the least number of days you need to schedule all the exams? (Students: work this by hand! c 1 c 2 c 3 c 4 c 5 c 6 c 7 c 1 * c 2 * c 3 * c 4 * * * * * c 5 * * c 6 * c 7 * * * * * courses 7 and 2 have at least one student in common

Scheduling • n processes or “jobs” (exams) on a system that can run multiple jobs concurrently but certain pairs of jobs cannot be scheduled at the same time because they both need a certain resource (student); over next k time steps (exam days) schedule each process to run in at least one time step; • Graph G: node represents process, edge represents conflict, k-coloring represents a conflict free schedule- all nodes colored j can be scheduled in time j

Applications: Register Allocation • Assign program variables to machine registers so that no more than k registers are used and no two program variables that are needed at the same time are assigned to the same register – Interference graph: Nodes are program variables names, edge between u and v if there exists an operation where both u and v are "live" at the same time – Observation: [Chaitin 1982] Can solve register allocation problem iff interference graph is k-colorable

3 -Colorability • 3 -COLOR: Given an undirected graph G does there exists a way to color the nodes red, green, and blue so that no adjacent nodes have the same color? yes instance

3 -Colorability • Claim: 3 -SAT P 3 -COLOR • Pf: Given 3 -SAT instance , we construct an instance of 3 -COLOR that is 3 -colorable iff is satisfiable • Construction – For each literal, create a node – Create 3 new nodes T, F, B; connect them in a triangle, and connect each literal to B – Connect each literal to its negation – For each clause, add gadget of 6 nodes and 13 edges as in next slide true false T F B base

3 -Colorability true false T F B 6 -node gadget base