The Halting Problem Can we design a program

The Halting Problem • Can we design a program that, given any other program and its input, tells whether that program will halt when run on that input? • Output of this program, called HALT, when applied to program P and input I, is “halts” if P(I) eventually halts • Output of this program is “never halts” if P(I) never halts • Theorem (Alan Turing 1937): No computer program can solve the halting problem.

Proof by Contradiction • Assume the hypothesis to be proven is false, i. e. there is a computer program that can solve the halting problem • Show that this assumption leads to a contradiction • Do it by creating a program and an input to it that generate this contradiction

Proof by Contradiction • Suppose there was a computer program, called HALT, which solved the halting problem. • We can write a new program that uses HALT and leads to a contradiction: Program Non. Conformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program

Proof by Contradiction Program Non. Conformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program • Does Non. Conformist(Non. Conformist) halt? • Note: It calls HALT(Non. Conformist) • Yes? Means HALT(Non. Conformist) = never halts • No? That means HALT(Non. Conformist) = halts • Contradiction: There exists a program (Non. Conformist) for which the HALT program gives the wrong answer

Undecidability • Halting Problem: Given a computer program P and input “input”: • Output “halts” if P(input) eventually halts • Output “never halts” if P(input) never halts • We’ve shown that the Halting Problem is Undecidable – no computer program can ever solve it, no matter how powerful the computer is.

Undecidability • Many other problems are undecidable, too. • Moreover, the same ideas that we used to prove undecidability can be used to prove a very disturbing statement.

GÖDEL’S INCOMPLETENESS THEOREM • Until Gödel came along, Mathematicians were searching for the “one true logical framework, ” i. e. a framework in which every true statement can be proved. • Their faith was shattered by Gödel. • For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

GÖDEL’S INCOMPLETENESS THEOREM • In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. • “Not every true statement has a proof”

GÖDEL’S INCOMPLETENESS THEOREM • In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. • But wait, if it can’t be proved, how do we know it is true?

GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…

GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… • But that larger logical framework also has a true statement that cannot be proved!

GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… • But that larger logical framework also has a true statement that cannot be proved! • …and so on, and so on… • No matter when we stop, there will always be some true statement that doesn’t have a proof!

GÖDEL’S INCOMPLETENESS THEOREM • Until Gödel came along, Mathematicians were searching for the “one true logical framework, ” i. e. a framework in which every true statement can be proved. • Their faith was shattered by Gödel. • For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

More Undecidable Problems? Post's Correspondence Problem (PCP) • An instance of PCP of size s is a finite set of pairs of strings (gi , hi) [for i = 1. . . s; s>=1] over some alphabet . • A solution is a sequence i 1 i 2. . . in of selections from each set of strings (gi , hi) such that the strings gi 1 gi 2. . . gin and hi 1 hi 2. . . hin formed by concatenation are identical.

Sample PCP • g 1 = • g 2 = • g 3 = • g 4 = aba bbab baaa a h 1 = h 2 = h 3 = h 4 = abaa abab a bb So, 1, 3, 1, 2 would correspond to • aba baaa aba bbab from g’s • abaa abab from h’s Not a solution!

Another Sample PCP • g 1 = • g 2 = • g 3 = • g 4 = aba bbab baaa a h 1 = h 2 = h 3 = h 4 = abaa abab a bb 1, 4, 2, 1, 3 corresponds to • aba a bbab aba baaa from g • abaa bb abaa a from h Solution!

PCP is undecidable? • PCP shown to be undecidable by Post in 1946. • What about PCP with limited-size inputs • PCP with size 2 has been proved decidable. • PCP with size 7 has been proved undecidable • The decidablility of problems with size between 3 and 6 is still pending.

What Computers Can’t Do In Your Lifetime • We’ve now seen examples of problems that computers can’t solve, even if computers have unlimited speed and unlimited time. • Are there more real world problems (eg that arise in business, science, … ) that can be solved but take far too much time to be solved in practice?

“Search” Problems • We’ll focus on what we’ll call “search problems”. (In Computer Science terms, we’ll be talking about what are termed NP problems) • Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. • Recognizing a good solution is easy • Problem is finding it

Example: Coloring

Example: Coloring with 3 colors • Suppose we are given a collection of circles (nodes). • Some circles connect to others by edges, forming a graph • Rule: No two connected circles can have the same color. • You only have three colors (Green, Red, Yellow) • Is there a valid coloring? • Note: easy to check validity. Hard to find?

Search Problems: More Precise Definition • Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. • A bit more precisely, we require that there is a small circuit that can quickly check the validity of a solution. • For coloring, it would just be a circuit that checks that for every pair of connected circles, the colors are different.

Another Example: Traveling Salesperson Problem • A saleswoman wants to visit n different cities. • She knows the costs associated with flights between each city. • Can she visit all the cities spending less than $B in total? • Note: Easy to check that a given flight plan visits all the cities and costs less than $B. • Seems hard to find the flight plan …

Search Problems… • Unfortunately, we don’t know how to solve these and many other search problems with a computer in our lifetimes for large inputs. • Large graphs, large number of cities • For many years, computer scientists wondered which search problems could be solved, and which couldn’t. • But just because computer scientists couldn’t solve the Coloring problem for 40 years doesn’t mean it is impossible, right? • Proof of Fermat’s Last Theorem took over 300 years

Search Problems… • Can we prove that it is impossible to solve them quickly for large inputs? • Unfortunately, not yet • But now we are much more confident that they really are impossible to solve quickly • How do we make progress toward determining an answer?

Breakthrough: Reduction • A surprise: • In the early 70’s, Cook, Levin, and Karp showed us that if we can solve the Coloring problem quickly, then we can solve ALL search problems quickly! • But general search problems are defined in terms of circuits (that can validate their solutions), not colors… • Need to map circuits to graph coloring problems • Then we can map the circuit for any search problem to the corr. graph coloring problem

Coloring and Circuits • Suppose we think of : • Green as meaning True. • Red as meaning False • Yellow as meaning nothing

What is the circuit for this graph? Input T X So this is a graph for a NOT circuit F Valid coloring for this graph leads to a NOT gate Output NOT X

T F Y X Output X Y F F F T T

T F Y X F T Output X Y F F F T T

T F Y X Output X Y F F F T T

T F Y X Output X Y F F F T T

T F Y X Output X Y F F F T T

T F Y X Output X Y F F F T T T

T F Y X Output X Y F F T T T F T T

T F Y X = Output OR X Y OR F F T T T F T T

Coloring ! • And you thought coloring was for kids. . • In fact, we can encode any circuit into a collection of connected circles (graph) waiting to be colored. • Any valid coloring for the graph conforms to the circuit (which verifies a search problem) • Thus, we can efficiently reduce any search problem to Coloring. • If we can solve Coloring quickly, we can solve any search problem quickly!

Many more • Traveling Salesperson Problem is also as “hard” as any search problem. • Search Problems with this property are called complete problems • Although we don’t know how to prove that they are hard, we know that if find a way to solve one of them quickly, we can solve all search problems quickly! • Gives us more confidence that they really are hard.

Recap: Hard search problems • A hard search problem is a problem where • It is hard to find a solution • It is easy to check possible solutions for validity • A complete search problem is a problem that is as hard as any search problem • Search problem is believed to be hard because • No one found a way to solve any of the complete search problems quickly

What does “Quickly” mean? • In time polynomial in the size of the input • E. g. if one could solve an n-city TSP in time – n 2+2 n+5 – or even n 2000+7 n 1999+…+8 n 2+5 n+6 – Then these are a Polynomial-time algorithms (quick) • But if one can only solve it in time: – 2 n – or 2 n+6 n-1+…+3 – Then these are Exponential-time algorithms (slow)

2 n versus n 2 • n = 10 – n 2 = 100 – 2 n = 1024 • n = 100 – n 2 = 10, 000 – 2 n = (1024)10 > 1 trillion * 1 million Hence we call exponential-time algorithms slow

Classes of search problems • In computer-science terminology: • NP = All Search Problems • P = Problems we can solve quickly • We believe that P NP, i. e. not every search problem can be solved quickly on a computer.

NP-Complete Problems • Coloring is complete • In particular, we can reduce solving any search problem to finding a valid coloring for some collection of circles! • So, if we could solve Coloring quickly, then P = NP • That’s why we believe Coloring can’t be solved quickly by any computer. • We call such problems NP-Complete.