CMSC 28100 Third Lecture Undecidability Are there things

CMSC 28100 –Third Lecture Undecidability

Are there things we cannot compute? • Such questions are inherently hard. How do you prove something cannot be done? • If you propose a method, and show it cannot work, you only proved that particular method does not work. • We need to show that there cannot be ANY method! • We will proceed to do that. • We can do that, because we have a precise notion of what can be computed.

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable Technical remarks: we use <O> to denote an encoding of the object O. So <O> is simply a string of characters. We need this for the same reason we needed encodings to prove the existence of the Universal Turing Machine. We also need to encode pairs of objects as a single object—but this is easy – concatenate the strings…

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable Proof: By contradiction. Assume there is a Tm that computes the function H(<M, w>)= accept if M, accepts x; reject if M rejects x Now, since H is computed by a Tm, we can use it as a subroutine. So th following function can also be computed by a Tm: D(<M>)= if [H(<M, <M>>) = accept] then reject; if [H(<M, <M>>) = reject] then accept

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable Proof: Assume there is a Tm that computes the function H(<M, w>)= accept if M, accepts x; reject if M rejects x D(<M>)= if [H(<M, <M>>) = accept] then reject; if [H(<M, <M>>) = reject] then accept Which means, in English D(<M>)= accept if M does not accept <M> = reject if M accepts <M>

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable Proof: Assume there is a Tm that computes the function H(<M, w>)= accept if M, accepts x; reject if M rejects x Then exist a Tm-computable D, such that D(<M>)= accept if M does not accept <M> = reject if M accepts <M>. What happens with D(<D>)? ? ?

Are there things we cannot compute? • Theorem Let ATM be the language ATM ={ <M, w> : M is a Tm, M accepts w. } ATM is not decidable Proof: Assume there is a Tm that computes the function H(<M, w>)= accept if M accepts w; reject if M rejects w Then exists a Tm-computable D, such that D(<M>)= accept if M does not accept <M> = reject if M accepts <M>. What happens with D(<D>)? ? ? D(<D>)= accept if D does not accept <D> = reject if D accepts <D>.

Are there things we cannot compute? • YES!. . and there a zillions of such functions! Recipe to prove f not computable: Assume it is computable. Show that the assumption allows you to decide ATM.

Other Undecidable Problems • I stated that there are zillions of problems that are not decidable. • Remember that ATM ={<M, w> M accepts w} is not decidable. • Note that ATM ={<M, w> M accepts w} is trivially recognizable by a Tm: The Universal Turing Machine simulates M running on input w, and accepts iff M accepts w. Lemma: L and co-L are both recognizable iff L is decidable. (co-L is the set of strings not in L)

Other Undecidable Problems • Lemma: L and co-L are both recognizable iff L is decidable • Proof: A decider for L accepts L. Exchanging qaccept and qreject in this Tm gives us a Tm that accepts co-L. For the other direction, let M 1 and M 2 be Tms that recognize L and co-L respectively. Consider the Tm M that on input x simulates both M 1 and M 2 (say it simulates one step of M 1 , then one step of M 2 , then the next step of M 1 , …) One of the two processes will terminate. M outputs the correct answer.

Other Undecidable Problems • Lemma: L and co-L are both recognizable iff L is decidable • We can now exhibit a language that cannot be recognized by any Tm. Claim: co-ATM is not c. e. Proof: if co-ATM were c. e. , since ATM is c. e. , by the Lemma ATM would be decidable.

Other Undecidable Problems L be a language recognized by a Tm M. We can assume wlog that for all strings x not in L M does not halt. Proof: modify M so that all transitions into the state qreject go instead to a new state qnew, and all transition from qnew lead back to qnew Such a machine rejects by not halting, and accepts by halting.

Other Undecidable Problems L be a language recognized by a Tm M. We can assume wlog that for all strings x not in L, M does not halt. Such a machine rejects by not halting, and accepts by halting. Consider the language HALTTM={<M, w> Tm M halts on input w} This is called the Halting Problem.

Other Undecidable Problems L be a language recognized by a Tm M. We can assume wlog that for all strings x not in L, M does not halt. Such a machine rejects by not halting, and accepts by halting. Consider the language HALTTM={<M, w> Tm M halts on input w} This is called the Halting Problem. Claim: The Halting Problem is undecidable. Proof: if it were decidable, we could decide ATM

Other Undecidable Problems L be a language recognized by a Tm M. We can assume wlog that for all strings x not in L, M does not halt. Such a machine rejects by not halting, and accepts by halting. Consider the language HALTTM={<M, w> Tm M halts on input w} This is called the Halting Problem. Claim: The Halting Problem is undecidable. Proof: if it were decidable, we could decide ATM Note: this is Theorem 5. 1 in Sipser, who offers a slightly different proof.

Other Undecidable Problems Def. Tms M 1 and M 2 are equivalent if for every string x M 1 halts on x iff M 2 halts on x M 1 accepts x iff M 2 accepts x and (note that if we use Tms that reject by not halting, then the first condition suffices) EQTM ={<M 1, M 2> M 1 and M 2 are equivalent} Theorem: EQTM is undecidable.

Other Undecidable Problems EQTM ={<M 1, M 2> M 1 and M 2 are equivalent} Theorem: EQTM is undecidable. Proof: We will show that if it were decidable ATM would be decidable. Given <M, w>, let M 1 be the Tm that on every input x simulates M with input w. So, for every input x, M 1 halts (and accepts) iff M accepts w. Let M 2 be the Tm that halts for no input. Then < M 1, M 2> is NOT in EQTM iff M accepts w, iff <M, w> is in ATM So we produced an algorithm that decides ATM - a contradiction.

Diagonalization

Diagonalization • Many of the non-computability results rely on the technique of diagonalization. It can be captured by the following theorem: Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to a domain with at least 2 elements. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. For definiteness, let’s take the fi: N {0, 1}

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Proof: let f. D(i) = 1– fi(i)

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Proof: let f. D(i) = 1– fi(i) Comment: Consider the matrix M[i, j] = fi(j) Then the function f. D is defined to be different from the values on the diagonal of M, the elements M[i, i].

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Proof: let f. D(i) = 1– fi(i) Note that there is a 1 -1 correspondence between N and the set of strings on an alphabet of size k, for any fixed k. We may consider every such string a description of a Turing machine (if it is not a valid set of 5 -tuples, it represents a program that is in an infinite loop).

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Note that there is a 1 -1 correspondence between N and the set of strings on an alphabet of size k, for any fixed k. We may consider every such string a description of a Turing machine (if it is not a valid set of 5 tuples, it represents a program that is in an infinite loop).

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Note that there is a 1 -1 correspondence between N and the set of strings on an alphabet of size k, for any fixed k. We may consider every such string a description of a Turing machine (if it is not a valid set of 5 tuples, it represents a program that is in an infinite loop). We will now take a trip to 19 th century Mathematics.

Diagonalization Theorem: Let f 1, f 2, … fn, … be a sequence of (total) functions where each function is from N (the set of natural numbers) to {0, 1}. Then there is a (total) function that is not in the set {f 1, f 2, … fn, … } of functions. Note that there is a 1 -1 correspondence between N and the set of strings on an alphabet of size k, for any fixed k. We may consider every such string a description of a Turing machine (if it is not a valid set of 5 tuples, it represents a program that is in an infinite loop). We will now take a trip to 19 th century Mathematics.

Diagonalization—Cardinality of Sets Georg Cantor wanted to define cardinality of sets. [Disclaimer. Much of this is not rigorous. ] He defined Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B (see Def 4. 12)

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B The set of apples and the set of oranges have the same cardinality

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality.

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. # 1 2 3 …. i …. 1 2 3 4 …. i+1

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. The set of natural numbers and the set of even natural numbers have the same cardinality.

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. The set of integers and the set of even integers have the same cardinality. The correspondence is i < -- > 2 i

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. The set of natural numbers and the set of even natural numbers have the same cardinality. The correspondence is i < -- > 2 i

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. The set of natural numbers and the set of even natural numbers have the same cardinality. The set of natural numbers and the set of rationals have the same cardinality.

Diagonalization—Cardinality of Sets A and B have the same cardinality iff there is a 1 -1 onto function from A to B Adding an element to an infinite set does not increase its cardinality. The set of natural numbers and the set of even natural numbers have the same cardinality. The set of natural numbers and the set of rationals have the same cardinality. A set is countable iff it has the same cardinality as N, the set of natural numbers, or if it is finite.

Diagonalization—Uncountable Sets A set is countable iff it has the same cardinality as N, the set of natural numbers, or if it is finite. Theorem (Cantor) The set R of real numbers is not countable. Proof: Assume the reals in (0, 1) were countable. Then we could write R= {r 1, r 2, … ri, …} where each real ri can be written as an infinite sequence of the form 0. d 1 d 2 … dn … For example Pi/10 = 0. 314159 …… Now consider the real number d=0. d 1 d 2. . di… where we define di=if rii =9 then 8; else if rii =8 then 7; else rii +1

Diagonalization—Uncountable Sets •

Diagonalization—Uncountable Sets Theorem (Cantor) The set R of real numbers is not countable. Lemma Given a finite alphabet SIGMA, the set of languages over SIGMA is not countable. (proof by diagonalization). Corollary. There is a language not recognizable by Turing machines. Proof: The set of Tms is countable. [ATM is a concrete and interesting example…]