Chapter 1 Introduction to the Theory of Computation

Chapter 1 Introduction to the Theory of Computation 1

Theory of Computation m Can we mathematically model the computation? q Automata Theory Ø Regular languages: DFA, NFA, Regular expressions Ø Context-free languages: Context-free grammar, pushdown automata Ø Turing machines: the most powerful model of computation m Can the computer solve all problems? q Computability Theory Ø Halting problem m How efficient the solution is? q Complexity Theory Ø Time and space complexity (Big O notation) 2

1. 1 Mathematical Preliminaries Sets (1) m A set is a collection of objects q Order and repetition is irrelevant q X = {1, 2, 3} q X = {n | 1 n 3, n N } q Y = { } = Y is an empty set m The power set of X is the set of all subsets of X q (X) = 2 X = { Y | Y X} q | (X)| = 2|X| Example X = {1, 2, 3} q 2 X = (X)={{}, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}} q {1, 2} X is a proper subset of X q {1, 2, 3} X is NOT a proper subset of X q {1, 2, 3} is a superset 3 of {1, 2} m

Sets (2) m The symbol signifies membership (belong to): q x X indicates that x is a member or element of the set X q x X indicates that x is not a member or element of the set X Ø Assume X = {1, 2, 3}, then Ø 1 X, but 4 X m Two sets are equal if they contain the same members (elements) q If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} q Then X = Y, but X Z, and so Y Z 4

Sets (3) m Assume X = {1, 2, 3} and Y = {2, 4} m Union q X Y = {z | z X or z Y} = {1, 2, 3, 4} m Intersection q X Y = {z | z X and z Y} = {2} m Difference q X - Y = {z | z X and z Y} = {1, 3} 5

Sets (4) m m m Complement q Let X be a subset of Y q The complement of X ( X ) with respect to Y is the set of elements in Y but not in X q X = { z | z Y and z X } = Y - X Assume X = {1, 2, 3} is a subset of Y = {1, 2, 3, 4} q X = {4} De. Morgan’s Laws q (X Y) = X Y 6

Inductive Proofs m Prove a statement S(X) about a family of objects X (e. g. , integers, trees) in three parts: q Basis Step: Ø Prove for one or several small values of X directly q Inductive step: Ø Assume S(Y) for Y smaller than X q Prove S(X) using that assumption 7

Inductive Proofs (EX 1) m m Prove by induction that q S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2 Basis step: q n=1 Ø 1 ( 1 + 1) / 2 = 1 m m Induction step (assumption step): q Assume that S is true for all series of length < n Prove that S is true for a series of length n? q The series of length n is Ø 1+2+3+…+n-1+n Ø By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 Ø Add n to the summation proves S. 8

Inductive Proofs (EX 2) m m Example q A complete binary tree with n leaves has 2 n - 1 nodes Proof: q Formally, S(T): if T is a binary tree with n leaves, then T has 2 n – 1 nodes q Induction is on the size = number of nodes of T 9

Inductive Proofs (EX 2) m m m Basis: q If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK Induction: q Assume S(U) for trees with fewer nodes than T. In particular, assume for the subtrees of T q T must be a root plus two subtrees U and V. q If U and V have u and v leaves, respectively, and T has t leaves, then u + v = t. Proof: q By the inductive hypothesis, U and V have 2 u - 1 and 2 v - 1 nodes, respectively. q Then T has 1 + (2 u -1) + (2 v - 1) = 2(u + v) - 1 = 2 t 1 q Proving the inductive step 10

If-And-Only-If Proofs m m m Often, a statement we need to prove is of the form q “X if and only if Y” We are then required to do two things: q Prove the if-part: Assume Y and prove X q Prove the only-if-part: Assume X, prove Y Remember: q The if and only-if parts are converses of each other One part, say “if X then Y” says nothing about whether Y is true when X is false. An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former. 11

Equivalence of Sets m m Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set To prove sets S and T are the same, prove: q x is in S if and only if x is in T. That is: Ø Assume x is in S; prove x is in T. Ø Assume x is in T ; prove x is in S. 12

Functions m Let f be a function defined on a set A and taking values in a set B. Set A is called the domain of f; set B is called the range of f. m Function f is said to be one-to-one if, whenever f(x) = f(y), it must be the case that x = y. q In other words, f is one-to-one if it maps distinct objects to distinct objects. m Function f is said to be onto if, for any b B, there exists an a A for which f(a) = f(b). m A function is bijection if it is one-to-one and onto. m If function f is defined for each element of A, then f is said to be a total function; otherwise, f is said to be a partial function. 13

Relations m Given two sets, X and Y, a relation R is a set of ordered pairs (x, y) where x X and y Y m Equivalence relations q Reflexive Ø R is reflexive if for every a A, then (a, a) R q Symmetric Ø R is symmetric if for every a and b in A, if (a, b) R, then also (b, a) R q Transitive Ø R is transitive if for any a, b, c, if (a, b) and (b, c) R, then also (a, c) R m R is an equivalence relation if R is reflexive, symmetric and transitive 14

1. 2: Strings and Languages (1) m Alphabet: Finite, nonempty set of symbols q Examples: Ø = {0, 1}: binary alphabet Ø = {a, b, c, …, z}: the set of all lower case letters Ø The set of all ASCII characters m String: Finite sequence of symbols from an alphabet q Examples: Ø 01101 where = {0, 1} Ø abracadabra where = {a, b, c, …, z} 15

1. 2: Strings and Languages (2) m Empty String: The string with zero occurrences of symbols from and is denoted or m Length of String: Number of symbols in the string q The length of a string w is usually written |w| q |1010| = 4 q | | = 0 q |uv| = |u| + |v| m Reverse : w. R q If w = abc, w. R = cba 16

1. 2: Strings and Languages (3) m Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x q x = a 1 a 2 …ai, y = b 1 b 2 …bj q xy = a 1 a 2 …aib 1 b 2 …bj q Example: x = 01101, y = 110, xy = 01101110 q xe = ex = x 17

1. 2: Strings and Languages (4) m Power of an Alphabet: k = the set of strings of length k with symbols from q Example: = {0, 1} q 1 = = {0, 1} q 2 = {00, 01, 10, 11} q 0 = {e} m Question: How many strings are there in 3? m The set of all strings over is denoted * q * = 0 1 2 3 … Also q + = 1 2 3 … q * = + {e} q + = * - {e} m 18

1. 2: Strings and Languages (5) m Substring: any string of consecutive characters in some string w q If w = abc q e, a, abc are substrings of w m Prefix and suffix: q if w = vu q v is a prefix of w q u is a suffix of w q Example Ø If w = abc Ø a, abc are prefixes of w Ø c, bc, abc are suffixes of w 19

1. 2: Strings and Languages (6) m Suppose: S is the string banana q Prefix : ban, banana q Suffix : ana, banana q Substring : nan, ban, ana, banana m Proper prefix, suffix, or substring cannot be all of S 20

1. 2: Strings and Languages (8) m Language: set of strings chosen from some alphabet m A language is a subset of * q Example of languages: ØThe set of valid Arabic words ØThe set of strings consisting of n 0’s followed by n 1’s – {e, 01, 0011, 000111, …} Ø The set of strings with equal number of 0’s and 1’s – {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …} m Empty language: = { } m The language {e} consisting of the empty string m Note: {e} 21

1. 2: Strings and Languages (9) m Can concatenate languages L 1 L 2 = {xy | x L 1, y L 2} q Ln = L concatenated with itself n times q L 0 = {e}; L 1 = L q m Star-closure = L 0 L 1 L 2 . . . q L+ = L * - L 0 q L* m Languages can be finite or infinite q q L = {a, aba, bba} L = {an | n > 0} 22

1. 2: Strings and Languages (10) OPERATION union of L and M written L M concatenation of L and M written LM Kleene closure of L written L* DEFINITION L M = {s | s is in L or s is in M} LM = {st | s is in L and t is in M} L* = Li , i=0, . . , L* denotes “zero or more concatenations of “ L positive closure of L written L+ L+ = Li, i=1, . . , L+ denotes “one or more concatenations of “ L L+ = L L* 23

1. 2: Strings and Languages (11) L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9} L D = the set of letters and digits LD = all strings consisting of a letter followed by a digit L 2 = the set of all two-letter strings L 4 = L 2 = the set of all four-letter strings L* = { All possible strings of L plus }, L+ = L* - D+ = set of strings of one or more digits L (L D ) = set of all strings consisting of a letter followed by a a letter or a digit L (L D )* = set of all strings consisting of letters and digits beginning with a letter 24

1. 2: Strings and Languages (12) m The language L consists of strings over {a, b} in which each string begins with an a should have an even length q q aa, ab L aaaa, aaab, aaba, aabb, abaa, abab, abba, abbb L baa L A L 25

1. 2: Strings and Languages (13) m The language L consists of strings over {a, b} in which each occurring of b is immediately preceded by an a q q q e L abaab L bab L abb L 26

1. 2: Strings and Languages (14) m Let X = {a, b, c} and Y = {abb, ba}. Then q XY = {aabb, babb, cabb, aba, bba, cba} q X 0 = { e } q X 1 = X = {a, b, c} q X 2 = XX = {aa, ab, ac, ba, bb, bc, ca, cb, cc} q X 3 = XXX = {aaa, aab, aac, aba, abb, abc, aca, acb, acc, baa, bab, b ac, bba, bbb, bbc, bca, bcb, bcc, caa, cab, cac, cba, cbb, cbc, cca, ccb, ccc} 27

1. 2: Strings and Languages (15) m The language L = {a, b}*{bb}{a, b}* =(a|b)*bb(a|b)* = *bb * q consists of the strings over {a, b} that contain the substring bb q bb L q abb L q bbb L q aabb L q bbaaa L q bbabba L q abab L q b L 28

1. 2: Strings and Languages (16) m Let L be the language that consists of all strings that begin with aa or end with bb q L 1 = {aa}{a, b}* q L 2 = {a, b}*{bb} q L = L 1 L 2 = {aa}{a, b}* {a, b}*{bb}=aa * | *bb q bb L q abb L q bbb L q aabb L q bbaaa L q bbabba L q abab L q ba L 29

1. 2: Strings and Languages (16) m Let L 1 = {bb} and L 2 = {e { , bbbb} over b m The languages L 1* and L 2* both contain precisely the strings consisting of an even number of b’s. m e , with length zero, is an element of L 1* and L 2* 30

1. 2: Strings and Languages (17) m What is the language of all even-length strings over {a, b} q L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)* m What is the language of all odd-length strings over {a, b} q L = {a, b}* - {aa, bb, ab, ba}* or q L = {a, b}{aa, bb, ab, ba}* or q L = {aa, bb, ab, ba}* {a, b} or q L = (a|b)(aa|bb|ab|ba)* 31