Chapter 1 Introduction to the Theory of Computation

Chapter 1 Introduction to the Theory of Computation 1

Theory of Computation m Can we mathematically model the computation? q Automata Theory Ø Regular languages: DFA, NFA, Regular expressions Ø Context-free languages: Context-free grammar, pushdown automata Ø Turing machines: the most powerful model of computation m Can the computer solve all problems? q Computability Theory Ø Halting problem m How efficient the solution is? q Complexity Theory Ø Time and space complexity (Big O notation) 2

1. 1 Mathematical Preliminaries Sets (1) m A set is a collection of objects q Order and repetition is irrelevant q X = {1, 2, 3} q X = {n | 1 n 3, n N } q Y = { } = Y is an empty set m The power set of X is the set of all subsets of X q (X) = 2 X = { Y | Y X} q | (X)| = 2|X| Example X = {1, 2, 3} q 2 X = (X)={{}, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}} q {1, 2} X is a proper subset of X q {1, 2, 3} X is NOT a proper subset of X q {1, 2, 3} is a superset 3 of {1, 2} m

Sets (2) m The symbol signifies membership (belong to): q x X indicates that x is a member or element of the set X q x X indicates that x is not a member or element of the set X Ø Assume X = {1, 2, 3}, then Ø 1 X, but 4 X m Two sets are equal if they contain the same members (elements) q If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} q Then X = Y, but X Z, and so Y Z 4

Sets (3) m Assume X = {1, 2, 3} and Y = {2, 4} m Union q X Y = {z | z X or z Y} = {1, 2, 3, 4} m Intersection q X Y = {z | z X and z Y} = {2} m Difference q X - Y = {z | z X and z Y} = {1, 3} 5

Sets (4) m m m Complement q Let X be a subset of Y q The complement of X ( X ) with respect to Y is the set of elements in Y but not in X q X = { z | z Y and z X } = Y - X Assume X = {1, 2, 3} is a subset of Y = {1, 2, 3, 4} q X = {4} De. Morgan’s Laws q (X Y) = X Y 6

Inductive Proofs m Prove a statement S(X) about a family of objects X (e. g. , integers, trees) in three parts: q Basis Step: Ø Prove for one or several small values of X directly q Assumption step : Ø Assume S(Y) for Y smaller than X q Prove S(X) using that assumption 7

Inductive Proofs (EX 1) m m Prove by induction that q S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2 Basis step: q n=1 Ø 1 ( 1 + 1) / 2 = 1 m m Assumption step: q Assume that S is true for all series of length < n Prove that S is true for a series of length n? q The series of length n is Ø 1+2+3+…+n-1+n Ø By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 Ø Add n to the summation proves S. 8

Inductive Proofs (EX 2) m Example q A complete binary tree with n leaves has 2 n - 1 nodes 9

Inductive Proofs (EX 2) m Basis: q m If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK Assumption: Assume that the statement is true for all trees that have leaves less than n Proof: Proof that the statement is true for trees that have n leaves Ø T has two subtrees, U and V Ø U has u leaves and V has v leaves Ø According to the assumption: U has (2 u – 1) of nodes and V has (2 v – 1) of nodes Ø T must be a root plus two subtrees U and V. The number of leaves in T equal number of leaves in U + number of leaves in V = u + v Ø The number of nodes in T = (2 u – 1) + (2 v – 1) + 1 = 2 ( u + q m v) - 1 q 10

If-And-Only-If Proofs m m m Often, a statement we need to prove is of the form q “X if and only if Y” We are then required to do two things: q Prove the if-part: Assume Y and prove X q Prove the only-if-part: Assume X, prove Y Remember: q The if and only-if parts are converses of each other One part, say “if X then Y” says nothing about whether Y is true when X is false. An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former. 11

Equivalence of Sets m m Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set To prove sets S and T are the same, prove: q x is in S if and only if x is in T. That is: Ø Assume x is in S; prove x is in T. Ø Assume x is in T ; prove x is in S. 12

Functions m Let f be a function defined on a set A and taking values to a set B. Set A is called the domain of f; set B is called the range of f. m The arity of the functions q The number and type of input arguments the number and type of output q Examples: Ø The arity of the Multiplication: Num × Num Ø The arity of the Square. Root : Num 13

1. 2: Strings and Languages (1) m Alphabet: Finite, nonempty set of symbols q Examples: Ø = {0, 1}: binary alphabet Ø = {a, b, c, …, z}: the set of all lower case letters Ø The set of all ASCII characters m String: Finite sequence of symbols from an alphabet q Examples: Ø 01101 where = {0, 1} Ø abracadabra where = {a, b, c, …, z} 14

1. 2: Strings and Languages (2) m Empty String: The string with zero occurrences of symbols from and is denoted or m Length of String: Number of symbols in the string q The length of a string w is usually written |w| q |1010| = 4 q | | = 0 q |uv| = |u| + |v| m Reverse : w. R q If w = abc, w. R = cba 15

1. 2: Strings and Languages (3) m Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x q x = a 1 a 2 …ai, y = b 1 b 2 …bj q xy = a 1 a 2 …aib 1 b 2 …bj q Example: x = 01101, y = 110, xy = 01101110 q xe = ex = x 16

1. 2: Strings and Languages (4) m Power of an Alphabet: k = the set of strings of length k with symbols from q Example: = {0, 1} q 1 = = {0, 1} q 2 = {00, 01, 10, 11} q 0 = {e} m Question: How many strings are there in 3? m The set of all strings over is denoted * q * = 0 1 2 3 … Also q + = 1 2 3 … q * = + {e} q + = * - {e} m 17

1. 2: Strings and Languages (5) m Substring: any string of consecutive characters in some string w q If w = abc q e, a, abc are substrings of w m Prefix and suffix: q if w = vu q v is a prefix of w q u is a suffix of w q Example Ø If w = abc Ø a, abc are prefixes of w Ø c, bc, abc are suffixes of w 18

1. 2: Strings and Languages (6) m Suppose: S is the string banana q Prefix : ban, banana q Suffix : ana, banana q Substring : nan, ban, ana, banana 19

1. 2: Strings and Languages (8) m Language: set of strings chosen from some alphabet m A language is a subset of * q Example of languages: ØThe set of valid Arabic words ØThe set of strings consisting of n 0’s followed by n 1’s – {e, 01, 0011, 000111, …} Ø The set of strings with equal number of 0’s and 1’s – {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …} m Empty language: = { } m The language {e} consisting of the empty string m Note: {e} 20

1. 2: Strings and Languages (9) m Can concatenate languages L 1 L 2 = {xy | x L 1, y L 2} q Ln = L concatenated with itself n times q L 0 = {e}; L 1 = L q m Star-closure = L 0 L 1 L 2 . . . q L+ = L * - L 0 q L* m Languages can be finite or infinite q q L = {a, aba, bba} L = {an | n > 0} 21

1. 2: Strings and Languages (10) OPERATION union of L and M written L M concatenation of L and M written LM Kleene closure of L written L* DEFINITION L M = {s | s is in L or s is in M} LM = {st | s is in L and t is in M} L* = Li , i=0, . . , L* denotes “zero or more concatenations of “ L positive closure of L written L+ L+ = Li, i=1, . . , L+ denotes “one or more concatenations of “ L L+ = L L* 22

1. 2: Strings and Languages (11) L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9} L D = the set of letters and digits LD = all strings consisting of a letter followed by a digit L 2 = the set of all two-letter strings L 4 = L 2 = the set of all four-letter strings L* = { All possible strings of L plus }, L+ = L* - D+ = set of strings of one or more digits L (L D ) = set of all strings consisting of a letter followed by a a letter or a digit L (L D )* = set of all strings consisting of letters and digits beginning with a letter 23

1. 2: Strings and Languages (12) m The language L consists of strings over {a, b} in which each string begins with an a should have an even length q q aa, ab L aaaa, aaab, aaba, aabb, abaa, abab, abba, abbb L baa L a L 24

1. 2: Strings and Languages (13) m The language L consists of strings over {a, b} in which each occurring of b is immediately preceded by an a q q q e L abaab L bab L abb L 25

1. 2: Strings and Languages (14) m Let X = {a, b, c} and Y = {abb, ba}. Then q XY = {aabb, babb, cabb, aba, bba, cba} q X 0 = { e } q X 1 = X = {a, b, c} q X 2 = XX = {aa, ab, ac, ba, bb, bc, ca, cb, cc} q X 3 = XXX = {aaa, aab, aac, aba, abb, abc, aca, acb, acc, baa, bab, b ac, bba, bbb, bbc, bca, bcb, bcc, caa, cab, cac, cba, cbb, cbc, cca, ccb, ccc} 26

1. 2: Strings and Languages (15) m The language L = {a, b}*{bb}{a, b}* = *{bb} * q consists of the strings over {a, b} that contain the substring bb q bb L q abb L q bbb L q aabb L q bbaaa L q bbabba L q abab L q b L 27

1. 2: Strings and Languages (16) m Let L be the language that consists of all strings that begin with aa or end with bb q L 1 = {aa}{a, b}* q L 2 = {a, b}*{bb} q L = L 1 L 2 = {aa}{a, b}* {a, b}*{bb} q bb L q abb L q bbb L q aabb L q bbaaa L q bbabba L q abab L q ba L 28

1. 2: Strings and Languages (16) m Let L 1 = {bb} and L 2 = {e { , bbbb} over b m The languages L 1* and L 2* both contain precisely the strings consisting of an even number of b’s. m e , with length zero, is an element of L 1* and L 2* 29

1. 2: Strings and Languages (17) m What is the language of all even-length strings over {a, b} q L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)* m What is the language of all odd-length strings over {a, b} q L = {a, b}* - {aa, bb, ab, ba}* or q L = {a, b}{aa, bb, ab, ba}* or q L = {aa, bb, ab, ba}* {a, b} 30