Chapter 4 TopDown Parsing SungDong Kim Dept of

Chapter 4. Top-Down Parsing Sung-Dong Kim, Dept. of Computer Engineering, Hansung University

Abstract (1) l l Top-down parsing l Parse an input string of tokens l Leftmost derivation l Preorder traversal l From the root to the leaves Backtracking parsers l Unsuitable for practical compilers (2010 -1) Compiler 2

Abstract (2) l Predictive parsers l Predict the next construction using one or more lookahead tokens l Top-down parsing algorithms l Recursive-descent parsing l Handwritten parser (2010 -1) Compiler 3

Abstract (3) l l LL(1) parsing l Left to right l Leftmost derivation l 1: use only one symbol to predict the direction of the parse First, Follow l Lookahead sets (2010 -1) Compiler 4

1. TDP by Recursive-Descent (1) l Grammar rule l Definition for a procedure l RHS: structure of the procedure code exp addop term | term addop + | term multop factor | factor multop * factor ( exp ) | number (2010 -1) Compiler 5

1. TDP by Recursive-Descent (2) l Recursive-descent procedure for factor procedure factor; begin case token of ( : match( ( ); exp; match( ) ); number : match( number ); else error; end case; end factor; procedure match(expected. Token); begin if token = expected. Token then get. Token; else error; end if; end match; (2010 -1) Compiler 6

1. TDP by Recursive-Descent (3) l Example: if-stmt if ( exp ) statement | if ( exp ) statement else statement if-stmt if ( exp ) statement [ else statement ] procedure if. Stmt; Begin match( if ); match( ( ); exp; match( ) ); statement; if token = else then match( else ); statement; end if. Stmt; (2010 -1) Compiler 7

BNF: exp addop term | term EBNF: exp term {addop term } procedure exp; begin term; while token = + or token = - do match(token); term; end while; end exp; (2010 -1) Compiler 8

BNF: term multop factor | factor EBNF: term factor { multop factor } procedure term; begin factor; while token = * do match(token); factor; end while; end term; (2010 -1) Compiler 9

1. TDP by Recursive-Descent (4) l Problems l BNF EBNF: may be difficult l Choice difficulty l A α|β l Both α and β begin with nonterminal l First sets of α and β (2010 -1) Compiler 10

1. TDP by Recursive-Descent (5) l l A ε l What tokens can legally come after the nonterminal A l Follow set of A First, Follow: early error detection (2010 -1) Compiler 11

2. LL(1) Parsing (1) l Basic method of LL(1) parsing l Explicit stack l Generate: replace left with right (in reverse order) l Match: pop the stack + advance the input l Until the stack and input become empty (2010 -1) Compiler 12

2. LL(1) Parsing (2) l Example l Grammar: S ( S ) S | l Input: () 1 2 3 4 5 6 Parsing Stack $S $S)S( $S)S $S) $S $ reverse order Input ()$ )$ )$ $ $ Action S (S)S match S accept l List of generating actions = leftmost derivation l Tree construction: add node as each nonterminal or terminal is pushed onto the stack (2010 -1) Compiler 13

2. LL(1) Parsing (3) l LL(1) Parsing Table and Algorithm l l Top of the stack l Nonterminal: decision based on the current input token l Terminal: same as the current input token? l Choices: LL(1) parsing table M[N, T] l M = moves l N = the set of nonterminals l T = the set of terminals or tokens + $ (2010 -1) Compiler 14

2. LL(1) Parsing (4) l Table construction rules l If A is a production choice, and there is a derivation * aβ, where a is a token, then add A to the table entry M[A, a] = Given a token a in the input, if can produce an a for matching, select A l If A is a production choice, and there are derivations * ε and S$ * βA γ, where S is the start symbol and a is a token, then add A to the table entry M[A, a] = If A derives the empty string and if a comes after A, select A to make A disappear (2010 -1) Compiler 15

2. LL(1) Parsing (5) l Example l Grammar: S ( S ) S | l Input: () M[N, T] ( ) S S (S)S S (2010 -1) Compiler $ S 16

2. LL(1) Parsing (6) l LL(1) grammar l A grammar is an LL(1) grammar if the associated LL(1) parsing table has at most one production in each table entry (2010 -1) Compiler 17

Push the start symbol onto the top of the parsing stack; while the top of the parsing stack $ and the next input token $ do if the top of the parsing stack is terminal a and the next input token = a then (* match *) pop the parsing stack; advance the input; else if the top of the parsing is nonterminal A and the next input token is termal a and parsing table entry M[A, a] contains production A X 1 X 2…Xn then (* generate *) pop the parsing stack; for i : = n downto 1 do push Xi onto the parsing stack; else error; if the top of the parsing stack = $ and the next input token = $ then accept else error; (2010 -1) Compiler 18

2. LL(1) Parsing (7) l Example l Grammar l statement if-stmt | other l if-stmt if ( exp ) statement else-part l else-part else statement | ε l exp 0 | 1 (2010 -1) Compiler 19

M[N, T} if statement statemen stateme t ifnt stmt other if-stmt if (exp) statemen t elsepart else-part other else 0 1 Else-part else statement else-part ε elsepart ε exp 0 exp (2010 -1) Compiler $ exp 1 20

if (0) if (1) other else other Parsing Stack 1 $S 2 $I 3 $L S ) E ( i 4 $L S ) E ( 5 $L S ) E 6 $L S ) 0 7 $L S ) 8 $L S 9 $L I 10 $L L S ) E ( i 11 $L L S ) E ( 12 $L L S ) E 13 $L L S ) 1 14 $L L S ) 15 $L L S Input i(0) i(1) o e o$ 0) i(1) o e o$ )i(1) o e o$ (1) o e o$ o e o$ (2010 -1) Compiler Action S I I I(E)SL match E 0 match S I I i(E)SL match E 1 match S o 21

Parsing Stack 16 $L L o 17 $L L 18 $L S e 19 $L S 20 $L o 21 $L 22 $ Input o e o$ o$ o$ $ $ Action match L e. S match S o match L ε accept (2010 -1) Compiler 22

2. LL(1) Parsing (8) l Left recursion l l Simple immediate left recursion l A Aα|β [β α*] l A β A’ l Left recursion right recursion A’ α A’ | ε General immediate left recursion l A A 1 | A 2 | … | A n | 1 | 2 | … | m l A 1 A’ | 2 A’ | … | m. A’ l A’ 1 A’ | 2 A’ | … | n. A’ | (2010 -1) Compiler 23

exp addop term | term addop + | term multop factor | factor multop * factor ( exp ) | number exp term exp’ addop term exp’ | addop + | term factor term’ multop factor term’ | multop * factor ( exp ) | number (2010 -1) Compiler 24

2. LL(1) Parsing (9) l Left factoring l Two or more grammar rule choices share a common prefix string l Ex: A αβ | αγ A αA’ (2010 -1) Compiler A’ β | γ 25

statement assig-stmt | call-stmt | other assign-stmt identifier : = exp call-stmt identifier ( exp-list ) statement identifier : = exp | identifier ( exp-list ) | other statement identifier statement’ | other statement’ : = exp | ( exp-list ) (2010 -1) Compiler 26

2. LL(1) Parsing (10) l Syntax tree construction in LL(1) parsing l BNF: E E + n | n l LL(1) grammar: E n E’ E’ + n E’ | ε l Value stack: store the intermediate values l Operations l Push of a number: on match l Addition of two numbers in the stack § l Special symbol (#): when popped, an addition is performed E’ + n # E’ | ε (2010 -1) Compiler 27

3 + 4+ 5 Parsing Stack 1 $E 2 $E’ n 3 $E’ 4 $E’ # n + 5 $E’ # n 6 $E’ # 7 $E’ 8 $E’ # n + 9 $E’ # n 10 $E’ # 11 $E’ 12 $ Input 3 + 4 + 5$ 4 + 5$ 5$ $ Action value stack E n E’ $ match/push 3$ E’ + n # E’ 3$ match/push 4 3$ addstack 7$ E’ + n # E’ 7$ match push 5 7$ addstack 12$ E’ ε 12$ accept $ (2010 -1) Compiler 28

3. First and Follow Sets (1) l First sets l Definition l X = terminal or : First(X) = {X} l X = nonterminal § For each production X X 1 X 2…Xn, First(X) contains First(X 1)-{ } § For some i < n, if all the sets First(X 1), …, First(Xi) contains , then First(X) contains First(Xi+1) - { } § l If all the sets First(X 1), …, First(Xn) contains , then First(X) contains First( ) for = X 1 X 2…Xn is defined similarly (2010 -1) Compiler 29

3. First and Follow Sets (2) l Algorithm for all nonterminals A do First(A) : = {}; while there are changes to any First(A) do for each production choice A X 1 X 2…Xn do k : = 1; Continue : = true; while Continue = true and k <= n do add First(Xk) - { } to First(A); if is not in First(Xk) then Continue : = false; k : = k + 1; if Continue = true then add to First(A) (2010 -1) Compiler 30

3. First and Follow Sets (3) l Algorithm (with no -production) for all nonterminals A do First(A) : = {}; while there are changes to any First(A) do for each production choice A X 1 X 2…Xn do add First(X 1) to First(A) (2010 -1) Compiler 31

3. First and Follow Sets (4) l A nonterminal A is nullable if there exists a derivation A * l A nonterminal A is nullable if and only if First(A) contains (2010 -1) Compiler 32

3. First and Follow Sets (5) l Ex 4. 9: l exp addop term | term l addop + | - l term mulop factor | factor l mulop * l factor ( exp ) | number (2010 -1) Compiler 33

(1) exp addop term (2) exp term (3) addop + (4) addop (5) term mulop factor (6) term factor (7) mulop * (8) factor ( exp ) (9) factor number (2010 -1) Compiler 34

Grammar rule Pass 1 Pass 2 Pass 3 exp addop term exp term First(exp) = {( , number } addop + First(addop) = {+} addop - First(addop) = {+, -} termp mulop factor term factor First(term) = {( , number } mulop * First(mulop) = {*} factor ( exp ) First(factor) = { ( } factor number First(factor) = (2010 -1) { ( , number } Compiler 35

3. First and Follow Sets (5) l Ex 4. 10: l statement if-stmt | other l if-stmt if ( exp ) statement else-part l else-part else statement | ε l exp 0 | 1 (2010 -1) Compiler 36

(1) statement if-stmt (2) statement other (3) if-stmt if ( exp ) statement else-part (4) else-part else statement (5) else-part ε (6) exp 0 (7) exp 1 (2010 -1) Compiler 37

Grammar rule Pass 1 statement if-stmt Pass 2 First(statement) = { if, other } statement other First(statement) = { other } if-stmt if ( exp ) statement else-part First(if-stmt) = { if } else-part else statement First(else-part ) = { else } else-part ε First(else-part ) = { else , ε } exp 0 First(exp) = { 0 } exp 1 First(exp) = { 0, 1 } (2010 -1) Compiler 38

3. First and Follow Sets (6) l Follow sets l Definition l A = the start symbol • l B A • l $ is in Follow(A) First( ) - { } is in Follow(A) B A such that is in First( ) • Follow(A) contains Follow(B) (2010 -1) Compiler 39

Follow(start-symbol) : = {$}; for all nonterminals A start-symbol do Follow(A) : = {}; while there are changes to any Follow sets do for each production choice A X 1 X 2…Xn do for each Xi that is a nonterminal do add First(Xi+1 Xi+2…Xn) - { } to Follow(Xi) (* Note: if i = n, then Xi+1 Xi+2…Xn= if is in First(Xi+1 Xi+2…Xn) then add Follow(A) to Follow(Xi) (2010 -1) Compiler 40

Grammar rule Pass 1 Pass 2 exp addop term Follow(exp) = { $, +, - } Follow(addop) = { (, number } Follow(term) = { $, +, - , *, ) } Follow(term) = { $, +, -, * } Follow(mulop) = { (, number } Follow(factor) = { $, +, - , *, ) } exp term mulop factor term factor ( exp ) Follow(exp) = { $, +, -, ) } (2010 -1) Compiler 41

3. First and Follow Sets (7) l Constructing LL(1) Parsing Tables l Repeat the following two steps l For each token a in First(α), add A α to the entry M[A, a] l If ε is in First(α), for each element a of Follow(A), add A α to M[A, a] (2010 -1) Compiler 42

3. First and Follow Sets (8) l Theorem l A grammar in BNF is LL(1) if the following conditions are satisfied l For every production A α 1| α 2 | … | αn , First(αi) First(αj) is empty for all i and j, 1 i, j n, i j l For every nonterminal A such that First(A) contains ε, First(A) Follow(A) is empty (2010 -1) Compiler 43

3. First and Follow Sets (9) l Ex 4. 15 l First(exp) = {(, number} Follow(exp) = {$, )} l First(exp’) = {+, -, ε} Follow(exp’) = {$, )} l First(addop) = {+, -} Follow(addop) = {(, number} l First(term) = {(, number} Follow(term) = {$, ), +, -} l First(term’) = {*, ε} l First(mulop) = {*} l First(factor) = {(, number} Follow(factor) = {$, ), +, -, *} Follow(term’) = {$, ), +, -} Follow(mulop) = {(, number} (2010 -1) Compiler 44

3. First and Follow Sets (10) l Ex 4. 16 l First(statement) = {if, other} l Follow(statement) = {$, else} l First(if-stmt) = {if} l Follow(if-stmt) = {$, else} l First(else-part) = {else, ε} l Follow(else-part) = {$, else} l First(exp) = {0, 1} l Follow(exp) = {)} (2010 -1) Compiler 45

3. First and Follow Sets (11) l Ex. 4. 17 stmt-sequence stmt-seq’ ; stmt-sequence | ε stmt s l First(stmt-sequence) = {s} l Follow(stmt-sequence) = {$} l First(stmt) = {s} l Follow(stmt) = {; , $} l First(stmt-seq’) = {; , ε} l Follow(stmt-seq’) = {$} (2010 -1) Compiler 46

M[N, T] s stmt-sequence stmt-seq’ stmt s stmt-seq’ ; $ stmt-sequence stmt-seq’ ε (2010 -1) Compiler 47