CSCE 531 Compiler Construction Lecture 7 Predictive Parsing

CSCE 531 Compiler Construction Lecture 7 Predictive Parsing Topics n Review Top Down Parsing n First Follow LL (1) Table construction n n Readings: 4. 4 Homework: Program 2 – 1– February 8, 2018 CSCE 531 Spring 2018

Overview Last Time n Ambiguity in classic programming language grammars l Expressions l If-Then-Else n n Top-Down Parsing Modifying Grammars to facilitate Top-down parsing Today’s Lecture n n n Regroup halfway to Test 1 First and Follow LL(1) property References: Homework: – 2– CSCE 531 Spring 2018

Removing the IF-ELSE Ambiguity Stmt if Expr then Stmt | if Expr then Stmt else Stmt | other stmts Stmt Matched. Stmt | Unmatched. Stmt Matched. Stmt if Expr then Matched. Stmt else Matched. Stmt | Others. Statements Unmatched. Stmt if Expr then Matched. Stmt else | if Expr then Matched. Stmt else Umatched. Stmt – 3– CSCE 531 Spring 2018

Recursive Descent Parsers l Recall the parser from Chapter 2 l A recursive descent parser has a routine for each nonterminal. These routines can call each other. If one of these fails then it may backtrack to a point where there is an alternative choice. l In certain cases the grammar is restricted enough where backtracking would never be required. l Such a parser is called a predictive parser. l The parser from Chapter 2 is a predictive parser. – 4– CSCE 531 Spring 2018

Transition Diagrams for Predictive Parsers To construct the transition diagram for a predictive parser: 1. Eliminate left recursion from the grammar 2. Left factor the grammar 3. For each nonterminal A do Create an initial state and final state. For each production A X 1 X 2 … Xn create a path from the initial state to the final state labeled X 1 X 2 … Xn end – 5– CSCE 531 Spring 2018

Example E T E’ E’ + T E’ | - T E’ | ε T F T’ T’ * F T’ | / F T’ | ε F id | num | ( E ) ε E E’ 1 T 2 E’ 3 + 1 - T 4 F 5 T’ 2 2 T T 3 3 E’ 6 Etcetera Some of the rest in the text. – 6– CSCE 531 Spring 2018

Predictive Parsing using Transition Diagrams – 7– CSCE 531 Spring 2018

Table Driven Predictive Parsing input x Stack W X + ( Predictive Parsing Program … output Y S R $ – 8– Parsing Table M CSCE 531 Spring 2018

Table Driven Predictive Parsing l The stack is initialized to contain $S, the $ is the “bottom” marker. l The input has a $ added to the end. l The parse table, M[X, a] contains what should be done when we see nonterminal X on the stack and current token “a” l Parse Actions for n n X = top of stack, and a = current token 1. If X = a = $ then halt and announce success. 2. If X = a != $ then pop X off the stack and advance the input pointer to the next token. 3. If X is nonterminal consult the table entry M[X, a], details on next slide. – 9– CSCE 531 Spring 2018

M[X, a] Actions 3. If X is nonterminal then consult M[X, a]. l The entry will be either a production or an error entry. l If M[X, a] = {X UVW} the parser n n n – 10 – replaces X on the top of the stack with W, V, U with the U on the top As output print the name of the production used. CSCE 531 Spring 2018

Algorithm 4. 3 Set ip to the first token in w$. Repeat Let X be the top of the stack and a be the current token if X is a terminal or $ then if X = a then pop X from the stack and advance the ip else error() else /* X is a nonterminal */ if M[X, a] = X Y 1 Y 2 …Yk then begin pop X from the stack push Yk Yk-1 …Y 2 Y 1 onto the stack with Y 1 on top output the production X Y 1 Y 2 …Yk end else error() CSCE 531 Spring 2018 – 11 – Until X=$

Parse Table for Expression Grammar id E * / ( ) $ E’ ε T’ ε E TE’ E’ +TE’ E’ -TE’ T FT’ T’ ε T’ F - E TE’ E’ T + T’ ε T’ *FT’ T’ /FT’ F id F (E) Figure 4. 15+ – 12 – CSCE 531 Spring 2018

Parse Trace of (z + q) * x + w * y Stack Input Output $E ( id + id ) * id + id * id $ $E’T ( id + id ) * id + id * id $ E T E’ $E’T’F ( id + id ) * id + id * id $ T F T’ $E’T’)E( ( id + id ) * id + id * id $ F ( E ) $E’T’)E id + id ) * id + id * id $ $E’T’)E’T id + id ) * id + id * id $ E T E’ $E’T’)E’T’F id + id ) * id + id * id $ T F T’ $E’T’)E’T’id id + id ) * id + id * id $ F id $E’T’)E’T’ + id ) * id + id * id $ $E’T’)E’ + id ) * id + id * id $ – 13 – T’ ε CSCE 531 Spring 2018

First and Follow Functions We are going to develop two auxilliary functions for facilitating the computing of parse tables. FIRST(α) is the set of tokens that can start strings derivable from α, also if α ε then we add ε to First(α). FOLLOW(N) is the set of tokens that can follow the nonterminal N in some sentential form, i. e. , FOLLOW(N) = { t | S * αNtβ } – 14 – CSCE 531 Spring 2018

Algorithm to Compute First Input: Grammar symbol X Output: FIRST(X) Method 1. If X is a terminal, then FIRST(X) = {X} 2. If X є is a production, then add є to FIRST(X). 3. For each production X Y 1 Y 2 … Yk a. If Y 1 Y 2 … Yi-1 є then add all tokens in FIRST(Yi) to FIRST(X) b. If Y 1 Y 2 … Yk є then add є to FIRST(X) – 15 – CSCE 531 Spring 2018

Example of First Calculation E T E’ E’ + T E’ | - T E’ | є T F T’ T’ * F T’ | / F T’ | є F id | num | ( E ) FIRST(token) = {token} for tokens: + - * / ( ) id num FIRST(F) = { id, num, ( } FIRST(T’) = ? T’ є so … T’ *FT’ so … T’ /FT’ so … FIRST(T’) = {є … } FIRST(T) = FIRST(F) FIRST(E’) = ? – 16 – FIRST(E) = ? CSCE 531 Spring 2018

Algorithm to Compute Follow (p 189) Input: nonterminal A Output: FOLLOW(A) Method 1. Add $ to FOLLOW(S), where n n $ is the end_of_input marker And S is the start state 2. If A αBβ is a production, then every token in FIRST(β) is added to FOLLOW(B) (note not є) 3. If A αB is a production or if A αBβ is a production and β є then every token in FOLLOW(A) is added to FOLLOW(B) – 17 – CSCE 531 Spring 2018

Example of FOLLOW Calculation E T E’ 1. Add $ to FOLLOW(E) E’ + T E’ | - T E’ | є 2. E TE’ T F T’ T’ * F T’ | / F T’ | є F id | num | ( E ) n 3. E’ + T E’ (similarly E’ +T E’) n n N E {$ E’ { T {+- FOLLOW(N) – 18 – Add FIRST*(E’) to FOLLOW(T) E’ є, so FOLLOW(E’) is added to FOLLOW(T) 4. T F T’ n n T’ { F { Add FIRST*(E’) to FOLLOW(T) Add FIRST*(T’) to FOLLOW(F) T’ є, so FOLLOW(T’) is added to FOLLOW(F) 5. F ( E ) n CSCE 531 Spring 2018 Add FIRST( ‘)’ ) to FOLLOW(E)

Construction of a Predictive Parse Table Algorithm 4. 4 Input: Grammar G Output: Predictive Parsing Table M[N, a] Method 1. For each production A α do 2. 3. For each a in FIRST(α), add A α to M[A, a] If є is in FIRST(α), add A α to M[A, b] for each token b in FOLLOW(A) is in FIRST(α) and $ is in FOLLOW(A) then add A α to M[A, $] If є 4. Mark all other entries of M as “error” – 19 – CSCE 531 Spring 2018

Predictive Parsing Example 4. 18 in text table in Figure 4. 15 (slide 11) Example 4. 19 FOLLOW(S) = { $, e } S’ e. S | є FOLLOW(S’) = { $, e} E b a S S a FIRST(S’) = {є, e } FIRST(E) = { b } S i. Et. SS’ | a Nonterminals FIRST(S) = { i, a } FOLLOW(E) = { t b e i t $ S i. Et. SS’ S’ e. S S’ S’ є E – 20 – E b CSCE 531 Spring 2018

LL(1) Grammars A grammar is called LL(1) if its parsing table has no multiply defined entries. LL(1) grammars l Must not be ambiguous. l Must not be left-recursive. l G is LL(1) if and only if whenever A α | β 1. 2. 3. – 21 – FIRST(α) ∩ FIRST(β) = Φ At most one of α and β can derive є If β * є then FIRST(α) ∩ FOLLOW(A) = Φ CSCE 531 Spring 2018

Error Recovery in Predictive Parsing Panic Mode Error recovery l If M[A, a] is an error, then throw away input tokens until one in a synchronizing set. l Heuristics for the synchronizing sets for A 1. Add FOLLOW(A) to the synchronizing set for A 2. If ‘; ’ is a separator or terminator of statements then keywords that can begin statements should not be in synchronizing set for the nonterminal “Expr” because a missing “; ” would cause skipping keywords. 3. … – 22 – CSCE 531 Spring 2018

Parse Table with Synch Entries Figure 4. 18 – 23 – CSCE 531 Spring 2018

Trace with Error Recovery Figure 4. 19 – 24 – CSCE 531 Spring 2018

Bottom up Parsing Idea – recognize right hand sides of productions so that we produce a rightmost derivation “Handle-pruning” – 25 – CSCE 531 Spring 2018

Reductions in a Shift-Reduce Parser Figure 4. 21 E E + E | E * E | ( E ) | id Right-Sentential Form Handle Reducing Production id 1 + id 2 * id 3 id 1 E id E + id 2 * id 3 id 2 E id E + E * id 3 E id How? E+E*E E E*E E+E E – 26 – CSCE 531 Spring 2018

– 27 – CSCE 531 Spring 2018