Chapter 4 2 Chang ChiChung 2007 06 07

Chapter 4 -2 Chang Chi-Chung 2007. 06. 07

Bottom-Up Parsing n LR methods (Left-to-right, Rightmost derivation) q n LR(0), SLR, Canonical LR = LR(1), LALR Other special cases: q q Shift-reduce parsing Operator-precedence parsing

Operator-Precedence Parsing n n Special case of shift-reduce parsing See textbook section 4. 6

Bottom-Up Parsing reduction E → T * F → T * id → F * id → id * id rightmost derivation E→E+T|T T→T*F|F F → ( E ) | id

E→E+T|T T→T*F|F F → ( E ) | id Handle Pruning n Handle q A handle is a substring of grammar symbols in a right-sentential form that matches a right-hand side of a production Right Sentential Form Handle Reducing Production id 1 * id 2 id 1 F → id F * id 2 F T→F T * id 2 F → id T*F E→T*F

Handle Pruning n A rightmost derivation in reverse can be obtained by “handle pruning” S = γ 0 →rm γ 1 → rm γ 2 →rm …. →rm γn-1 →rm γn =ω n Handle definition q S →*rm αAω →*rm αβω S A α β ω A handle A →β in the parse tree for αβω

Example: Handle Grammar S a. ABe A Abc|b B d abbcde a. Ade a. ABe S abbcde a. AAe …? Handle NOT a handle, because further reductions will fail (result is not a sentential form)

Shift-Reduce Parsing n Shift-Reduce Parsing is a form of bottom-up parsing q q n A stack holds grammar symbols An input buffer holds the rest of the string to parsed. Shift-Reduce parser action q q shift reduce accept error

Shift-Reduce Parsing n Shift q n Reduce q q n The right end of the string to be reduced must be at the top of the stack. Locate the left end of the string within the stack and decide with what nonterminal to replace the string. Accept q n Shift the next input symbol onto the top of the stack. Announce successful completion of parsing Error q Discover a syntax error and call recovery routine

Shift-Reduce Parsing Stack $ $id 1 $F $T $T * id 2 $T * F $T $E Input id 1 * id 2 $ id 2 $ $ $ E→E+T|T T→T*F|F F → ( E ) | id Action shift reduce by F → id reduce by T → F shift reduce by F → id reduce by T → T * F reduce by E → T accept

Example: Shift-Reduce Parsing Grammar: S a. ABe A Abc|b B d These match production’s right-hand sides Shift-reduce corresponds to a rightmost derivation: S rm a A B e rm a A d e rm a A b c d e rm a b b c d e Reducing a sentence: abbcde a. Ade a. ABe S S A A A B A B a b b c d e

Conflicts During Shift-Reduce Parsing n Conflicts Type shift-reduce q reduce-reduce Shift-reduce and reduce-reduce conflicts are caused by q n q q The limitations of the LR parsing method (even when the grammar is unambiguous) Ambiguity of the grammar

Shift-Reduce Conflict Grammar E E+E E E*E E (E) E id Find handles to be reduced Stack Input $ id+id*id$ $id +id*id$ $E+id *id$ $E+E*id $ $E+E*E $ $E+E $ $E $ Action shift reduce E id shift (or reduce? ) shift reduce E id reduce E E * E reduce E E + E accept How to resolve conflicts?

Shift-Reduce Conflict Ambiguous grammar: S if E then S | if E then S else S | other Stack $… $…if E then S Input Action …$ … else…$ shift or reduce? Shift else to if E then S or Reduce if E then S Resolve in favor of shift, so else matches closest if

Reduce-Reduce Conflict Grammar C AB A a B a Resolve in favor of reduce A a, otherwise we’re stuck! 動彈不得 Stack $ $a Input aa$ a$ Action shift reduce A a or B a ?

LR n LR parser are table-driven q n Much like the nonrecursive LL parsers. The reasons of the using the LR parsing q q An LR-parsering method is the most general nonbacktracking shift-reduce parsing method known, yet it can be implemented as efficiently as other. LR parsers can be constructed to recognize virtually all programming-language constructs for which context-free grammars can be written. An LR parser can detect a syntactic error as soon as it is possible to do so on a left-to-right scan of the input. The class of grammars that can be parsed using LR methods is a proper superset of the class of grammars that can be parsed with predictive or LL methods.

LR(0) n n An LR parser makes shift-reduce decisions by maintaining states to keep track. An item of a grammar G is a production of G with a dot at some position of the body. q Example A→XYZ items A → ．X Y Z A → X．Y Z A→ XY．Z A→ XYZ． A → X．Y Z stack next derivations with input strings Note that production A has one item [A • ]

LR(0) n Canonical LR(0) Collection q q One collection of sets of LR(0) items Provide the basis for constructing a DFA that is used to make parsing decisions. n n LR(0) automation The canonical LR(0) collection for a grammar q Augmented the grammar n q q If G is a grammar with start symbol S, then G’ is the augmented grammar for G with new start symbol S’ and new production S’ → S Closure function Goto function

Use of the LR(0) Automaton

Function Closure n n If I is a set of items for a grammar G, then closure(I) is the set of items constructed from I. Create closure(I) by the two rules: q q n add every item in I to closure(I) If A→α．Bβ is in closure(I) and B →γ is a production, then add the item B →．γ to closure(I). Apply this rule untill no more new items can be added to closure(I). Divide all the sets of items into two classes q Kernel items n q initial item S’ → ．S, and all items whose dots are not at the left end. Nonkernel items n All items with their dots at the left end, except for S’ → ．S

Example n n The grammar G E’ → E E →E+T | T T →T*F | F F → ( E ) | id Let I = { E’ → ．E } , then closure(I) = { E’ →．E E →．E + T E →．T T →．T * F T →．F F →．( E ) F →．id }

Exercise n n The grammar G E’ → E E →E+T | T T →T*F | F F → ( E ) | id Let I = { E → E +． T }

Function Goto n Function Goto(I, X) q q I is a set of items X is a grammar symbol Goto(I, X) is defined to be the closure of the set of all items [A α X‧β] such that [A α‧ Xβ] is in I. Goto function is used to define the transitions in the LR(0) automation for a grammar.

Constructing the LR(0) Collection 1. The grammar is augmented with a new start symbol S’ and production S’ S 2. Initially, set C = closure({[S’ • S]}) (this is the start state of the DFA) 3. For each set of items I C and each grammar symbol X (N T) such that GOTO(I, X) C and goto(I, X) , add the set of items GOTO(I, X) to C 4. Repeat 3 until no more sets can be added to C

Example: The Parse of id * id Line STACK (1) (2) (3) (4) (5) (6) (7) (8) (9) 0 05 03 02 0275 0 2 7 10 02 01 SYMBOLS INPUT $ $ id $F $T $T* $ T * id $T*F $T $E id * id $ id $ $ $ ACTION shift to 5 reduce by F → id reduce by T → F shift to 7 shift to 5 reduce by F → id reduce by T → T * F reduce by E → T accept

Model of an LR Parser

Structure of the LR Parsing Table n Parsing Table consists of two parts: q q n A parsing-action function ACTION A goto function GOTO The Action function, Action[i, a], have one of four forms: q Shift j, where j is a state. n q Reduce A→β. n q The action of the parser reduces β on the top of the stack to head A. Accept n q The action taken by the parser shifts input a to the stack, but use state j to represent a. The parser accepts the input and finishes parsing. Error

Structure of the LR Parsing Table(1) n The GOTO Function, GOTO[Ii, A], defined on sets of items. q If GOTO[Ii, A] = Ij, then GOTO also maps a state i and a nonterminal A to state j.

LR-Parser Configurations Configuration ( = LR parser state): ($s 0 s 1 s 2 … sm, ai ai+1 … an $) stack input ($ X 1 X 2 … Xm, ai ai+1 … an $) If action[sm, ai] = shift s then push s (ai), and advance input (s 0 s 1 s 2 … sm, ai ai+1 … an $) (s 0 s 1 s 2 … sm s, ai+1 … an $) If action[sm, ai] = reduce A and goto[sm-r, A] = s with r = | | then pop r symbols, and push s ( push A ) ( (s 0 s 1 s 2 … sm, ai ai+1 … an $) (s 0 s 1 s 2 … sm-r s, ai ai+1 … an $) If action[sm, ai] = accept then stop If action[sm, ai] = error then attempt recovery

Example LR Parse Table Grammar: 1. E E + T 2. E T 3. T T * F 4. T F 5. F ( E ) 6. F id shift & goto 5 reduce by production #1 action state 0 id s 5 + * ( s 4 goto ) $ 1 s 6 2 r 2 s 7 r 2 3 r 4 r 4 4 s 4 r 6 T 2 F 3 8 2 3 9 3 acc s 5 5 E 1 r 6 6 s 5 s 4 7 s 5 s 4 r 6 10 8 s 6 s 11 9 r 1 s 7 r 1 10 r 3 r 3 11 r 5 r 5

Line Example Grammar 0. S E 1. E E + T 2. E T 3. T T * F 4. T F 5. F ( E ) 6. F id STACK SYMBOLS (1) 0 INPUT ACTION id * id + id $ shift 5 (2) 0 5 id * id + id $ reduce 6 goto 3 (3) 0 3 F * id + id $ reduce 4 goto 2 (4) 0 2 T * id + id $ shift 7 (5) 0 2 7 T* (6) 0 2 7 5 T * id + id $ reduce 6 goto 10 (7) 0 2 7 10 T*F + id $ reduce 3 goto 2 (8) 0 2 T + id $ reduce 2 goto 1 (9) 0 1 E + id $ shift 6 (10 0 1 6 ) E+ (11 0 1 6 5 ) E + id $ reduce 6 goto 3 (12 0 1 6 3 ) E+F $ reduce 4 goto 9 (13 0 1 6 9 E+T $ reduce 1 goto 1 id + id $ shift 5

SLR Grammars n n SLR (Simple LR): a simple extension of LR(0) shift-reduce parsing SLR eliminates some conflicts by populating the parsing table with reductions A on symbols in FOLLOW(A) Shift on + S E E id + E E id State I 2: State I 0: goto(I 0, id) E id • + E goto(I 3, +) S • E E id • E • id + E E • id FOLLOW(E)={$} thus reduce on $

SLR Parsing Table n n Reductions do not fill entire rows Otherwise the same as LR(0) 1. S E 2. E id + E 3. E id 0 id s 2 + E 1 acc 1 s 3 2 3 $ r 3 4 s 2 4 Shift on + FOLLOW(E)={$} thus reduce on $ r 2

Constructing SLR Parsing Tables n Augment the grammar with S’ S n Construct the set C={I 0, I 1, …, In} of LR(0) items n State i is constructed from Ii n If [A • a ] Ii and goto(Ii, a)=Ij then set action[i, a]=shift j n If [A • ] Ii then set action[i, a]=reduce A for all a FOLLOW(A) (apply only if A S’) n If [S’ S • ] is in Ii then set action[i, $]=accept n If goto(Ii, A)=Ij then set goto[i, A]=j set goto table n Repeat 3 -4 until no more entries added n The initial state i is the Ii holding item [S’ • S]

Example SLR Grammar and LR(0) Items I 0 = closure({[C’ • C]}) I 1 = goto(I 0, C) = closure({[C’ C • ]}) … State I 1: State I 4: final C’ C • C A B • goto(I 0, C) Augmented grammar: 0. C’ C 1. C A B 2. A a 3. B a start goto(I 2, B) State I 0: State I 2: goto(I , A) C’ • C 0 C A • B C • A B B • a goto(I 2, a) A • a goto(I 0, a) State I 3: A a • State I 5: B a •

Example SLR Parsing Table State I 0: C’ • C C • A B A • a State I 1: C’ C • 0 1 C start 0 A B 4 2 a a 3 State I 2: C A • B B • a 5 State I 3: A a • a s 3 1 $ State I 4: C A B • C 1 A 2 B acc 2 s 5 3 r 2 4 4 r 1 5 r 3 State I 5: B a • Grammar: 0. C’ C 1. C A B 2. A a 3. B a

SLR and Ambiguity n n Every SLR grammar is unambiguous, but not every unambiguous grammar is SLR, maybe LR(1) Consider for example the unambiguous grammar S L=R|R L * R | id R L I 0 : S’ • S S • L=R S • R L • *R L • id R • L I 1 : S’ S • I 2 : S L • =R R L • FOLLOW(R) = {=, $} I 3 : S R • I 4 : L * • R R • L L • *R L • id I 5 : L id • action[2, =]=s 6 Has no SLR no action[2, =]=r 5 parsing table I 6 : S L= • R R • L L • *R L • id I 7 : L *R • I 8 : R L • I 9 : S L=R •

LR(1) Grammars n n n SLR too simple LR(1) parsing uses lookahead to avoid unnecessary conflicts in parsing table LR(1) item = LR(0) item + lookahead LR(0) item [A • ] LR(1) item [A • , a]

SLR Versus LR(1) n n Split the SLR states by adding LR(1) lookahead Unambiguous grammar I 2 : S L=R|R S L • =R split R L • L * R | id R L S L • =R R L • action[2, =]=s 6 Should not reduce, because no right-sentential form begins with R=

LR(1) Items n n n An LR(1) item [A • , a] contains a lookahead terminal a, meaning already on top of the stack, expect to see a For items of the form [A • , a] the lookahead a is used to reduce A only if the next input is a For items of the form [A • , a] with the lookahead has no effect

The Closure Operation for LR(1) Items n n n Start with closure(I) = I If [A • B , a] closure(I) then for each production B in the grammar and each terminal b FIRST( a) add the item [B • , b] to I if not already in I Repeat 2 until no new items can be added

The Goto Operation for LR(1) Items n n For each item [A • X , a] I, add the set of items closure({[A X • , a]}) to goto(I, X) if not already there Repeat step 1 until no more items can be added to goto(I, X)

Exercise n n n Let I = { (S → C • C, $) } I 2 = closure(I) = ? I 3 = goto(I 2, c) = ? The grammar G S’ → S S →CC C →c. C | d

Construction of the sets of LR(1) Items n n Augment the grammar with a new start symbol S’ and production S’ S Initially, set C = closure({[S’ • S, $]}) (this is the start state of the DFA) For each set of items I C and each grammar symbol X (N T) such that goto(I, X) C and goto(I, X) , add the set of items goto(I, X) to C Repeat 3 until no more sets can be added to C

LR(1) Automation

Construction of the Canonical LR(1) Parsing Tables Augment the grammar with S’ S Construct the set C={I 0, I 1, …, In} of LR(1) items State i of the parser is constructed from Ii n n n q q q n n n If [A • a , b] Ii and goto(Ii, a)=Ij then set action[i, a]=shift j If [A • , a] Ii then set action[i, a]=reduce A (apply only if A S’) If [S’ S • , $] is in Ii then set action[i, $]=accept If goto(Ii, A)=Ij then set goto[i, A]=j Repeat 3 until no more entries added The initial state i is the Ii holding item [S’ • S, $]

Example state 0 ACTION c d s 3 s 4 1 GOTO $ S C 1 2 acc 2 s 6 s 7 5 3 s 4 8 4 r 3 5 6 r 1 s 6 s 7 7 8 9 9 r 3 r 2 r 2 The grammar G S’ → S S →CC C →c. C | d

Example Grammar and LR(1) Items n n n Unambiguous LR(1) grammar: S L=R|R L * R | id R L Augment with S’ S LR(1) items (next slide)

I 0: [S’ • S, [S • L=R, [S • R, [L • *R, [L • id, [R • L, $] goto(I 0, S)=I 1 $] goto(I 0, L)=I 2 $] goto(I 0, R)=I 3 =/$] goto(I 0, *)=I 4 =/$] goto(I 0, id)=I 5 $] goto(I 0, L)=I 2 I 6: [S L= • R, [R • L, [L • *R, [L • id, $] goto(I 6, R)=I 9 $] goto(I 6, L)=I 10 $] goto(I 6, *)=I 11 $] goto(I 6, id)=I 12 I 7: [L *R • , =/$] I 1: [S’ S • , $] I 8: [R L • , =/$] I 2: [S L • =R, [R L • , $] goto(I 0, =)=I 6 $] I 9: [S L=R • , $] I 3: [S R • , $] I 4: [L * • R, [R • L, [L • *R, [L • id, =/$] goto(I 4, R)=I 7 =/$] goto(I 4, L)=I 8 =/$] goto(I 4, *)=I 4 =/$] goto(I 4, id)=I 5 I 5: [L id • , =/$] I 10: [R L • , $] I 11: [L * • R, [R • L, [L • *R, [L • id, $] goto(I 11, R)=I 13 $] goto(I 11, L)=I 10 $] goto(I 11, *)=I 11 $] goto(I 11, id)=I 12: [L id • , $] I 13: [L *R • , $]

Example LR(1) Parsing Table 0 id * s 5 s 4 = 1 Grammar: 1. S’ S 2. S L = R 3. S R 4. L * R 5. L id 6. R L S L R 1 2 3 8 7 10 4 10 13 acc 2 s 6 3 4 $ r 6 r 3 s 5 s 4 5 r 5 6 s 12 s 11 7 r 4 8 r 6 9 r 2 10 r 6 11 s 12 s 11 12 r 5 13 r 4

LALR(1) Grammars n n n LR(1) parsing tables have many states LALR(1) parsing (Look-Ahead LR) combines LR(1) states to reduce table size Less powerful than LR(1) q q n Will not introduce shift-reduce conflicts, because shifts do not use lookaheads May introduce reduce-reduce conflicts, but seldom do so for grammars of programming languages SLR and LALR tables for a grammar always have the same number of states, and less than LR(1) tables. q Like C, SLR and LALR >100, LR(1) > 1000

Constructing LALR Parsing Tables n Two ways q Construction of the LALR parsing table from the sets of LR(1) items. n n q Union the states Requires much space and time Construction of the LALR parsing table from the sets of LR(0) items n n Efficient Use in practice.

Example state 0 ACTION c d GOTO $ s 36 s 47 1 S C 1 2 acc state 0 ACTION c d s 3 s 4 1 8 s 4 36 s 47 89 4 r 3 47 r 3 r 2 r 2 LALR(1) 2 5 s 3 89 1 s 7 3 r 1 C s 6 5 5 S 2 s 36 s 47 r 3 $ acc 2 r 3 GOTO 5 6 r 1 s 6 s 7 7 8 9 r 3 r 2 9 r 2 LR(1)

Constructing LALR(1) Parsing Tables n Construct sets of LR(1) items n Combine LR(1) sets with sets of items that share the same first part I 4: [L * • R, [R • L, [L • *R, [L • id, =] =] I 11: [L * • R, [R • L, [L • *R, [L • id, $] $] [L * • R, [R • L, [L • *R, [L • id, =/$] Shorthand for two items in the same set

Example LALR(1) Grammar n n n Unambiguous LR(1) grammar: S L=R|R L * R | id R L Augment with S’ S LALR(1) items (next slide)

I 0: [S’ • S, [S • L=R, [S • R, [L • *R, [L • id, [R • L, $] goto(I 0, S)=I 1 $] goto(I 0, L)=I 2 $] goto(I 0, R)=I 3 =/$] goto(I 0, *)=I 4 =/$] goto(I 0, id)=I 5 $] goto(I 0, L)=I 2 I 6: [S L= • R, [R • L, [L • *R, [L • id, $] goto(I 6, R)=I 8 $] goto(I 6, L)=I 9 $] goto(I 6, *)=I 4 $] goto(I 6, id)=I 5 I 7: [L *R • , =/$] I 1: [S’ S • , $] I 8: [S L=R • , $] I 2: [S L • =R, [R L • , $] goto(I 0, =)=I 6 $] I 9: [R L • , =/$] I 3: [S R • , $] I 4: [L * • R, [R • L, [L • *R, [L • id, =/$] goto(I 4, R)=I 7 =/$] goto(I 4, L)=I 9 =/$] goto(I 4, *)=I 4 =/$] goto(I 4, id)=I 5 I 5: [L id • , =/$] Shorthand for two items [R L • , =] $]

Example LALR(1) Parsing Table 0 Grammar: 1. S’ S 2. S L = R 3. S R 4. L * R 5. L id 6. R L id s 5 * s 4 = 1 s 6 3 7 s 5 R 3 9 7 9 8 r 6 s 4 r 5 s 5 r 5 s 4 r 4 8 9 L 2 r 3 5 6 S 1 acc 2 4 $ r 4 r 2 r 6

LL, SLR, LALR Summary n LL parse tables computed using FIRST/FOLLOW q q n LR parsing tables computed using closure/goto q q n Nonterminals productions Computed using FIRST/FOLLOW LR states terminals shift/reduce actions LR states nonterminals goto state transitions A grammar is q q LL(1) if its LL(1) parse table has no conflicts SLR if its SLR parse table has no conflicts LR(1) if its LR(1) parse table has no conflicts LALR(1) if its LALR(1) parse table has no conflicts

LL, SLR, LALR Grammars LR(1) LALR(1) LL(1) SLR LR(0)

YACC yacc specification yacc. y Yacc or Bison compiler y. tab. c C compiler a. out output stream input stream y. tab. c