Predictive Parsing Find derivation for an input string

Predictive Parsing Find derivation for an input string, l Build a abstract syntax tree (AST) l – a representation of the parsed program l Build a symbol table – Describe the program objects Type l Location l Scope and access l l Munge the AST – Optimize – Modify Subscript checking l Communication interactions l l Generate executable code

Terminology l Derivation step Sequence starting with the start symbol S and proceeding through a sequence of derivation steps A non-terminal on the left side of a derivation is replaced by the right hand side of a production rule in the next step of the derivation A if A , is a production, and are arbitrary strings of grammar symbols l Sentential Form l Sentence Any derivation step Sentential form with no non-terminals

Derivation example l Grammar E E + E | E * E | ( E ) | - E | id l Simple derivation l Derivation of -(id+id) E - E E -(E) -(E+E) -(id+id) E -(id+id) 1 2 … n derives it or simply Select a nonterminal to replace and an alternative at each step of a derivation

Leftmost Derivation l The derivation E -(E) -(E+E) -(id+id) is leftmost which we designate as E lm -(E) lm -(E+E) lm -(id+id) A sentential form is a derivation step. l A leftmost derivation step is a left sentential form, for example: l (Denoted *lm for typographical convenience) l

Leftmost Derivation A derivation in which only the leftmost nonterminal in any sentential form is replaced at each step. l Unique derivation for a string most of the time l

Rightmost Derivation The rightmost non-terminal is replaced in the derivation process in each step. l Also referred to as Canonical Derivation l Right sentential form: a sentential form produced via a rightmost derivation l – If S * , then is a sentential form of the CFG – If S *rm , then is a right sentential form

Right Sentential Form Grammar: S a. ABe A Abc | e B d l l Reduce abbcde to S by four steps abbcde a. Ade a. ABe S In reverse, it is S rm a. ABe rm a. Ade rm a. Abcde rm abbcde – abbcde is a right-sentential form (replace b in position 2 by A) – a. Abcde is a right-sentential form (replace Abc in position 2 by A)

Top Down Parsing

Top Down l For a given string, builds a parse tree from the start symbol of the grammar, and grows towards the roots Suitable grammar is LL(k) l Recursive descent parsing l Predictive parsing l – Leftmost processing, leftmost derivation, look at most k symbols ahead – Should be left-factored, and without immediate left recursion – May backtrack, for certain grammars – Good error-detection and handling capabilities – No backtracking – Given the partial derivation and the leading terminal, exactly one production rule is applicable – Parsers include l l Hands written recursive descent Table-driven LL(k) parser (for later)

Two: Operator Precedence

Operator Precedence l Operator grammar no right side of a production contains adjacent nonterminals S Eo. E | id , o + | - | * | / (X) l If a grammar is an operator grammar and it has no productions with null of the RHS, then there is a operator-precedence parser for that grammar l Special case of a shift-reduce parser l

Precedence Grammars l l l Parse with shift/reduce No production right side is , and no right side has two adjacent nonterminals bad multi-precedence operator (-) difficult can’t be sure parser matches the grammar! only works for some grammars l l good simple, simple Build on non-reflexive precedence relations that we denote as. > , , <. (typographical convenience for dotted forms of <, =, > as in text)

Computing Precedence l Precedence is disjoint. Can have a <. b and a. > b c <. b, c. > b is read “yields precedence” or “equal precedence” Obtain precedence by manual assignment using traditional associative and precedence rules, or mechanically from nonambiguous grammar l How to process l – “Ignore” nonterminals, and then delimit handle from right side. > and then back up to the left side <.

Operator Precedence Parser l Remove (hide) nonterminals and place precedence relationship between pairs of terminals (1) Represent id + id * id as $ <. id. > + <. id. > * <. id. > $ (2) Apply scanning method a) scan from left toward right to find. > b) backwards scan to <. or $ c) handle is located between start and end of scan d) reduce the handle to the corresponding nonterminal l Relies on the grammar’s special form (1) In grammar rule, no adjacent nonterminals on the right hand-side (by definition), so no right sentential form will have two non-terminals (2) Form is 0 a 1 1. . . an n i is nonterminal or ai is a nonterminal

Bottom Up Parsing (shift-reduce)

Bottom Up l What – For a given string, builds a parse tree starting at the leaves and reaches the root. – Known as shift-reduce parsing – Rightmost derivation in reverse l How – Start with the given input string (I. e. , program) – Find a substring that matches the right side of a production – Substitute left-side of grammar rule when right-side matches l Terminology – A handle is a substring that matches the right hand side of a production – Handles will be reduced to the left-hand side of the matching production

Handles l Handle – Substring that matches the right side of a production – When reduced (to the left side) , this represents one step along the reverse of a rightmost derivation l Handle Pruning – Replacing a handle by the left hand side

Implementation l Stack is a good data type to implement a shift -reduce parser – The stack contains the grammar symbols Nonterminals recognized from the input l Terminals not yet recognized as belonging to any production l – An input tape contains the input string l The parser – Shift next input symbol from the input tape and into the stack – Keep shifting until a handle appears at top of stack – Reduce when a handle appears on top of stack – Terminate when the stack contains S and the input tape is null

LR(k) Grammar LR grammar if it can be recognized by a bottom-up parser on a left to right scan recognize handles when they appear on the top of the stack l LR(k) – left to right scanning l LR(k) -- rightmost derivation in reverse l Supports non-backtracking shift-reduce parsing method. Deterministic parsing! l Too hard to construct parse-table by hand (so use a parser-generator) l

LR Parsing Algorithm Parsing program l Parsing table l Input tape l Stack l Output tape l

Parsing Table Action[] What to do when a symbol appears on the tape Action[State, input symbol] Shift, reduce, accept, reject Goto[] A finite automaton that can recognize a handle on the top of the stack by scanning the stack top to bottom Goto[State, grammar symbol] State

Whats so great about LR Grammars? “The LR requirement is that we be able to recognize the occurrence of the right side of a production, having seen what is derived from that side. This is far less stringent than the requirement for predictive parsing, namely that we be able to recognize the apparent use of the production seeing only the first symbol that it derives” [AU 77: 202] l The state symbol on top of the stack contains all the information the parser needs l

The Underlying Basis of LR (a Lasting Relationship) l l l Whereas the nondeterministic n. PDA is equivalent to the CFG, the deterministic d. PDA is only equivalent to a subclass of deterministic CFL Every LR(k) grammar generates a deterministic CFL Every deterministic CFL has an LR(1) grammar No ambiguous grammar can be LR(k) for any k However, it remains undecidable whether a CFG G is ambiguous

LR Parsing Algorithm A Parsing Program l A Parsing Table l An input Tape l A stack l An output Tape l

Parsing Table l Action – (State, input symbol) – Shift, reduce, accept, reject l Goto – A finite automaton that can recognize a handle on the top of the stack by scanning the stack top to bottom – (State, grammar symbol) – State

LR Algorithm l action[sm, ai] {shift s, reduce a , accept, error } l goto[s, a] takes state and grammar symbol and produces a state It is the transition function of a DFA on viable prefixes of G Viable prefix is a prefix that can appear on the stack during a rightmost derivation l Progresses through configurations Configuration – right sentential forms with states intermixed Written as pairs – (stack contents, unexpended input) l To obtain the next move – read a (current input) – Look at sm (state at top-of-stack) – consult action[]

LR Actions action[sm, a] = shift s execute a shift move: (s 0 X 1 s 1 X 2 s 2. . Xmsm, ai ai+1. . an$) shifted current input ai and next state s off of input and onto stack (s 0 X 1 s 1 X 2 s 2. . Xmsmais, ai+1. . an$) action[sm, a] = reduce a execute a reduce move: (s 0 X 1 s 1 X 2 s 2. . Xmsm, ai ai+1. . an$) popped 2 r symbols off (r for state, r for grammar) pushed A and s onto stack no change in input (s 0 X 1 s 1 X 2 s 2. . Xm-r sm-r A s, ai ai+1. . an$) where s=goto[sm-r, A], and r is length of (rule right-hand side) See example 4. 33, pages 218 -220 for detail

Construction of LR parser table SLR (Simple LR) l LALR (Look Ahead LR) l Canonical LR l

Yet Another Compiler Parser Generator l Converts a Context Free Grammar into a set of tables for an automaton. l Generates LALR parser l This automaton executes the LALR(1) parser algo. l