Syntax Analysis Organization Introductory Ideas Top Down Parsing

Syntax Analysis

Organization Introductory Ideas Top Down Parsing Backtracking Top Down Parser Predictive Top Down Parser • Recursive Descent Parsing • Table driven predictive parser Bottom Up Parsing Operator Precedence Parsing LR Parsing

Introductory Ideas

The role of parser Source program Lexical Analyzer token Parser Parse tree get. Next Token Symbol table Rest of Intermediate Front End representation

Syntax Analysis Verifies if the tokens are properly sequenced in accordance with the Grammar of the Language. Grammar naturally describes the hierarchal structure of programming language. For ex -if else statement can have the form: -> If(expression) statement else statement If-else statement is the concatenation of keyword if, an opening parenthesis, an expression, a closing parenthesis , a statement, the keyword else, and other statement

Syntax Analysis

Specifying Legal Syntax The sequence of Tokens that are legal in a Programming Language are specified by Context Free Grammar (CFG). stmt-> if(expr) stmt else stmt Such rule are called as production rule. Elements like keyword If and the paranthesis are called terminals. Variables like expr and stnt represent sequence of terminals and are called non-termianls

Production of a grammer specify the manner in which terminals and non-terminals can be combined to form strings. Each production consist of: A non-terminal called the head or left side of the production. The symbol ->. Sometimes : : =has been used in place of the array. A body or right side consisting of zero or more terminals and non-terminals.

Language of Grammar Word of grammar is generated by: Productions Derivation of string starts from Start symbol and repeatedly replacing a non-terminal by the body of production for that non-terminal. Final state should contain terminals

Parsing Taking a string of terminals and figuring out how to derive it from the start symbol of the grammar. If string can not be derived from the start symbol of the grammar , then syntax error is reported.

Derivations are represented by: a) Sentential Form (Using Productions) b) Parse Tree (rep. by tree) a) b) Recursion Grouping of Symbols

Grammars E -> E+E | E*E | id T={id, +, *} N={E} S={E} Derivation: a) Right Most Derivation (rightmost non-terminal is always chosen also called canonical derivation) b) Left Most Derivation( leftmost non-terminal is always chosen)

Parse tree -(id+id)

Grammars (Example) Given Production is: E -> E+E | E*E | id Derive string : id+id*id using a) Right Most Derivation b) Left Most Derivation

Grammars (Example)

Grammars (Example)

Grammars (Example) Given Production is: E-> A 1 B A -> 0 A | epsilon B-> 0 B | 1 B | epsilon Derive string : 00101 using a) Right Most Derivation b) Left Most Derivation

Grammars (Example)

Parse Tree Given Production is: E -> E+E | E*E | id Draw Parse tree to derive string : id+id*id

Parse Tree Given Production is: E-> A 1 B A -> 0 A | epsilon B-> 0 B | 1 B | Epsilon Draw Parse tree to derive string : 00101

Parse Tree

Example: Parse Tree Given Production is: S-> a. S | Sa |a Draw Parse tree to derive string : aa

Example: Parse Tree Given Production is: S-> a. Sb. S | b. Sa. S | epsilon Draw Parse tree to derive string : abab

Parse Tree: Example Given Production is: R-> R+R | RR| R* |a|b|c Draw Parse tree to derive string : a+bc Given the production is: s -> ss+|ss*|a String is: aa+a* a) Give the left most derivation b) Give the right most derivation c) Give the parse tree

s-> 0 s 1|01 with string 000111 s-> +ss|*ss|a with string +*aaa s -> s(s)s|�� with string (()()) s->s+s|ss|(s)|s*|a with string (a+a)*a

Ambiguity For some strings grammar generate more than one parse tree Or more than one leftmost derivation Or more than one rightmost derivation Example: id+id*id

Associativity and Precedence

Associativity Because we are not defining grammar in any order. Left most element on RHS = LHS For associativity, grammar should be left recursive.

Precedence High precedence expression should be evaluated first Should be at lower level of tree Replace the non terminal with another non terminal

Precedence If * , + , Operator which is close to Start symbol Low Precedence

Example: unambiguous grammar

Example: unambiguous grammar

Example: unambiguous grammar

Example: unambiguous grammar * -> left associative { *. >*} + -> right associative { +<. +} - -> both left and right associative. - has highest precedence * and + has equal precedence. So grammar is unambiguous

Recursion Left recursion: -> Left most variable of R. H. S is equal to L. H. S A -> A α Right recursion : -> Right most variable of R. H. S is equal to L. H. S A -> α A

To remove left recursion: A -> A α|β A -> β A’ A’ -> �� |αA’

Examples: S -> Aa|b A ->Ac | Sd| ��

S -> Aa|a A -> Sb|b

A-> AB |Aab |BA| a B->Bb |Aa| b

Left Factoring Non-deterministic grammar: A -> ���� 1| ���� 2| ���� 3……. • Decision is taken on the basis of ��. • Grammar is non-deterministic • It is called left-factoring, we need to remove left factoring

To remove left factoring we need to postpone the decision till �� is encountered A -> �� A’ A’ -> �� 1|�� 2|�� 3……

S -> i. Et. S|i. Et. Se. S|a E->b S ->a. SSb. S|a. Sb|abb|b S-> b. SSaa. S|b. SSa. Sb|b. Sb|a E -> T+E|T T->int|int*T|(E) E-> int|int+E|int-E

S-> a|ab|abcd S->a. Ad|a. B A->a|ad B->ccd|ddc

Error Handling Common programming errors Lexical errors Syntactic errors Semantic errors Lexical errors Error handler goals Report the presence of errors clearly and accurately Recover from each error quickly enough to detect subsequent errors Add minimal overhead to the processing of correct programs

Error-recover strategies Panic mode recovery Discard input symbol one at a time until one of designated set of synchronization tokens is found Phrase level recovery Replacing a prefix of remaining input by some string that allows the parser to continue Error productions add to the grammar, productions that generate the erroneous constructs Global correction Choosing minimal sequence of changes to obtain a globally least-cost correction

Derivation

Parse Tree A Parse Tree is a graphical representation of a derivation

Benefits of CFG A parser can be generated from the CFG by using parser generator tools like yacc

Error Reporting and Recovery Report errors in the input source program and emit an informative diagnostic message – Error Reporting Offending line number should be emitted Information to help the programmer correct the mistake Recovering from the first error and continuing to process the rest of the input source program - Error Recovery Spurious errors should not be reported Not get into a bind on a totally unexpected input Error repair

Parsing Techniques

Top Down Parsing

Top Down Parsing Left most derivation for an input string. Constructing a parse tree for the input starting from the root (start symbol) Creating the nodes of the parse tree in preorder Implementation of Top Down Parsing Backtracking Parser Predictive Parsing • Recursive Descent Parsing • Table Driven Parsing

Backtracking Top Down Parser Begin with the start symbol and repeatedly apply productions until we get the desired string If this choice leads to a dead-end, the parser would have to backtrack to that decision point moving backwards through the input and start again making a different choice. This goes on until the parser ends up with the appropriate production for matching the complete input or it runs out of choices.

Backtracking Top Down Parser Advantages Can cope up with any kind of grammar Disadvantages Slow due to the combinatorial explosion Undo the semantic effects of erroneous expansions on backtracking is difficult the order in which alternates of production rules are tried can also affect the language considered difficulty in error reporting

Predictive Top Down Parser Current Non terminal being processed combined with the next input symbol can guide the parser to take the correct production rule Does not require backtracking Grammar for the language is transformed such that backtracking is not needed Elimination of Left Recursion Left factoring

Predictive Top Down Parser A Predictive Top Down Parser can be implemented as Recursive descent Parser Table driven predictive parser

Recursive Descent Parser Basic Idea Collection of procedures one for each nonterminal Advantages Intuitive, easy to code from grammar Disadvantages large stack required due to the recursion of the procedures cannot be automatically generated easily from the grammar compared to table driven approaches Difficult to Print a good error recovery message

Table Driven Predictive Parser

Table Driven Predictive Parser Advantages Easy to generate a Table driven parser from a given grammar. Parsing program is independent of the grammar The parsing table depends on the grammar and can be generated by using the FIRST and FOLLOW set. The error recovery and reporting done by having entries in the table, pointing to the error recovery and reporting routines. Disadvantages A predictive parser can work on LL(1) grammars only.

Table Driven Predictive Parser Automatically generate the parsing table entries from the grammar Parsing table can be generated using FIRST and FOLLOW sets

Error Detection Error detection in Table Driven Parser When the terminal on the top of the stack does not match the next input symbol When a nonterminal A is on the top of stack, a is the next input symbol and the parsing table entry M[A, a] is empty.

Error Recovery in Table Driven Predictive Parser Panic Mode Recovery Phrase Level Recovery Panic mode recovery The parser should skip the input symbols until it finds a synchronizing token in the input Phrase level recovery Filling in blank entries in the predictive parsing table with pointers to error handling routines

Bottom Up Parsing

Bottom Up Parsing The parse tree for an input string is constructed beginning at the leaves and working up towards the root “Reducing” an input string ‘w’ to the start symbol of the grammar. Derivation is exactly the reverse of a right most derivation starting from the root Eliminate right recursion to make it suitable for bottom up parsing

Basics of Bottom Up Parsing Reduction Handle Right Sentential Form The need for identifying the correct handle out of all possible handles (handle pruning) The choice of which production rule to apply, in order to reduce a handle is very important to have the correct outcome of parsing

Bottom Up Parsing Bottom-up parsing is commonly implemented by using a stack. Depending on the content of the stack and the next input symbol, the actions of a bottom up parser can be Shift/Reduce/Accept/Error Also called as “Shift-Reduce” Parsing due to the Shifting and Reducing on the stack. The parsers generated by bison/yacc are bottom up parsers implemented using a stack

Bottom Up Parsing Methods The challenges in a Bottom up parser are Mechanism to identify the correct handle for reducing. What production needs to be used for reducing Two methods of shift-reduce parsing, which use different techniques to resolve the above issues Operator Precedence Parsing Method LR Parsing Method

Bottom Up Parsing Methods Operator Precedence Parsing Applied to a small class of grammar called as Operator Precedence Grammar • No Epsilon Production • No production would contain two adjacent terminals LR Parsing Applied to a class of grammar called LR grammar LR Grammar • No serious restrictions on the productions. • Can parse most of the programming constructs

Operator Precedence Parsing No ε productions in this type of grammar No Production would have two adjacent nonterminals Uses a Table called as Operator Precedence Relations table for making informed decisions with regard to identifying the correct handle during a reduction step Possible to define precedence functions f and g, and replace the Table in case it gets large

Operator Precedence Parsing

Operator Precedence Parsing Error Reporting No Precedence relation exists between the top of the stack and the current input The handle emerging out of the popped elements does not form the RHS of a production Error Recovery Build intelligence in both of these points to indicate not only the error but also the source of error

LR Parsing Table is used to detect correct handles and make informed shift/reduce decisions The parsing table is used to figure out whether a shift or a reduce needs to be done on the receipt of a given input In case of a reduce operation, it tells us which production needs to be used for it.

LR Parsing

LR Parsing Table Construction 3 methods for automatically constructing the parsing table from the grammar. Simple LR or SLR(1) Method Succeeds in constructing the parsing table for a subset of LR grammar called SLR(1) Canonical LR or LR(1) Method Succeeds in constructing a parsing table for a large class of grammars called as LR(1) Grammars Look ahead LR or LALR(1) Method Succeeds in constructing a parsing table for a subset of LR(1) grammar called as LALR(1) grammar

The Steps The steps in Constructing a LR Parsing Table from the Grammar G

Basic Concepts Augmented Grammar LR(0) Item Closure Operation goto Operation LR(1) Item

Comparison

Error Detection in LR Parsing Detect an error, when there is no entry for the given state, input symbol in action table Errors can never be detected by consulting the goto table A Canonical LR parser will not make a single reduction before announcing an error. The SLR and LALR parsers can make several reductions before detecting an error, but will not make a single shift before detecting an error.

Error Recovery in LR Parsing Panic mode recovery Scans down the stack until it finds a state S for which a valid goto[S][A] entry is found, where A is any Nonterminal Input symbols are discarded until a element b in FOLLOW(A) is encountered. The elements in FOLLOW(A) act as the synchronization token set. The parser then stacks the state goto[S][b] and continues the parsing A can be any nonterminal. Multiple choices for A are pruned for the particular language.

Error Recovery in LR Parsing Phrase level recovery Appropriate error handling routines in the action table. Error-handling routines would typically modify the top of the stack or the next input symbol to recover from the error Ensure that the error handling routines do not get the parser to go into an infinite loop