Chap 2 TOPDOWN PARSING We introduce the basic

Chap 2. TOP-DOWN PARSING We introduce the basic ideas behind top-down parsing and show to construct an efficient non-backtracking form of top-down parsing called a predictive parser. We define the class of LL(1) grammars from which predictive parsers can be constructed automatically. Top-down parsing cannot be performed on left recursive grammars. 1

Recursive-Descent Parsing Top-down parsing can be viewed as an attempt to find a leftmost derivation for an input string. We can also view it as an attempt to construct a parse tree for the input starting from the root and creating the nodes of the parse tree in preorder. Let's consider a general form of top-down parsing, called recursive descent, that may involve backtracking, that is, making repeated scans of the input. However, backtracking parsers are not seen frequently, since backtracking is rarely needed to parse programming 2 language constructs.

Example Consider the grammar S c. Ad A ab|a and the input string w = cad. S S c A d c A d a b a 3

Elimination of Left Recursion A grammar is immediately left recursive if it has a nonterminal A such that there is a production A A for some string . In order to be able to parse in the top-down way, left recursion must be eliminated. An immediate left recursion can be eliminated very easily. Let's take a production E E+T. Suppose the procedure for E decides to apply this production. The right side begins with E so the procedure for E is called recursively, and the parser loops forever. A left recursive production can be eliminated by rewriting the offending production. 4

Consider a nonterminal A with two productions A A | where and are sequences of terminals and nonterminals that do not start with A. For example, in E E+T | T A=E, =+T, and =T. Repeated application of the production A A builds up a sequence of 's to the right of A. A A . . …… 5

When A is finally replaced by , we have a followed by a sequence of 0 or more 's. The same effect can be achieved, by rewriting the productions for A in the following manner. A R R R | A R R ………… R 6

Here R is a new nonterminal. The production R R is right recursive, because this production for R has R itself as the last symbol on the right side. Right recursive productions lead to trees that grow down towards the right. 7

Example Consider the following grammar for arithmetic expressions. E E + T | T T T * F | F F (E) | id. Eliminating the immediate left recursion from the productions for E and then for T, we obtain E TE' E' + TE' | T FT' T' * FT' | F (E) | id. 8

No matter how many A-productions there are, we can eliminate immediate left recursion from them A A 1 | A 2 | …| A m | 1 | 2 | …| n where no i begins with an A. Then, we replace the Aproductions by A 1 A' | 2 A' | … | n. A' A' 1 A' | 2 A' | … | m. A' | The nonterminal A generates the same strings as before but is no longer left recursive. 9

Nonrecursive Predictive Parsing It is possible to build a nonrecursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls. The key problem during predictive parsing is that of determining the production to be applied for a nonterminal. The nonrecursive parser looks up the production to be applied in a parsing table. In what follows, we shall see how the table can be constructed directly from certain grammars. 10

Input a + b $ Output Predictive Parsing Program Stack X Y Z $ Parsing Table M 11

A table-driven predictive parser has an input buffer, a stack, a parsing table, and an output stream. The input buffer contains the string to be parsed, followed by $, a symbol used as a right endmarker to indicate the end of the input string. The stack contains a sequence of grammar symbols with $ on the bottom, indicating the bottom of the stack. Initially, the stack contains the start symbol of the grammar on top of $. The parsing table is a two-dimensional array M [A, a], where A is a nonterminal, and a is a terminal or the symbol $. 12

The parser is controlled by a program that behaves as follows. The program considers X, the symbol on top of the stack, and a, the current input symbol. 1. If X = a = $, the parser halts and announces successful completion of parsing. 2. If X = a $, the parser pops X off the stack and advances the input pointer to the next input symbol. 3. If X is a nonterminal, the program consults entry M [A, a] of the parsing table M. This entry will be either an X-production of the grammar or an error entry. If, for example, M[X, a] = {X UVW}, the parser replaces X on top of the stack by WVU (with U on top). As output, we shall assume that the parser just prints the production used; any other code could be executed here. If M[X, a] = error, the parser calls an error recovery routine. 13

Algorithm for Nonrecursive predictive parsing. Input. A string w and a parsing table M for grammar G. Output. If w is in L(G), a leftmost derivation of w, otherwise, an error indication. Method. Initially, the parser is in a configuration in which it has $S on the stack with S, the start symbol of G on top, and w$ in the input buffer. 14

set ip to point to the first symbol of w$; repeat let X be the top stack symbol and a the symbol pointed to by ip; if X is a terminal or $ then if X = a then pop X from the stack and advance ip else error () else /* X is a nonterminal */ if M[X, a] = X Y 1 Y 2 Yk then begin pop X from the stack; push Yk , Yk– 1 , . . . , Y 1 onto the stack, with Y 1 on top; output the production X Y 1 Y 2 Yk end else error () until X = $ /* stack is empty */ 15

Example Consider the grammar for arithmetic expression 16

Input Sumbol Nonterminal id + * ( ) $ E E TE' E’ ε T T FT' T’ T’ ε T ‘ *FT' T’ ε F F id F (E) E’ +TE' 17

With input id + id * id the predictive parser makes the sequence of moves. The input pointer points to the leftmost symbol of the string in the input column. If we observe the actions of this parser carefully, we see that it is tracing out a leftmost derivation for the input, that is, the productions output are those of a leftmost derivation. 18

FIRST and FOLLOW The construction of a predictive parser is aided by two functions associated with a grammar G. These functions, FIRST and FOLLOW, allow us to fill in the entries of a predictive parsing table for G, whenever possible. If α is any string of grammar symbols, let first(α) be the set of terminals that begin the strings derived from α. If α , then is also in FIRST(α). Define FOLLOW (A), for nonterminal A, to be the set of terminals a that can appear immediately to the right of A in some sentential form, that is the set of terminals a such that there exists a derivation of the form S j. Aaβ for some α and β. 19

1. 2. If X is terminal, then FIRST(X) is {X}. If X is a production, then add to FIRST(X). 3. If X is nonterminal and X — Y 1 Y 2 • • • Yk is a production, then place a in FIRST(X) if for some i, a is in FIRST(Yi), and is in all of FIRST(Y 1), . . FIRST(Yi– 1); that is, Y 1 • • • Yi– 1 . If is in FIRST(Yj) for all j = 1, 2, . . k, then add to FIRST(X). For example, everything in FIRST(Y 1) is surely in FIRST(X). If Y 1 does not derive , then we add nothing more to FIRST(X), but if Y 1 . , then we add FIRST(Y 2) and so on. 20

Now, we can compute FIRST for any string X 1 X 2 • • • Xn, as follows. Add to FIRST(X 1 X 2 • • • Xn) all the non- symbols of FIRST(X 1). Also add the non- symbols of FIRST(X 2) if is in FIRST(X 1), the non- symbols of FIRST(X 3) if is in both FIRST(X 1) and FIRST(X 2), and so on. Finally, add to FIRST(X 1 X 2 • • • Xn) if, for all i, FIRST(Xi) contains . 21

To compute FOLLOW(A) for all nonterminals A, apply the following rules until nothing can be added to any FOLLOW set. 1. Place $ in FOLLOW(S), where S is the start symbol and $ is the input right endmarker. 2. If there is a production A , then everything in FIRST( ) except for is placed in FOLLOW(B). 3. If there is a production A , or a production A , where FIRST( ) contains (i. e. , ), then everything in FOLLOW(A) is in FOLLOW(B). 22

Example E TE' E' +TE' | T FT' T' *FT' | F (E) | id Then FIRST(E) = FIRST(T) = FIRST(F) = {(, id} FIRST(E') = {+, } FIRST(T') = {*, } FOLLOW(E) = FOLLOW(E') = {), $} FOLLOW(F) = {+, *, ), $} 23

Construction of Predictive Parsing Tables Algorithm. Construction of a predictive parsing table Input. Grammar G Output. Parsing table M Method 1. For each production A of the grammar, do steps 2 and 3. 2. For each terminal a in FIRST( ), add A to M[A, a] 3. If is in FIRST( ) for each a FOLLOW(A), add A to M[A, a]. 4. Make each undefined entry of M be error. 24

Example Let's apply the algorithm above to the arithmetic expression's grammar. Since FIRST(TE') = FIRST(T) = {(, id}, production E TE' causes M[E, (] and M[E, id] to acquire the entry E TE'. Production E' TE' causes M[E', )] and M[E', $] to acquire E' since FOLLOW(E') = {), $}. We will get the table that we have already seen. 25

LL(1) Grammars If G is left recursive or ambiguous, then M will have at least one multiplydefined entry. Example Let's consider a grammar : S i. Et. SS' | a S' e. S | E b FIRST(S) = {i, a} FIRST(S') = {e, } FIRST(E) = {b} FOLLOW(S) = FOLLOW(S') ={$, e} FOLLOW(E) = {t} 26

Nonter minal Input Symbol a b e i t $ S S a S i. Et. SS' S' S' e. S S' E E b 27

The entry M[S', e] contains both S' and S' e. S, since FOLLOW(S') = {e, $}. The grammar is ambiguous and the ambiguity is manifested by a choice in what production to use when an e (else) is seen. 28

A grammar whose parsing table has no multiplydefined entries is LL(1). The first "L" is for scanning the input from left to right. The second "L" for producing the leftmost derivation. The "1" for using one input symbol of lookahead at each step to make parsing action decisions. 29