The CYK Parsing Method Chiyo Hotani Tanya Petrova

The CYK Parsing Method Chiyo Hotani Tanya Petrova CL 2 Parsing Course 28 November, 2007

Overview l CYK Recognition with CF grammar Basic Algorithm ¡ Problems: unit-rules, є-rules ¡ Recognition with a grammar in CNF ¡ l CYK Parsing with CNF ¡ Recognition Table ¡ l l Chart Parsing Summary Advantages and Disadvantages ¡ Other remarks ¡

Basic Algorithm of CYK Recognition (1) Example Grammar: A grammar describing numbers in scientific notation Input: 32. 5 e+1

Basic Algorithm of CYK Recognition (2) Digit -> 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 Sign -> + | - derivations of substrings of length 1

Basic Algorithm of CYK Recognition (3) Number. S -> Integer | Real Integer -> Digit | Integer Digit -> 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 derivations of substrings of length 1 l Unit Rule: rules of the form A B, where A and B are non -terminals. We can have chains of them in a derivation.

Basic Algorithm of CYK Recognition (4) Number. S -> Integer | Real Integer -> Digit | Integer Digit Fraction ->. Integer Scale -> e Sign Integer | Empty

Basic Algorithm of CYK Recognition (5) Number. S -> Integer | Real -> Integer Fraction Scale Number does indeed derive 32. 5 e+1.

Basic Algorithm of CYK Recognition (6) є-rules

Basic Algorithm of CYK Recognition (7) l Rє = { Empty, Scale } l sentence: z = z 1 z 2. . . zn substring of z starting at position i, of length l. si, l = zizi+1. . . zi+l-1 l Rsi, l: the set of non-terminals deriving the substring si, l A graphical presentation of substrings

CYK recognition with a grammar in CNF l Required restrictions: ¡ Eliminate є-rules and unit rules ¡ Limit the maximum length of RHS of the rule to 2 l CNF ¡ No є-rules and unit rules ¡ all rules have one of the following two forms: A a A BC

Our example grammar in CNF

CYK Parsing with CNF l Building the recognition table l Input : Our example grammar in CNF input sentence: 32. 5 e + 1

CYK Parsing with the CNF l bottom-row : read directly from the grammar (rules of the form A a )

Two Ways to Copmute a R s i, l: l check each right-hand side l compute possible right-hand sides from the recognition table

How this is done Example: 2. 5 e ( = s 2, 4) 1) N 1 not in R s 2, 1 or R s 2, 2 N 1 is a member of R s 2, 3 But Scale´ is not a member of R s 5, 1 2) R s 2, 4 is the set of Non- Terminals that have a right-hand side AB where either: A in R s 2, 1 and B in R s 3, 3 A in R s 2, 2 and B in R s 4, 2 A in R s 2, 3 and B in R s 5, 1 Possible combinations: N 1 T 2 or Number T 2 In our grammar we do not have such a righthand side, so nothing is added to R s 2, 4.

Recognition table l i

As a result we find out that: l This process is much less complicated than the one we saw before

Reasons We do not have to repeat the process again and again until no new Non-Terminals are added to R s i, l (The substrings we are dealing with are really substrings and cannot be equal to the string we start with) • We only have to find one place where the substring must be split into two A B C • Here !

Chart Parsing A chart is just a recognition table.

A short retrospective of CYK l First: recognition table using the original grammar. l Then: transforming grammar to CNF.

A short retrospective of CYK cont. l CNF is useful for improving the efficiency, but it is actually a bit too restrictive l Disadvantage of CNF: ¡ Resulting recognition table lacks the information we need to construct a derivation using the original grammar!

A short retrospective of CYK cont. l In the transformation process, some non-terminals were thrown away (non-productive) l Missing information could be added.

A short retrospective of CYK cont. l Result: almost the same recognition table. ¡ Extra information on non-terminals ¡ Obtained in a simpler and much more efficient way.

Thank you for your attention!