THE CYK PARSING METHOD 2 Cornelia Kempa Carla

THE CYK PARSING METHOD (2) Cornelia Kempa Carla Parra Escartín WS 2006 -2007 Tübingen, 4 th December 2006

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. 4. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 2

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. 4. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 3

2. Undoing the effect of the CNF transformation 2. 1. Overview (1) Tübingen, 4 th December 2006 4

2. Undoing the effect of the CNF transformation 2. 1. Overview (2) Tübingen, 4 th December 2006 5

2. Undoing the effect of the CNF transformation 2. 1. Overview (3) Tübingen, 4 th December 2006 6

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. 4. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 7

2. Undoing the effect of the CNF transformation 2. 2. Adding removed elements (1) Tübingen, 4 th December 2006 8

2. Undoing the effect of the CNF transformation 2. 2. Adding removed elements (2) l CYK does not require all non-terminals reachable. + Rule added!!! Tübingen, 4 th December 2006 9

2. Undoing the effect of the CNF transformation 2. 2. Adding removed elements (3) New line Tübingen, 4 th December 2006 10

2. Undoing the effect of the CNF transformation 2. 2. Adding removed elements (4) l l Set Rsi, 0 non-terminals that can be considered as possibly occurring just in front of symbol zi Set Rsn +1, 0 non-terminals that can occur at the end of the sentence. Tübingen, 4 th December 2006 11

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. 4. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 12

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 1. Methodology (1) Derivation starts with start-symbol S. l Is A 1 A 2···Am a right hand side of S? l Does A 1 A 2···Am derive S 1, n? We check this starting with A 1 2 cases: Tübingen, 4 th December 2006 13

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 1. Methodology (2) A) A 1 is a terminal symbol it must be the first symbol of s 1, n then check if A 2···Am derives s 2, n-1 B) A 1 is a non-terminal symbol it must be a member of Rs 1, k then check if A 2···Am derives s 2, k+1, n-k Tübingen, 4 th December 2006 14

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 1. Methodology (3) l Now: determined whether the rule is applicable if so, which parts of the rule derive which substrings. Next step: determine how the substrings can be derived. Tübingen, 4 th December 2006 15

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. 4. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 16

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 2. Left-most derivation (1) Tübingen, 4 th December 2006 17

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 2. Left-most derivation (2) Tübingen, 4 th December 2006 18

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 2. Left-most derivation (3) This is what we saw so far: If we continue. . . The derivation is: Tübingen, 4 th December 2006 19

2. Undoing the effect of the CNF transformation 2. 3. Parsing 2. 3. 2. Left-most derivation (4) Tübingen, 4 th December 2006 20

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing Tübingen, 4 th December 2006 21

3. A short retrospective of CYK (1) First: recognition table using the original grammar. l Then: transforming grammar to CNF. l Tübingen, 4 th December 2006 22

3. A short retrospective of CYK (2) For efficiency, CNF is a bit too restrictive l Disadvantage of CNF: l • Resulting recognition table lacks the information we need to construct a derivation using the original grammar! Tübingen, 4 th December 2006 23

3. A short retrospective of CYK (3) In the transformation process, some non-terminals were thrown away (non-productive) l Missing information could be added. l Tübingen, 4 th December 2006 24

3. A short retrospective of CYK (2) l Result: almost the same recognition table. • Extra information on non-terminals • Obtained in a simpler and much more efficient way. Tübingen, 4 th December 2006 25

AGENDA 1. 2. Building the recognition table Undoing the effect of the CNF transformation 2. 1. Overview 2. 2. Adding removed elements 2. 3. Parsing 2. 3. 1. Methodology 2. 3. 2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing Tübingen, 4 th December 2006 26

4. Chart parsing (1) l l l CYK algorithm also known as chart parsing. Both techniques variants (some of them identical among the techniques) Difference: implementation. Tübingen, 4 th December 2006 27

4. Chart parsing (2) A chart is just a recognition table. Tübingen, 4 th December 2006 28

FINAL OVERVIEW (1) 1. 2. 3. 4. 5. Building the recognition table Possible Derivations Undoing the effect of the CNF transformation. A short retrospective of CYK Chart Parsing Tübingen, 4 th December 2006 29

FINAL OVERVIEW (2) PARSING METHODS Non-directional methods Top-down Bottom-up Unger Parser CYK Parser Tübingen, 4 th December 2006 30

THANK YOU VERY MUCH Tübingen, 4 th December 2006