Parsing I Contextfree grammars and issues for parsers

Parsing I Context-free grammars and issues for parsers

Bibliography • More or less all books on CL or NLP will have chapters on parsing, and some may be all or mostly about parsing • Many are written for computer scientists – They explain linguistic things like POSs and PS grammars – They go into detail about implementation (eg talk of Earley’s algorithm, shiftreduce parsers) D Jurafsky & JH Martin Speech and Language Processing, Upper Saddle River NJ (2000): Prentice Hall. Chs 9 & 10 RM Kaplan ‘Syntax’ = Ch 4 of R Mitkov (ed) The Oxford Handbook of Computational Linguistics, Oxford (2003): OUP J Allen Natural Language Understanding (2 nd ed) (1994): Addison Wesley 2

Parsing • Bedrock of (almost) all NLP • Familiar from linguistics • But issues for NLP are practicalities rather than universality – Implementation – Grammar writing – Interplay with lexicons – Suitability of representation (what do the trees show? ) 3

Basic issues • Top-down vs. bottom-up • Handling ambiguity – Lexical ambiguity – Structural ambiguity • Breadth first vs. depth first • Handling recursive rules • Handling empty rules 4

Some terminology • Rules written A B c o Terminal vs. non-terminal symbols o Left-hand side (head): always non-terminal o Right-hand side (body): can be mix of terminal and non-terminal, any number of them o Unique start symbol (usually S) o ‘ ’ “rewrites as”, but is not directional (an “=” sign would be better) 5

1. Top-down with simple grammar the man shot an elephant S NP VP S NP det n NP det {an, the} n {elephant, man} VP v NP v shot VP det n v the man shot S NP VP NP det n VP v NP Lexicon det {an, the} n {elephant, man} v shot No more rules, but input is not completely accounted for… So we must backtrack, and try the other VP rule 6

1. Top-down with simple grammar S NP VP NP det n VP v NP the man shot an elephant S NP VP S Lexicon det {an, the} n {elephant, man} v shot NP det n NP det {an, the} n {elephant, man} VP v NP v shot det the VP n v NP man shot det n NP det n det {an, the} n {elephant, man} an elephant No more rules, and input is completely accounted for 7

Breadth-first vs depth-first (1) • When we came to the VP rule we were faced with a choice of two rules • “Depth-first” means following the first choice through to the end • “Breadth-first” means keeping all your options open • We’ll see this distinction more clearly later, • And also see that it is quite significant 8

2. Bottom-up with simple grammar S NP VP NP det n VP v NP S det {an, the} n {elephant, man} v shot VP S NP det n NP VP Lexicon det {an, the} n {elephant, man} v shot NP VP VP vv NP SS NP NPVP VP det the n v det n man shot an elephant We’ve reached the top, but input is not completely accounted for… We’ve reached the top, and input is completely accounted for So we must backtrack, and try the other VP rule 9

Same again but with lexical ambiguity S NP VP NP det n VP v NP Lexicon det {an, the} n {elephant, man, shot} v shot can be v or n 10

3. Top-down with lexical ambiguity S NP VP NP det n VP v NP the man shot an elephant S NP VP S NP det n NP det {an, the} n {elephant, man} VP v NP det the Same as before: at this point, we are looking for a v, and shot fits the bill; the n reading never comes into play VP n v NP man shot det an Lexicon det {an, the} n {elephant, man, shot} v shot n elephant 11

4. Bottom-up with lexical ambiguity det {an, the} n {elephant, man, shot} v shot S NP VP NP det n VP v NP S NP det n VP v S Lexicon det {an, the} n {elephant, man, shot} v shot VP VP v NP S NP VP NP Terminology: graph nodes arcs (edges) NP VP n det the n v man shot det an n elephant 12

4. Bottom-up with lexical ambiguity S NP VP NP det n VP v NP Let’s get rid of all the unused arcs S S NP Lexicon det {an, the} n {elephant, man, shot} v shot VP NP VP n det the n v man shot det an n elephant 13

4. Bottom-up with lexical ambiguity S NP VP NP det n VP v NP Let’s get rid of all the unused arcs S Lexicon det {an, the} n {elephant, man, shot} v shot VP NP NP det the n v man shot det an n elephant 14

4. Bottom-up with lexical ambiguity And let’s clear away all the arcs… S NP VP NP det n VP v NP S Lexicon det {an, the} n {elephant, man, shot} v shot VP NP NP det the n v man shot det an n elephant 15

4. Bottom-up with lexical ambiguity And let’s clear away all the arcs… S NP VP NP det n VP v NP S Lexicon det {an, the} n {elephant, man, shot} v shot VP NP NP det the n v man shot det an n elephant 16

Breadth-first vs depth-first (2) • In chart parsing, the distinction is more clear cut: • At any point there may be a choice of things to do: which arcs to develop • Breadth-first vs. depth-first can be seen as what order they are done in • Queue (FIFO = breadth-first) vs. stack (LIFO= depth-first) 17

Same again but with structural ambiguity the man shot an elephant in his pyjamas S NP VP NP det n PP VP v NP PP PP prep NP S NP det the VP n v man shot NP det n an elephant PP prep in NP det n his pyjamas Lexicon det {an, the, his} n {elephant, man, shot, pyjamas} v shot prep in We introduce a PP rule in two places 18

Same again but with structural ambiguity the man shot an elephant in his pyjamas S NP VP NP det n PP VP v NP PP PP prep NP S NP det the VP n v man shot NP det n an elephant PP prep in NP det n his pyjamas Lexicon det {an, the, his} n {elephant, man, shot, pyjamas} v shot prep in We introduce a PP rule in two places 19

5. Top-down with structural ambiguity the man shot an elephant in his pyjamas S NP VP NP det n PP det {an, the, his} n {elephant, man, shot, pyjamas} S NP det VP n PP prep NP prep in the man The next word, shot, isn’t a prep, So this rule simply fails S NP VP NP det n PP VP v NP PP PP prep NP PP prep NP At this point, depending on our strategy (breadth-first vs. depth-first) we may consider the NP complete and look for the VP, or we may try the second NP rule. Let’s see what happens in the latter case. 20

5. Top-down with structural ambiguity the man shot an elephant in his pyjamas S NP VP NP det n PP det {an, the, his} S NP VP S NP VP NP det n PP VP v NP PP PP prep NP n {elephant, man, shot, pyjamas} det n v v NP VP v NP PP the man shot det n v shot NP det n an elephant NP det n PP det {an, the, his} As before, the first VP rule works, n {elephant, man, But does not account for all the input. shot, pyjamas} Similarly, if we try the second VP rule, and the first NP rule … 21

5. Top-down with structural ambiguity the man shot an elephant in his pyjamas S NP VP NP det n PP det {an, the, his} n {elephant, man, shot, pyjamas} det VP v NP PP the v shot NP det n PP S NP VP NP det n PP VP v NP PP PP prep NP S NP VP n v v man shot det an NP Depth-first: it’s a stack, LIFO n Breadth-first: it’s a queue, FIFO elephant So what do we try next? This? Or this? 22

5. Top-down with structural ambiguity (depth-first) the man shot an elephant in his pyjamas S NP VP NP det n PP det {an, the, his} S NP VP S NP VP NP det n PP VP v NP PP PP prep NP n {elephant, man, shot, pyjamas} det n v NP VP v NP PP PP the man shot det n v shot NP det n NP an elephant prep NP det n PP det {an, the, his} n {elephant, man, shot, pyjamas} in his pyjamas PP prep NP prep in 23

5. Top-down with structural ambiguity (breadth-first) the man shot an elephant in his pyjamas S NP VP NP det n PP det {an, the, his} S NP VP S NP VP NP det n PP VP v NP PP PP prep NP n {elephant, man, shot, pyjamas} det n v NP PP VP v NP PP the man shot det n prep NP v shot NP det n an elephant in his pyjamas NP det n PP det {an, the, his} n {elephant, man, shot, pyjamas} PP prep NP prep in 24

Recognizing ambiguity • Notice how the choice of strategy determines which result we get (first). • In both strategies, there are often rules left untried, on the list (whether queue or stack). • If we want to know if our input is ambiguous, at some time we do have to follow these through. • As you will see later, trying out alternative paths can be quite intensive 25

S NP VP NP det n PP VP v NP PP PP prep NP 6. Bottom-up with structural ambiguity S VP v NP det n VP S PP prep NP NP det n PP VP v NP S VP VP v NP PP S NP VP NP S NP VP det n v the man shot PP det an NP n prep det n elephant in his pyjamas 26

S NP VP NP det n PP VP v NP PP PP prep NP 6. Bottom-up with structural ambiguity S VP NP S NP VP det n v the man shot PP det an NP n prep det n elephant in his pyjamas 27

Recursive rules • “Recursive” rules call themselves • We already have some recursive rule pairs: NP det n PP PP prep NP • Rules can be immediately recursive Adj. G adj Adj. G (the) big fat ugly (man) 28

Recursive rules Left recursive Adj. G adj Adj. G adj Right recursive Adj. G adj Adj. G big Adj. G adj adj fat rich Adj. G adj old big Adj. G fat rich old 29

7. Top-down with left recursion NP det Adj. G n Adj. G adj Adj. G adj the big fat rich old man NP det Adj. G n Adj. G adj Adj. G adj NP NP det Adj. G n the Adj. G adj n You can’t have leftrecursive rules with a top -down parser, even if the non-recursive rule is first Adj. G adj 30 Adj. G adj

7. Top-down with right recursion NP det Adj. G n Adj. G adj the big fat rich old man NP det Adj. G n Adj. G adj NP det Adj. G the adj n Adj. G big adj man Adj. G fat adj Adj. G rich adjadj Adj. G old adj Adj. G 31

8. Bottom-up with left and right recursion NP det Adj. G n Adj. G Adv. G adj Adj. G adj Adv. G adv Adv. G adv Adj. G adj Adv. G adv Adj. G Adv. G adj Adj. G Adv. G adv NP Adj. G Adv. G adj Adj. G NP det Adj. G n Adj. G rule is right recursive, Adv. G rule is left recursive Adv. G Adj. G Quite a few useless paths, but overall no difficulty Adv. G Adj. G det adv adj n the very fat ugly man 32

8. Bottom-up with left and right recursion NP det Adj. G n Adj. G Adv. G adj Adj. G adj Adv. G adv Adv. G adv Adj. G adj Adv. G adv Adj. G Adv. G adj Adj. G Adv. G adv NP Adj. G Adv. G adj Adj. G NP det Adj. G n Adj. G rule is right recursive, Adv. G rule is left recursive Adv. G det adv Adj. G adv adj n the very fat ugly man 33

Empty rules • For example NP det Adj. G n Adj. G adj Adj. G ε • Equivalent to NP det Adj. G n NP det n Adj. G adj Adj. G • Or NP det (Adj. G) n Adj. G adj (Adj. G) 34

7. Top-down with empty rules NP det Adj. G n Adj. G adj Adj. G ε the man the big fat man NP det Adj. G n Adj. G adj Adj. G Adjg. G ε NP det the adj NP Adj. G n Adj. G man det Adj. G n Adj. G man the adj big adj Adj. G fat 35

8. Bottom-up with empty rules NP det Adj. G n Adj. G adj Adj. G ε Adj. G adj Adj. G NP det Adj. G n NP Lots of useless paths, especially in a long sentence, but otherwise no difficulty Adj. G det Adj. G adj Adj. G the fat n man Adj. G 36

Some additions to formalism • Recursive rules build unattractive tree structures: you’d rather have flat trees with unrestricted numbers of daughters NP Adj. G • Kleene star • Adj. G adj* det adj adj n the big fat old ugly man 37

Some additions to formalism • As grammars grow, the rule combinations multiply and it gets clumsy NP det n NP det Adj. G n NP det n PP NP det Adj. G n PP NP n NP Adj. G n NP n PP NP Adj. G n PP NP (det) n (Adj. G) n (PP) 38

Processing implications • Parsing with Kleene star – Neatly combines empty rules and recursive rules 39

9. Top-down with Kleene star NP det adj* n the man the big fat man NP det adj* n adj* = adj adj* = ε NP det the adjadj* NP n man adjadj* det adj* the big fat n man 40

10. Bottom-up with Kleene star NP det adj* n NP NP det adj n the fat man the fat ugly man 41

Processing implications • Parsing with bracketed symbols – Parser has to expand rules • either in a single pass beforehand • or (better) on the fly (as it comes to them) – So bracketing convention is just a convenience for rule-writers NP (det) n (Adj. G) n (PP) NP det n (Adj. G) n (PP) NP n (Adj. G) n (PP) 42

Top down vs. bottom-up • Bottom-up builds many useless trees • Top-down can propose false trails, sometimes quite long, which are only abandoned when they reach the word level – Especially a problem if breadth-first • Bottom-up very inefficient with empty rules • Top-down CANNOT handle left-recursion • Top-down cannot do partial parsing – Especially useful for speech • Wouldn’t it be nice to combine them to get the advantages of both? 43

Left-corner parsing • The “left corner” of a rule is the first symbol after the rewrite arrow – e. g. in S NP VP, the left corner is NP. • Left corner parsing starts bottom-up, taking the first item off the input and finding a rule for which it is the left corner. • This provides a top-down prediction, but we continue working bottom-up until the prediction is fulfilled. • When a rule is completed, apply the left-corner principle: is that completed constituent a leftcorner? 44

9. Left-corner with simple grammar S NP VP NP det n VP v NP the man shot an elephant NP det n S S NP VP VP v but text not all accounted for, so try VP v NP NP det n NP VP det n v NP the man shot det an n elephant 45