SynQG Syntactic and Shallow Semantic Rules for Question

  • Slides: 18
Download presentation
Syn-QG: Syntactic and Shallow Semantic Rules for Question Generation Kaustubh D. Dhole, Christopher D.

Syn-QG: Syntactic and Shallow Semantic Rules for Question Generation Kaustubh D. Dhole, Christopher D. Manning ACL 2020

Syntactic Shallow Semantic/Event Based Sub-Event Based Role Descriptors Declarative Sentences Question Generation Question-Answer Pairs

Syntactic Shallow Semantic/Event Based Sub-Event Based Role Descriptors Declarative Sentences Question Generation Question-Answer Pairs • Grammatically Correct • Transparent • Highly Relevant • Modifiable • Variety

Rule Based vs Neural Approaches • Rule Based Architectures: • • • highly transparent

Rule Based vs Neural Approaches • Rule Based Architectures: • • • highly transparent easily extensible perform clearly defined syntactic transformations Increased variety with multiple sentence views Difficult to scale to ungrammatical sentences • Neural Architectures • • Not transparent Not reliable to deploy in production systems (temporarily) Can ingest noisy sentences Can exploit large amount of QA data available

Syn-QG • A rule-based framework which generates questions by identifying potential short answers in

Syn-QG • A rule-based framework which generates questions by identifying potential short answers in 1. 2. 3. 4. 5. the nodes of crucial dependency relations the modifying arguments of each predicate in the form of semantic roles named entities and other generic entities the states of Verb. Net’s thematic roles in the form of semantic predicates and Prop. Bank roleset specific natural language descriptions • + Custom Rules • + Handling Negations using Implicatives • + Back. Translation

Dependency Templates

Dependency Templates

SRL Templates

SRL Templates

Named Entities, Custom Entities and Hypernyms

Named Entities, Custom Entities and Hypernyms

Handling modals and auxilliaries • When there are multiple auxiliaries, only invert the first

Handling modals and auxilliaries • When there are multiple auxiliaries, only invert the first auxiliary while the second and further auxiliaries remain as they are just before the main verb. • Harry eats the food? Does Harry eat the food? • Harry can eat the food? Can Harry eat the food? • Harry would like to eat the food? Would Harry like to eat the food? • We make the question auxiliary finite and agree with the subject. • We ensure that the object is kept immediately after the verb. • For passive cases, subj-verb-obj is changed to obj-verb-by-subj.

Handling Factualness via Implicature • Lauri Karttunen’s Implication Rules (2012) • simple implications in

Handling Factualness via Implicature • Lauri Karttunen’s Implication Rules (2012) • simple implications in the form of 92 verbs • phrasal implications in the form of 9 sets of verbs and 8 sets of nouns making 1002 verb-noun collocations • Picard performed X POS Did Picard perform X? : YES • Picard failed to perform X NEG Did Picard perform X? : NO • Picard did not fail to perform X POS Did Picard perform X? : YES • Bojack hesitated to X? UNSURE DO NOT GENERATE

Verb. Net Predicate Templates

Verb. Net Predicate Templates

Prop. Bank Description Templates • Prop. Bank rolesets’ coursegrained annotation of verbspecific argument definitions

Prop. Bank Description Templates • Prop. Bank rolesets’ coursegrained annotation of verbspecific argument definitions • • Killer Payer Wanter …

Back-Translation 1. correct grammatical errors propagated from incorrect parsing and template restrictions, and 2.

Back-Translation 1. correct grammatical errors propagated from incorrect parsing and template restrictions, and 2. eliminate rarely used Prop. Bank descriptions and generate highly probable questions.

Human Evaluation

Human Evaluation

Stephen Hawking once threw a party for time-travelers on June 28, 2009 but he

Stephen Hawking once threw a party for time-travelers on June 28, 2009 but he only announced the party the next day. • NQG: Who took a party for time-travelers in 2009 ? • Syn. QG: Pobj. Template: : SUBJECT-BASED Pobj. Template: : BOOLEAN Dobj. Qg. Template: : OBJECT-BASED Dobj. Qg. Template: : SUBJECT-BASED Dobj. Qg. Template: : BOOLEAN Am. Tmp. Qg. Template Am. Pnc. Qg. Template Am. Tmp. Qg. Template Who threw a party once on June 28 ? Did Stephen Hawking throw a party once on June 28 ? What did Stephen Hawking throw for time - travelers once on June 28 ? Who threw a party for time - travelers once on June 28 ? Did Stephen Hawking throw a party for time - travelers once on June 28 ? Who threw a party for time - travelers ? Did Stephen Hawking throw a party for time - travelers ? What did he announce only the next day ? Who announced the party only the next day ? Did he announce the party only the next day ? When did Stephen Hawking throw a party for time - travelers ? For what purpose did Stephen Hawking throw a party once on June 28 ? When did he announce the party only ?

Due to previous guilt, John didn’t fail to pay $15000 for the car to

Due to previous guilt, John didn’t fail to pay $15000 for the car to Sam at the big garage. • NQG: Why did John pay $15000 to Sam ? • Syn. QG (besides SRL and Dependency): • VN: • • What is the cost of the car? $15000 Does Sam have possession of $15000? Yes Does John have possession of the car? Yes … • PB • Due to previous guilt, who was the payer or buyer who paid $15000 for the car to Sam ? • …

Thank You!

Thank You!