Span based models in NLP Wentao Ding 2021222

Span based models in NLP Wentao Ding 2021/2/22 1

Outline Background n Span Representations n for Constituency Parsing l for Coreference Resolution l for Semantic Role Labeling l (ACL’ 17) (EMNLP’ 18) 2021/2/22 2

Background n Span based NLP model in recent years l Many state of the art Deep NLP models use representations of spans as the basic building block for models rather than representations of words. Chinese segmentation & POS Segmental RNN (ICLR’ 16 Poster) LSTM-Minus (PKU) (ACL 16) Frame-Semantic Parsing Extractive Question Answering Coreference Resolution Constituency Parsing 2021/2/22 3

Span Representations Segmental RNN LSTM-Minus 2021/2/22 4

Span Representations for Constituency Parsing n A Minimal Span-Based Neural Constituency Parser (ACL’ 17) l Chart Parsing and Top-down Parsing 2021/2/22 5

Span Representations for Constituency Parsing n A Minimal Span-Based Neural Constituency Parser (ACL’ 17) l Chart Parsing and Top-down Parsing 2021/2/22 6

Span Representations for Constituency Parsing n A Minimal Span-Based Neural Constituency Parser (ACL’ 17) l Chart Parsing and Top-down Parsing 2021/2/22 7

Span Representations for Constituency Parsing n A Minimal Span-Based Neural Constituency Parser (ACL’ 17) l Chart Parsing and Top-down Parsing 2021/2/22 8

Span Representations for Constituency Parsing n A Minimal Span-Based Neural Constituency Parser (ACL’ 17) l Chart Parsing and Top-down Parsing 2021/2/22 9

Span Representations for Constituency Parsing n 2021/2/22 10

Span Representations for Coreference Resolution n 2021/2/22 11

Span Representations for Coreference Resolution n F 1 on MUC F 1 on B 3 F 1 on CEAFϕ 4 AVG F 1 This(ensemble) 77. 2 66. 6 62. 6 68. 8 This(single) 75. 8 65. 0 60. 8 67. 2 Previous STOA 74. 6 63. 4 59. 2 65. 7 2021/2/22 12

Span Selection Model for Semantic Role Labeling n A Span Selection Model for Semantic Role Labeling (EMNLP’ 18) Given a sentence words, predict a set of labeled spans l Learning a score for span-label pairs l – l Select the highest scoring span for each label Constraints – – Can not select overlap span Select at most one span for each core label 2021/2/22 13

Span Selection Model for Semantic Role Labeling n Evaluation 2021/2/22 14

References n n n Lingpeng Kong, Chris Dyer, Noah A. Smith: Segmental Recurrent Neural Networks. ICLR (Poster) 2016 Wenhui Wang, Baobao Chang: Graph-based Dependency Parsing with Bidirectional LSTM. ACL (1) 2016 Mitchell Stern, Jacob Andreas, Dan Klein: A Minimal Span-Based Neural Constituency Parser. ACL (1) 2017: 818 -827 Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer: End-to-end Neural Coreference Resolution. EMNLP 2017: 188 -197 Hiroki Ouchi, Hiroyuki Shindo, Yuji Matsumoto: A Span Selection Model for Semantic Role Labeling. EMNLP 2018: 1 630 -1642 2021/2/22 15

Thanks for listening n Q & A 2021/2/22 16