Recent Advances in Distantly Supervised Relation Extraction William

Recent Advances in Distantly Supervised Relation Extraction William Wang Department of Computer Science University of California, Santa Barbara Joint work with Jiawei Wu, Lei Li, Pengda Qin, Weiran Xu. CIPS Summer School 2018 Beijing, China 1/20

Agenda • Motivation • Challenges in Semi-Supervised Learning • Reinforced Co-Training (Wu et al. , NAACL 18) • Reinforced Distant Supervision Relation Extraction (Qin et al. , ACL 18 a) • DSGAN (Qin et al. , ACL 18 b) • Conclusions 2

Motivation • Most of the existing successful stories of deep learning are still based on supervised learning. • For example, object recognition, machine translation, text classification. • However, in many applications, it is not realistic to obtain large amount of labeled data. • We need to leverage unlabeled data. 3

A Classic Example of Semi-Supervised Learning • Co-Training (Blum and Mitchell, 1998) 4

Challenges • The two classifiers in co-training have to be independent. • Choosing highly-confident self-labeled examples could be suboptimal. • Sampling bias shift is common. 5

Our Approach: Reinforced SSL • Assumption: not all the unlabeled data are useful. • Idea: performance-driven semi-supervised learning that learns an unlabeled data selection policy with RL, instead of using random sampling. • 1. Partition the unlabeled data space • 2. Train a RL agent to select useful unlabeled data • 3. Reward: change in accuracy on the 6

Reinforcement Learning Agent Deep Q-Network Environment Unlabeled Data 7

Reinforced Co-Training (Wu et al. , NAACL 2018) 8

Deep Q-Learning 9

Experiment 1: Clickbait Detection 10

Experiment 1: Clickbait Detection 11

Experiment 2: Generic Text Classification 12

Experiment 2: Generic Text Classification 13

Conclusion • We proposed a novel RL framework for semisupervised learning • Strong results in SSL text classification • Also showed effectiveness in relation extraction 14

Deep Reinforcement Learning for Distantly Supervised Relation Extraction 15

Outline • Motivation • Algorithm • Experiments • Conclusion 16

Outline • Motivation • Algorithm • Experiments • Conclusion 17

Relation Extraction Relation Type with Labeled Dataset Plain Text Corpus (Unstructured Info) Classifier Entity-relation Triple (Structured Info) Relation Type without Labeled Dataset 18

Distant Supervision “If two entities participate in a relation, any sentence that contains those two entities might express that relation. ” (Mintz, 2009) 19

Distant Supervision Data(x): <Belgium, Nijlen> Label(y): /location/contains Target Corpus (Unlabeled) DS Label(y): /location/contains DS Data(x): Nijlen is a municipality located in the Belgian province of Antwerp. 20

Wrong Labeling v Within-Sentence-Bag Level § Hoffmann et al. , ACL 2011. § Surdean et al. , ACL 2012. § Zeng et al. , ACL 2015. § Li et al. , ACL 2016. v Entity-Pair Level § None 21

Wrong Labeling § Place_of_Death i. Some New York city mayors – William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad. v Entity-Pair ii. Plenty of. Level local officials have, too, including two New York city mayors, James J. Walker, in 1932, and William O’Dwyer, in 1950. 22

Wrong Labeling v Most of entity pairs only have several sentences[CATEGORY NAME] [PERCENTAGE] v Lots of entity pairs have repetitive sentences 2 Sentence [PERCENTAGE ] 1 Sentence [PERCENTAGE ] 23

Outline • Motivation • Algorithm • Experiments • Conclusion 24

Requirements Entity-Pair Level Wrong Labeling Problem Sentence-Level Indicator Without Supervised Information General Purpose and Offline Process Learn a Policy to Denoise the Training Data 25

Overview DS Dataset Cleaned Dataset Negative set Positive set False Positive Classifi er Policy Based Agent Negative set False Positive set 26

Deep Reinforcement Learning v State § Sentence vector § The average vector of previous removed sentences v Action § Remove & retain v Reward § ? ? ? 27

Deep Reinforcement Learning v One relation type has an agent v Sentence-level § Positive: Distantly-supervised positive § sentences Negative: Randomly sampled v Split into training set and validation set 28

Deep Reinforcement Learning Remove d part RL Agent Cleaned dataset Relation Train Classifie r Remove d part Cleaned dataset Relation Trai Classifie n r 29

Reward § Accurate § Steady False Positive § Fast § Obvious Positive Set Negative Set 30

Reward Negativ e Positiv e False Positive Trai n False Positive Pre. Trai n Relation Classifi er Relation Classifier Calculat e Positive Set Negative Set 31

Outline • Motivation • Algorithm • Experiments • Conclusion 32

Evaluation on a Synthetic Noise Dataset v Dataset: Sem. Eval-2010 Task v 8 True Positive: Cause-Effect v False Positive: Other v True Positive + False Positive: 1331 samples 33

Evaluation on a Synthetic Noise Dataset (180/279) (195/308) (179/260) (197/339) (198/388 ) False Positive Removed Part 34

Evaluation on a Synthetic Noise Dataset (0/59) (0/32) (0/121) (0/150 ) (0/258) 35

Distant Supervision on NYT Freebase Dataset v CNN+ONE, PCNN+ONE § Distant supervision for relation extraction via piecewise convolutional neural networks. (Zeng et al. , 2016) v CNN+ATT, PCNN+ATT § Neural relation extraction with selective attention over instances. (Lin et al. , 2016) 36

Distant Supervision CNN-based 1 CNN+ONE 0. 9 CNN+ONE_RL CNN+ATT 0. 8 CNN+ATT_RL 0. 7 0. 6 0. 5 0. 4 0 0. 1 0. 2 0. 3 0. 4 37

Distant Supervision PCNN-based 1 PCNN+ONE 0. 9 PCNN+ONE_RL PCNN+ATT 0. 8 PCNN+ATT_RL 0. 7 0. 6 0. 5 0. 4 0 0. 1 0. 2 0. 3 0. 4 38

Outline • Motivation • Algorithm • Experiments • Conclusion 39

Conclusion v We propose a deep reinforcement learning method for robust distant supervision relation Extraction. v Our method is model-agnostic. v Our method boost the performance of recently proposed neural relation extractors. 40

DSGAN: Adversarial Learning for Denoising Distantly Supervised Relation Extraction (Qin et al. , ACL 2018 b) 41

Distant Supervision Data Distribution DS data space DS Positive Data DS Negative Data 42

Data Distribution DS data space DS True Positive Data DS Negative Data DS False Positive Data The Decision Boundary of DS Data The Desired Decision Boundary 43

Adversarial Training Noisy Positive Set Label 1 True Positive Label 1 False Positive True Positive Label 0 False Positive Label 1 Generator 44

DSGAN (Qin et al. , ACL 2018 b) DS Positiv e Datase t Sampling Generato r … G … Lowconfidence samples Highconfidence samples Discriminato r Pretraining DS positive dataset D DS negative dataset 45

Characteristics v Sentence-Level Noise Reduction v Training Without Supervised Information v Model-Agnostic 46

Distant Supervision Relation Extraction CNN-based 1 CNN+ONE_GANs CNN+ATT_GANs 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0 0. 1 0. 2 0. 3 0. 4 47

Distant Supervision Relation Extraction PCNN-based 1 PCNN+ONE_GANs PCNN+ATT_GANs 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0 0. 1 0. 2 0. 3 0. 4 48

Conclusion • We introduce Reinforced Co-Training, a new approach that combines reinforcement learning and semi-supervised learning. • We show that in weakly-supervised relation extraction, reinforcement learning can be utilized to de-noise the training signals. • Adversarial learning serves as a joint learning framework, and it can also be applied to denoising distantly supervised IE data. 49

Thanks! http: //nlp. cs. ucsb. edu 50