Distant Supervision for Knowledge Base Population Mihai Surdeanu

Distant Supervision for Knowledge Base Population Mihai Surdeanu, David Mc. Closky, John Bauer, Julie Tibshirani, Angel Chang, Valentin Spitkovsky, Christopher Manning

Definition and Approach • We took part in TAC KBP 2010 this year (both tasks) • Slot filling task: learning a pre-defined set of relations and attributes for target entities based on documents in a collection – “Warren Buffett began studying at the Warton School of Finance at the University of Pennsylvania, but transferred to the University of Nebraska where he graduated. ” • (per: schools_attended, Warren Buffett, University of Pennsylvania) • (per: schools_attended, Warren Buffett, University of Nebraska • Distant supervision approach: generate training data automatically from Wikipedia infoboxes

Training Evaluation Infobox KB KBP query: entity name Map infobox fields to KBP slots (one to many mapping) IR: find relevant sentences Query: entity name + slot value Map KBP slots to fine-grained NE labels Extract +/- slot candidates Train multiclassifier IR: find relevant sentences Query: entity name + trigger words Extract slot candidates Classify candidates Inference (greedy, local) Extracted slots

Results Training on 2/3 of infoboxes, evaluating on 1/3 Evaluating only on sentences that contain at least a valid slot Label Correct Predict Actual P R F 1 UNRELATED 268085 289135 295590 92. 7 90. 7 91. 7 org: city_of_ headquarters 5835 9040 7514 64. 5 77. 7 70. 5 org: country_of_ headquarters 2851 4638 3725 61. 5 76. 5 68. 2 org: founded 3896 8199 6662 47. 5 58. 5 52. 4 org: parents 1158 2292 2525 50. 5 45. 9 48. 1 org: top_members/empl oyees 1282 3067 3596 41. 8 35. 7 38. 5 per: city_of_birth 1799 3920 3252 45. 9 55. 3 50. 2 per: country_of_birth 1984 4122 3204 48. 1 61. 9 54. 2 per: date_of_birth 3938 5427 4362 72. 6 90. 3 80. 5 per: member_of 1771 3018 2887 58. 7 61. 3 60 per: title 1714 3364 3054 51 56. 1 53. 4 37169 68822 62367 54 59. 6 56. 7 Total Top 10 most common slots Total for all slots

Challenges • Improve quality of data generated through distant supervision • Improve IR recall – Use relation-specific trigger words (or n-grams or dependency paths etc. ) to boost sentences likely to contain answers to the top – How to acquire these automatically? • Better classifiers for noisy text (e. g. , web snippets)