Hierarchical Semisupervised Classification with Incomplete Class Hierarchies Bhavana

Hierarchical Semi-supervised Classification with Incomplete Class Hierarchies Bhavana ¶* Dalvi , Aditya † Mishra , and William W. * Cohen ¶ Allen Institute for Artificial Intelligence, * School Of Computer Science, Carnegie Mellon University, † Department of Computer Science & Software Engineering, Seattle University Motivation Ø In an entity classification task, topic or concept hierarchies are often incomplete. This can lead to semantic drift of known classes or topics. Method: Opt. DAC Exploratory EM Experimental Results Opt. DAC reduces semantic drift of seeded classes. Comparison: macro averaged seeded-class F 1 Ø Our previous work on Exploratory Learning (Dalvi et al. ECML 2013) extends the semisupervised EM algorithm by dynamically adding new classes when appropriate. In this paper, we present Exploratory learning techniques for hierarchical semi-supervised learning tasks. 75 55 45 35 25 Level = 2 3 Opt. DAC with varying amount of training data Text-Small Ø KB categories are arranged in an ontology. There are subset and disjointness constraints defined between these classes. Further, the class hierarchy can be incomplete. Datasets Optimal Label Assignment given Class Constraints This dataset is made publicly available at http: //rtw. ml. cmu. edu/wk/Web. Sets/hierar chical_Exploratory. Learning_WSDM 2016 /index. html Statistic 3 11 1, 3, 7 4 Table-Small Runtime of Flat vs. Opt. DAC method on different datasets Dataset Avg. Runtime Avg. runtime in multiple of Flat Semiin sec. supervised EM FLAT Opt. DAC Semi. Exploratory supervised EM EM EM Text-Small 53. 5 8 7 17 Table-Small 50. 7 3 10 21 Text-Medium 524. 7 5 11 25 Table 5932. 4 4 7 10 Medium Evaluation of extended class Ontology Small Medium #Classes #levels in the hierarchy #classes per level FLAT-Explore. EM 65 Ø We focus on entity classification task where each entity is represented by either text context or table co-occurrence features. Given a few seed examples per Knowledge Base(KB) category, the task is to classify unlabeled entities into KB categories. Ø Our proposed method (Opt. DAC) can learn new examples of existing classes, as well as extend the class hierarchy in a single unified framework. denotes statistically significant improvements (0. 05 significance level) w. r. t. FLAT Exlore. EM 4 39 1, 4, 24, 10 hierarchies Small Ontology Subset constraint Penalty Score of label assignment Medium Ontology Mutex constraint Penalty Subset constraint Mutex Constraint When New Classes Are Created? 1 Dataset Statistics #Entitie #Feature # (Entity, s s label) pairs Text-Small Text-Medium Table-Small 2. 5 K 12. 9 K 4. 3 K 3. 4 M 6. 7 M 0. 96 M 7. 2 K 42. 2 K 12. 2 K Table-Medium 33. 4 K 2. 2 M 126. K 5 Ø An example Text pattern feature for entity “Pittsburgh” is (“lives in ARG”, 1000), indicating that the entity Pittsburgh appeared in position ARG of the text context “live in ARG” for 1000 times in the sentences from Clueweb 09 dataset. Ø An example Table context feature for entity “Pittsburgh” is (“clueweb 09 -en 0011 -94 -04: : 2: 1”, 1) indicates that the entity “Pittsburgh” appeared once in HTML table 2, column 1 from Clue. Web 09 document id “clueweb 09 en 0011 -94 -04”. 3 2 6 7 4 8 9 10 Near uniform? Cnew Test: Best assignment using the mixed integer program should pick Cnew 11 Conclusions Ø In this paper, we propose the Hierarchical Exploratory EM approach that can take an incomplete class ontology as input, along with a few seed examples of each class, to populate new instances of seeded classes and extend the ontology with newly discovered classes. Ø Our proposed hierarchical exploratory EM method, named Opt. DAC-Explore. EM performs better than flat classification and hierarchical semi-supervised EM methods at all levels of hierarchy, especially as we go further down the hierarchy. Ø Experiments show that Opt. DAC-Explore. EM outperforms its semisupervised variant on average by 13% in terms of seed class F 1 scores. It also outperforms both previously proposed exploratory learning approaches FLAT-Explore. EM and DAC-Explore. EM in terms of seed class F 1 on average by 10% and 7% respectively. Ø In the future, we would like to apply our method on datasets with non-tree structured class hierarchies. Acknowledgements : This work is supported in part by Google Ph. D fellowship in Information Extraction, and NSF grant No. IIS 1250956 NSFCOHEN.