Finding Domain Terms using Wikipedia Jorge Vivaldi Palatresi

Finding Domain Terms using Wikipedia Jorge Vivaldi Palatresi Horacio Rodríguez Hontoria Applied Linguistics Institute TALP Research Center Universitat Pompeu Fabra Universitat Politécnica de Catalunya jorge. vivaldi@upf. edu horacio@lsi. upc. es

Outline • • • Introduction Related approaches Methodology Evaluation Conclusions and future work 2

Introduction • Problem: to automatically extract terminological units from specialized texts • Result: list of all the WP categories and page titles that our system considers that belong to the domain of interest.

Related approaches • • Magnini et al. , 2000 Montoyo et al. , 2001 Missikoff et al. , 2002 Vivaldi, Rodríguez, 2004 Bernardini et al. , 2006 Cui et al. , 2008 4

Graph structure of Wikipedia WP categories … … … A C B D … E … F … WP pages P 1 Redirection table P 2 P 3 G … … Disamb. pages Interwiki links External links Info. Box

Methodology: overview domain filtering WP top categories Categories domain categories Pages bootstrapping filtering domain pages final domain term set Main steps: 4) Filter 5) Remove categories proper names and pages and service 1) 2) 3) To find Look Extract for in allall WP descendants thethe subcategories/pages domain from name theas domain aclasses related category. name to the avoiding domain loops

Methodology: filtering • Category level • Page level

Methodology: filtering Top Category of the Domain • Category level Direct super-categories Cat. Set 1 C Cat. Set 1 Direct neutral super-categories Category Score

Methodology: filtering Top Category of the Domain neutral categories • Page level categories Cat. Set 2 Pages C . . . P . . . categories Cat. Set 2 Page Score C Cat. Set 2

Methodology: category filtering

Methodology: page filtering Additional category filtering using pages scores: cat. Term: set of pages associated to a category - Micro. Strict: accept cat if # elements of cat. Term with positive scoring is greater that # elements with negative scoring - Micro. Loose: Idem with greater or equal test. - Macro: instead of counting the pages with positive/negative scoring we use the components of such scores.

Page filtering example: “semantics” (in Computing domain) theoretical computer science Computing semantics software engineering formal methods semantics {linguistics, philosophy of language, semiotics, theoretical computer science, philosophical Logic} WPCD(semantics) = 0. 25

Category filtering example using pages score: “chemistry” # 1 DTC electroquímica (electrochemistry) 2 quesos (cheeses) 3 óxidos de carbono (carbon monoxide) Micro Strict ok ko Micro Loose ok ko Macro ok Vote Result ko 13 5 16 2 36 12 +3 Accept 0 8 6 2 8 12 -1 Reject 1 1 2 0 4 3 +2 Accept

Evaluation • Partial evaluation: “chemistry” and “astronomy”: – Test against Magnini et al. , 2000 (Word. Net 1. 6) – Low coverage: 25% for Chemistry and 15% for Astronomy • Full evaluation. “Medicine” – Test against SNOMED-CT Spanish Edition (2009) – Wide coverage of the clinical domain: 800 K terms

Partial evaluation

Full evaluation Validation issues Accepts whisky cigar udder fire Reject oral cancer renal colic phoniatrics surgical instruments

Conclusions • Good results when evaluated against a specialised resource • Term list filtering must be improved (ex. Eliminate proper names) 17

Future work • Apply this method to other languages/domains • Improve filtering using in/out links of selected pages • Improve filtering using also the page content • Use this WP knowledge to improve a term extractor 18

Finding Domain Terms using Wikipedia Jorge Vivaldi Palatresi Horacio Rodríguez Hontoria Applied Linguistics Institute TALP Research Center Universitat Pompeu Fabra Universitat Politécnica de Catalunya jorge. vivaldi@upf. edu horacio@lsi. upc. es 19