Meeting Information Extraction from Meeting Announcement in Korean

Meeting Information Extraction from Meeting Announcement in Korean Kyoungryol Kim

Table of Contents 1. Introduction § Motivation § Goal § Problem Definition 2. The Proposed Method § Problem Modeling / Checklist § Overall Architecture § Normalization Process 2

Introduction

Motivation § Everyday we receive a lot of Meeting Announcement § Conference, Seminar, Workshop, Meeting, Appointment… § Meeting announcement accounts for 17% (30, 201 out of 183, 022) of emails in Enron Email Dataset. * Enron Email Dataset, August 21, 2009 version, http: //www. cs. cmu. edu/~enron/ § Smartphone era § Many people manage schedule using online-calendar via smartphone e. g. Google Calendar § But, typing by touch screen keyboard make many errors and even it’s difficult. 4

Goal § Extracting schedule information from meeting announcement, and update them to the calendar, automatically. start. Time 2011 -07 -19 T 14: 00 Administrative Address is. Held. At Geocode Semantic Type 대한민국 서울특별시 중구 명동 1가 1 -1 민들레영토 명동점 (37. 5647312, 126. 9861426) Café Meeting Announcement 무더운 날씨가 본격적으로 시작 되는 즈음하여 유니브캐스트의 상반기 평가와 하반기 운영을 위 한 정기팀장회의를 개최합니다. 날짜 : 7월 19일(토) 오후 2시 장소 : 민들레영토 오는길 지도와 같이 명동역 8번 출구로 나오셔서 쭉 상가 끼고 걸어가시 면 저기 YMCA빌딩 1층에 있습 니다. Extract Update 5

The Proposed Method

Problem Modeling Meeting Announcement Text Extract location strings 1. How to extract location string? Extract address information and limit the boundary Search the location from the DB Location on the Map 7. How to represent Location? 2. How to extract address information? 3. What kind of DB can we use? 4. How to manipulate the query? Search the location from external resources 5. What kind of external resources can we use? Disambiguation among found locations 6. What are the measures to find desired location? 8

Problem List (1/2) 1. 2. 3. 4. 5. 6. 7. How to extract location strings from the given text? How to extract address information from location strings? To search the location, what kind of database can we use? To search the location, how to manipulate the query? To search the location, what kind of external resources can we use? What are the measures to find desired locations among candidates? How to represent the location ? 9

Problem List (2/2) - Reorganized 1. 2. How to extract location strings from the given text? How to extract address information from location strings? 1) How to check whether address information is included or not? 2) How to construct database which can limits boundary of address 3. To search the location, what resources can we use? 1) Internal database : How to construct internal database? 2) External resources : What external resources available? 4. 5. 6. To search the location, how to manipulate the query? What are the measures to find desired locations among candidates? How to represent the location ? 1) To store the location to the DB 2) To represent the location on the map 10

Problem Checklist : (6/6) How to represent the location ? 1) To store the location to the DB § Uses Open. Street. Map representation § Node / Way / Relation 2) To represent the location on the map § WGS 84 (standard) : ( latitude, longitude [, altitude] ) 11

Representation of Meeting Location § Follows basic representations of the Node in Open. Street. Map to represent location. § Regard the meeting location as Point-of-Interest § Variable attributes (key-value pair) http: //wiki. openstreetmap. org/wiki/Map_Features § used_as_meeting_location=true § search_query=user’s query (comma separated) § Meeting location can be imported to OSM server (interoperability) <node id="850918486" lat="37. 4936384" lon="127. 0137745" user="cyana" uid="74529" visible="true" version="3" changeset="5478335" timestamp="2010 -08 -13 T 02: 26: 19 Z"> <tag k="name" v="교대 (Gyodae)"/> <tag k="name: en" v="Gyodae"/> <tag k="name: ko_rm" v="Gyodae"/> <tag k="railway" v="station"/> </node> <node id="368738707" lat="37. 4990100" lon="127. 0275800" user="cyana" uid="74529" visible="true" version="2" changeset="4370541" timestamp="2010 -04 -09 T 08: 09: 50 Z"> <tag k="amenity" v="dentist"/> <tag k="name" v="미소드림치과 (Misodeurim Dental Clinic)"/> <tag k="name: en" v="Misodeurim Dental Clinic"/> <tag k="name: ko" v="미소드림치과"/> <tag k="name: ko_rm" v="Misodeurimchigwa"/> <tag k="ncat" v="치과"/> </node>

node changeset id int node_id int lat double id int lng double created_at varchar(20) user varchar(100) : email num_changes int version int closed_at varchar(20) changeset int open boolean timestamp varchar(20) user varchar(100) : email node_tag node_id int key varchar(100) value varchar(100) changeset_tag bounds id int country_code char(2) : ISO-3166 admin_div 1 varchar(100) admin_div 2 varchar(100) admin_div 3 varchar(100) admin_div 4 varchar(100) southwest_lat double southwest_lng double northeast_lat double node_id int changeset_id int key varchar(100) value varchar(100)

Example : bounds § Bounds information constructed by using Google Maps API § Closed-world is South Korea area (possibly can be expanded)

Overall Architecture Training System Testing System Input Document Personal Information Expand Gazetteer Train Models Trained Models (CRFs, SVMs) Corpus Expansion Document Annotation Adding Document to Corpus Training Corpus Finding Target Locations Location NER Relation-type Classification Open. API Map Services Normalization Disambiguation OUTPUT 15

Input Document Normalization Finding Target Locations Location NER Relation-type Classification Normalization Process Open. API Map Services Normalization Disambiguation Personal Informati on Gazette er Input Query: 프란치스코교육회관 2층 OUTPUT Trained Models (CRFs, SVMs) Pre-Processing : Remove HTML-tag/URL/㈜ Replace (), [], {} with space { “query” : { “full” : “프란치스코교육회관 2층”, } } Split the Query into 2 parts : Main Part / Extra-Part { “query” : { “full” : “프란치스코교육회관 2층”, “main” : “프란치스코교육회관”, “extra” : “ 2층” } } Main : Chunks include Main location information. Extra : Chunks include Floor/room information.

Input Document Normalization Finding Target Locations Location NER Relation-type Classification Open. API Map Services Extract Address Information Normalization Disambiguation Personal Informati on Gazette er No OUTPUT Trained Models (CRFs, SVMs) has Address Info? Yes { “query” : { “full” : “프란치스코교육회관 2층”, “main” : “프란치스코교육회관”, “extra” : “ 2층” } } Get Bounds info from Address (SW, NE) No Bounds DB “query” : { “full” : “소공동 코리아나 호텔”, “main” : “소공동 코리아나 호텔”, “extra” : “” }, “limited_bound” : { “name” : “대한민국 서울특별시 중구 소공동”, “southwest” : { lat : 37. 4346000, lng : 126. 7968000}, “northeast” : { lat : 37. 6956000, lng : 127. 1823000} } } has Address info? 1) main query 를 space 단위로 chunking 하고 include House no? Yes Geocoding by Query { 1. if query doesn’t have Address information: Without boundary limitation, just do search from the databases and APIs 2) 각 chunk 를 iteration 하면서 - chunk가 “-시”, “-시/-구/-군”, “-동/-가/-면/-읍”, “-리” 로 끝나는지, - DB의 시/구/동/리 칼럼의 값으로 시작되는지 확인하여, 찾아진 칼럼과 값을 저장한다. 3) 주소정보가 포함되어 있다면, 뒤에 번지수까지 포함하고 있는지 확인한다. [0 -9]+, [0 -9]+-[0 -9]+, [0 -9]+번지, [0 -9]+-[0 -9]+번지 - 번지수까지 포함되어 있으면, 바로 geocoding. - 번지수는 없으면, 해당지역까지의 bounds 를 db에서 가져옴. { “query” : { “full” : “서울시 강남구 삼성동 159 -1 무역회관 2001호”, “main” : “서울시 강남구 삼성동 159 -1 무역회관”, “extra” : “ 2001호” }, found_locations : [ { “title” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “administrative_address” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “geometry_location” : { “lat” : 37. 5103598, “lng” : 127. 0611803 } ] }

Input Document Normalization Finding Target Locations Location NER Relation-type Classification Open. API Map Services Extract Address Information Normalization Disambiguation Personal Informati on Gazette er No OUTPUT Trained Models (CRFs, SVMs) has Address Info? 2. if the query have address information (with house number) : Geocode the address information and return. (Disambiguation finished) Yes { “query” : { “full” : “프란치스코교육회관 2층”, “main” : “프란치스코교육회관”, “extra” : “ 2층” } } Get Bounds info from Address (SW, NE) No Bounds DB include House no? Yes Geocoding by Query { “query” : { “full” : “소공동 코리아나 호텔”, “main” : “소공동 코리아나 호텔”, “extra” : “” }, “limited_bound” : { “name” : “대한민국 서울특별시 중구 소공동”, “southwest” : { lat : 37. 4346000, lng : 126. 7968000}, “northeast” : { lat : 37. 6956000, lng : 127. 1823000} } } { “query” : { “full” : “서울시 강남구 삼성동 159 -1 무역회관 2001호”, “main” : “서울시 강남구 삼성동 159 -1 무역회관”, “extra” : “ 2001호” }, found_locations : [ { “title” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “administrative_address” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “geometry_location” : { “lat” : 37. 5103598, “lng” : 127. 0611803 } ] }

Input Document Normalization Finding Target Locations Location NER Relation-type Classification Open. API Map Services Extract Address Information Normalization Disambiguation Personal Informati on Gazette er No OUTPUT Trained Models (CRFs, SVMs) has Address Info? 3. if the query have address information (no house number) : Get bound information and search the location in the bound. Yes { “query” : { “full” : “프란치스코교육회관 2층”, “main” : “프란치스코교육회관”, “extra” : “ 2층” } } Get Bounds info from Address (SW, NE) No Bounds DB include House no? Yes Geocoding by Query { “query” : { “full” : “소공동 코리아나 호텔”, “main” : “소공동 코리아나 호텔”, “extra” : “” }, “limited_bound” : { “name” : “대한민국 서울특별시 중구 소공동”, “southwest” : { lat : 37. 4346000, lng : 126. 7968000}, “northeast” : { lat : 37. 6956000, lng : 127. 1823000} } } { “query” : { “full” : “서울시 강남구 삼성동 159 -1 무역회관 2001호”, “main” : “서울시 강남구 삼성동 159 -1 무역회관”, “extra” : “ 2001호” }, found_locations : [ { “title” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “administrative_address” : “대한민국 서울특별시 강남구 삼성동 159 -1”, “geometry_location” : { “lat” : 37. 5103598, “lng” : 127. 0611803 } ] }

Input Document { Normalization “query” : { “full” : “소공동 코리아나 호텔”, “main” : “소공동 코리아나 호텔”, “extra” : “” }, “limited_bound” : { “name” : “대한민국 서울특별시 중구 소공동”, “southwest” : { lat : 37. 4346000, lng : 126. 7968000}, “northeast” : { lat : 37. 6956000, lng : 127. 1823000} } Finding Target Locations Location NER Relation-type Classification Open. API Map Services Normalization Disambiguation Personal Informati on Gazette er } OUTPUT Trained Models (CRFs, SVMs) Local Search Find Candidate Locations User DB User Meeting Location DB (Priority 1) Geocoding SWRC DB SWRC Meeting Location DB (Priority 2) Open API WMS Open API (Open. Street. Map, Naver) (Priority 3) Remove Duplicated Addresses { “query” : { “full” : “소공동 코리아나 호텔”, “main” : “소공동 코리아나 호텔”, “extra” : “” }, “limited_bound” : { “name” : “대한민국 서울특별시 중구 소공동”, “southwest” : { lat : 37. 4346000, lng : 126. 7968000}, “northeast” : { lat : 37. 6956000, lng : 127. 1823000} }, found_locations : [ { “query” : “밀레니엄 힐튼 서울”, “title” : “밀레니엄 힐튼 서울”, “administrative_address” : “대한민국 서울특별시 중구 태평로 1가 61 -1”, “geometry_location” : { “lat” : 37. 5103598, “lng” : 127. 0611803 }, {. . . } ] } Coordinate Conversion KTM -> WGS 84

Disambiguation Title Original Query 밀레니엄 힐튼 서울 동강밀레니엄래프팅 밀레니엄피시방 서현점 밀레니엄모텔 서울힐튼호텔 | Query | 밀레니엄 힐튼 서울 Address 대한민국 강원도 영월군 영월읍 거운리 547 -1 대한민국 경기도 성남시 분당구 서현동 307 대한민국 광주광역시 북구 오룡동 1114 -1 대한민국 서울특별시 중구 남대문로 5가 395 Input Document Disambiguation Finding Target Locations Location NER Relation-type Classification Open. API Map Services - Number of Matched characters query-title, query-original query, query-address - (Can be used ) Semantic Type / Personal Annotation DB / Distance between location. Landmark - Personal Address book/Search history/GPS log Normalization 서울힐튼호텔 : 대한민국 서울특별시 중구 남대문로 5가 395 (36. 3414225, 127. 3914705) (Hotel) Disambiguation Personal Information Gazetteer OUTPUT Trained Models (CRFs, SVMs) 21