Multilingual Information Access in a Digital Library Vamshi

Multilingual Information Access in a Digital Library Vamshi Ambati, Rohini U, Pramod, N Balakrishnan and Raj Reddy International Institute of Information Technology Hyderabad, India IIIT Hyderabad - http: //dli. iiit. ac. in

Context l Digital Library of India l 155, 000 English books l 145, 000 Other language books l Population of literates l 20% of India understand English l 80% can not IIIT Hyderabad - http: //dli. iiit. ac. in 2

Multilingual Access to Information l Retrieve a book l By metadata l By keyword / content l Cross Lingual Information Retrieval l Read a book l Help understand sentences in a language l Help understand sentences across languages l Machine Translation IIIT Hyderabad - http: //dli. iiit. ac. in 3

Approaches to Multilingual Access l Cross Lingual Retrieval l Translate Query to Document Language l Translate Document to Query Language l Machine Translation l Knowledge Based Approaches l Corpus Based Approaches l Hybrid Approaches IIIT Hyderabad - http: //dli. iiit. ac. in 4

Challenges in Multilingual Access l Corpus Based Approaches l Unavailability of Parallel Corpus for pairs of languages l Unavailability of Computational Linguistics Resources l Dictionary Based Approaches l Unavailability of multiple bilingual dictionaries IIIT Hyderabad - http: //dli. iiit. ac. in 5

Resources l Universal Dictionary l Conceived and implemented by Michael Shamos at CMU, USA l ITRANS l. A transcription scheme and associated tool built by IISc, IIIT and CMU l Corpus l Data Entry by TTD and DLI project l TIDES project IIIT Hyderabad - http: //dli. iiit. ac. in 6

Universal Dictionary IIIT Hyderabad - http: //dli. iiit. ac. in 7

�How are we doing it l Cross Lingual Search (Identify Information) Dictionary lookup l User feedback based l Lucene Search Engine l l Machine Translation (Understand Information) Corpus based technique (EBMT) l Dictionary based word-word lookup l Good-enough translation vs Perfect translation l IIIT Hyderabad - http: //dli. iiit. ac. in 8

Cross Lingual Retrieval IIIT Hyderabad - http: //dli. iiit. ac. in 9

Cross Lingual Retrieval IIIT Hyderabad - http: //dli. iiit. ac. in 10

Reading Assistant System IIIT Hyderabad - http: //dli. iiit. ac. in 11

Reading Assistant IIIT Hyderabad - http: //dli. iiit. ac. in 12

Status Today l CLIR for 6 languages l MT for 3 languages l Shakti (a knowledge based MT system) l Parallel Corpus for Hindi-Eng l UDICT l About 40 Foreign Languages l 6 Indian Languages IIIT Hyderabad - http: //dli. iiit. ac. in 13

What more is needed? l UDICT Improving coverage of existing languages l Adding new languages l l Machine Translation Corpus acquisition l State of art techniques applied to Indian Languages l Multi-way parallel corpus development l l Textual format for the books Books currently are in Image formats l OCR should be developed for textual content l IIIT Hyderabad - http: //dli. iiit. ac. in 14

Thank You Questions ? IIIT Hyderabad - http: //dli. iiit. ac. in