Internationalization Localization Indian Perspective and requirements By Swaran

Internationalization & Localization : Indian Perspective and requirements By : Swaran Lata Country Manager, W 3 C India Office 6, CGO complex, Electronics Niketan New Delhi E-mail : slata@mit. gov. in

Internationalization and localization are subsets of globalization Taking a product and making it linguistically and culturally appropriate to the target locale (country/ region and language) where it will be used and sold" Process of generalizing a product so that it can handle multiple languages and cultural conventions without the need for re-design. Localization Internationalization

Localization Vs Internationalized Ø Designing and developing in a way that removes barriers to localization or international deployment. Ø Providing support for features that may not be used until localization occurs. Ø Enabling code to support local, regional, language, or culturally related preferences. Ø Separating localizable elements from source code or content, such that localized alternatives can be loaded or selected based on the user's international preferences as needed. Ø It can be localized quickly.

The Internationalization Approach to Globalization Gather Information Globally Customize : add culturally specific functionality, presentation, or content to an application. Enabling : the same code supports multiple regions or cultures Test and globally Externalize : makes localization for specific languages, regions Support Localize

Localized Application Deployment Internationalized application to be Localized Translation, Engineering, and Testing of online help/web content/documentation/ multimedia/icons etc. Review QA Localization activities Functionality testing of localized software or web applications Translation and engineering of software Project management Locale Data substitution

Complexity of Indian Scenario India is Multilingual Multi script Country with 22 languages and 11 scripts; population over 1 Billion Less than 5 percent of people can read & write English. Over 95 percent population is deprived of the benefits of English-based Information Technology Issues regarding Indian Languages • Orthography – Spelling issues • Pronunciation – may be directly mapped but not always • One script-many languages • Many languages – one Script

• Testing methodologies • Metrics for Linguistic Testing • Certification by Government for linguistic compliance • Machine Translation • Optical Character Recognition • Speech Technologies • Cross Lingual Information Retrieval • Presentation of dates, times, numbers, lists, and other values. • Collation and sorting • Alternate calendars, which may include holidays, work rules, weekday/weekend. • Currency • Tax or regulatory regime • • • Guidelines Best Practices Case Studies Consultancy Showcasing of Tools & Technologies • • Parallel Corpora Speech Corpora Lexical resources Ontologies Dictionaries Thesaurus Reference Terminologies • Certified Localization professionals • PG Specialization in Localization • Ph. D Programmes • Encoding Standards • Multimodal input device standards • Fonts & Rendering Engines • Transliteration & Translation • • Project Management Translation Memory Translation Tools Natural language for text processing: parsing, spell checking, and grammar checking etc • Automatic Testing Tools The Tree of Localization Complexities • Minimizing Time lag • Benchmarking w. r. t. English version • Political sensitivity • Pricing issues

Internationalization Activity in W 3 c The W 3 C Internationalization (I 18 n) Activity works with W 3 C working groups to make it possible to use Web technologies with different languages, scripts, and cultures. It is to ensure that W 3 C's formats and protocols are usable worldwide in all languages and in all writing systems. The Internationalization (I 18 n) Activity statement explains concepts relating to internationalization, as well as the current situation and the role within the W 3 C of the Internationalization Activity.

W 3 C Internationalization Resources Internationalization of Web design & Applications Character Model for World Wide Web Authoring Techniques for XHTML & HTML Authoring CSS Unicode in XML Internationalization of Web Architecture Language tags and Local Identifiers Internationalization Tag Set Internationalization of XML Best practices for XML Internationalization of Web Services Language tags and Local Identifiers for World Wide Web

Activities to be undertaken by W 3 C India Office v Internationalization Tag Set v Web Design and Applications v v Styling Html Xhtml Wai v Web Architecture v XML Web Architecture Web Design & Applications v Semantic Web Internationalization v OWL and RDF v XML Technology Semantic Web E-Government XML v XML associated standards v Web of Services v SOA v Web of Devices v Mobile Web Initiative v Voice v E-Government v Use cases Web of Devices Web of Services

MOBILE WEB Challenges : Adopting right encoding scheme Availability on handsets Usability of Mobile Web Browser Web support of all Indian languages Study on specific requirements for Indian languages for W 3 C Mobile Web Best Practices Must support standards and specifications Access to all handset features

MOBILE WEB Messaging Issues on Mobile Web • • Character Encoding • • Lack of standardization Bandwidth and Cost Presentation Issues • Lack of availability for all characters. • There is no guarantee that a message encoded will be displayed properly at the receiving terminal. • Issue of Multiple Script -one language not addressed. • Standardization of glyph support, syllable composition logic is also an important aspect and is dependent on the implementation level of handset manufacturer. • Legacy Systems Input Device Limitations Lack of standardization Fonts Backward Compatibility with Legacy Devices Rendering Issues

MOBILE WEB Issues in Mobile Keypads • Multi-tap issues • Too many taps per key for each char No way to know which char is on which Key • Dictionary Based issues • Difficult to learn and operate for the target segments. • Different spelling for मरत , मरत permutations. Which is the one to be mapped even मरथ , many

MOBILE WEB Suggestions • In terms of internationalization, operators must support appropriate character encoding on the signaling channel which would allow all characters of the world to be represented. • Need of investigation and study of major issues for enabling Mobile web in Indian languages • Standardization of mobile media also required to be addressed taking into consideration of specific requirements of each of Indic languages. Road Map : • Character encoding as per Unicode Standard • Enable Mobile Web in Indian languages • Initiation of study for Mobile Web Best Practices 1. 0 with respect to requirements for 4 Indian languages : Hindi, Bangla , Marathi, Tamil

PRONUNCIATION LEXICON SPECIFICATION Issues : Roadmap : Initiation of study for PLS 1. 0 with respect to requirements for 3 Indian languages : Hindi and Bangla , Tamil

Cascading Style Sheet Issues : : Drop Letter in Indian languages • Issues for Indian Languages with respect to first character used in Hindi, Malayalam, Bengali, Telugu and Gujarati etc

Cascading Style Sheet • • Underlining of characters There is some examples of Indian languages in which Matra’s are not readable due to underlining of characters

Cascading Style Sheet Vertical arrangements • • Formatting issues : Horizontal justification Bullets and Numbering

Cascading Style Sheet • Indentation of character Challenges : • Implementation of CSS standards developed by W 3 C regarding Indian languages • Standards however need to be provided to those developing CSS so that by default user could have the facility to use bulleting in his own Indic languages. Roadmap : Initiation of study for CSS 2. 0 specifications with respect to requirements for 5 Indian languages : Hindi, Bangla , Punjabi , Kannada , Tamil

E-Government Issues : • Some applications are Completely in English. • Some applications have static content in local language but forms in English. • Some applications are multi-lingual but only in limited languages (e. g. , English and only one local language). Use cases : Use Case for Land Records • As all type of data related to land, available in these land records • They are used for various planning processes although the manual maintenance of this land record does hinder in effective collation and analysis of the data contained in them.

E-Government Challenges : • To enable e-governance applications in Indian languages. • Compliance with W 3 C standards. Roadmap : § Major E-Gov Application in Indian languages need to be studied for improving access to Government through better use of the web. Target languages : Hindi, Bangla, Marathi , Telugu, Tamil

Common Locale Data Repository Screen shots of CLDR Updation : CLDR HINDI

Common Locale Data Repository Some of the Screen shots of CLDR Updation : CLDR Bengali

Common Locale Data Repository • Some of the Screen shots of CLDR Updation : CLDR Malayalam CLDR Assamese

Common Locale Data Repository DRAFT LOCALE DATA for HINDI :

Common Locale Data Repository Draft locale data for Bengali :

Common Locale Data Repository Draft locale data for Assamese :

Common Locale Data Repository Draft locale data for Malayalam :

WEB ACCESSIBILITY INITIATIVE Challenges : • Make Web content accessible to people with disabilities w. r. t Indian languages • WCAG 2. 0 Guidelines for success criteria vis-a-vis selected recommendations relevant to Indian context Roadmap : • Meet WCAG 2. 0 guidelines Initiative in India : • “Guidelines for Indian Government websites” by NIC , Govt. of India • STQC Implementing WCAG 2. 0 Accessibility through Website Quality Certification • Centre for Internet and Society developing authorized translation of WCAG 2. 0 Guidelines & techniques w. r. t Indian languages • Initiation with Hindi, Bangla, Marathi, Telugu, Tamil

Internet Domain Names (IDN’s) Issues • Spoofing issues- Homographs • Characters looks similar in address bar eg. . 1क &फ कमल फमल • No two scripts should get mixed • Normal generic rules have to be there with some added restrictions as per language demands are required • Spelling Variants eg.

• Browsers related Issues • • No backward compatibility Conversion from Unicode to Punycode is available in IE 7 and onwards Firefox directly converts Unicode to Punycode Some rendering issues in different browsers for different Indian languages • Vertical conjuncts DN Draft policy for Indian Language

Unified Language Character Table

Script Grammar Language experts from state government examined the Script Grammar of Bangla , Marathi, Gujarati, Konkani and Maithili. Some of the screen shots are shown below: Script Grammar – Bangla

Script Grammar – Gujarati

Script Grammar – Marathi

Challenges Multiplicity of Languages Evolution of Orthography Lack of Standardization Enabling Mobile web in Indian languages Initiation of study for a W 3 C recommendations with respect to requirements for Indian languages Adoption of W 3 C Internationalization standards in terms of Indian Websites should be fully W 3 C Complaint E-Gov Application in Indian languages need to be studied for better use of the web.

Integrated challenges for W 3 C India Office Language Tag Initiative in vetting / modification / developing Language Tags in all 22 official Indian languages as well as regional dialectical variation of Indian languages. CLDR • Six Languages in CLDR Hindi , Nepali, Bengali , Assamese, Malayalam and Gujarati are finalized. Other languages are in process. Revised Inscript Keyboard Layout – Enhanced to incorporate additional characters as per Unicode 5. 1. C-DAC, IBM, Microsoft & Redhat involved in this initiative.

Thank You