YALA Yet Another Localization Automation Yala changes the

YALA Yet Another Localization Automation Yala changes the way we do localization Ko-Haw K. Nieh, Jay Tong, Michele Carlson March 2007 Yahoo! Confidential

Topics § § § Objectives Concepts Data Flow Diagram Implementation Daily Cron Jobs Imports and Exports Yala CLI Yala Web Interface Yala Web Admin Interface One URL Does It All Yala Answers These Questions Demo 2 Yahoo! Confidential

Objectives? § § § § Automate localization process Connect developers to the localization process Connect translation vendors to the process Streamline localization workflow Locate translations quickly and precisely Track translation vertically and horizontally Track translation modification history Up-to-date operation status are always available 3 Yahoo! Confidential

Concepts § § § § § Automation Central Collaborative Database Scalability Templates Language/Template Inheritance Toggles Out-of-date Translations Full Scale of Process Recording and Monitoring ACL One URL Does It All 4 Yahoo! Confidential

Concept – Automation § The Yala automation is achieved by: - A central collaborative database A collection of cron jobs A web based user interface A set of CLI scripts 5 Yahoo! Confidential

Concept – Central Collaborative Database § § Use a central database to collaborate and to coordinate people and process across many time zones. One Yala installation has the capability to serve multiple products. Up-to-date translations are always available. Yala connects developers, build masters, QAs, Intl PMs, linguists, translators, translation vendors, and managements seamlessly. 6 Yahoo! Confidential

Concept – Scalability § § § Localize products to additional languages require no additional engineering resource. Once a product is setup with Yala, no additional l 10 n engineering resource is required. One Yala installation servers multiple products. 7 Yahoo! Confidential

Concept -- Templates § § § Templates are the basis of Yala. All localizable strings are marked within <!!>…</!!> pair. It is suggested to use resource identifiers instead of actual English strings in <!!>…</!!> pair. Templates can be specialized per language. Templates can include other templates. 8 Yahoo! Confidential

Concept – Language/Template Inheritance § § Yala provides both language and template inheritance mechanisms. It is suggested not to over use the template inheritance. (Products use complicated template inheritance are difficult to localize. ) 9 Yahoo! Confidential

Concept – Toggles § § § Provides flexibilities with various international rollouts. Replaces the usage of product configuration facilities. Provides a clear separation of responsibilities between developers and localization. 10 Yahoo! Confidential

Concept – Out-of-date Translations § § Out-of-date translations are caused by modifications of English strings after translations have been committed. Yala captures out-of-date translations. Translators can update out-of-date translations easily. 11 Yahoo! Confidential

Concept – Recoding and Monitoring § § § Full scale process recording and monitoring. All transactions to Yala database are recorded. The life cycle of a piece of translation is fully documented. Following questions are answered. - When the translation was created? Who modified the translation? When was the modification occurred? What was the reason for modification? 12 Yahoo! Confidential

Concept – One URL Does It All § Yala web based UI is simple and selfexplanatory. 13 Yahoo! Confidential

Concept – ACL § Yala ACL is a facility to control which user has permissions to access which databases and which languages. 14 Yahoo! Confidential

What Developers Need to Do? § Maintain template files - Put localizable strings between <!!>…</!!> pair - Put configurable parameters in Yala toggles - Reshuffle resource ID among template files as needed § § § Run yala_assign. pl to associate resource ID with English strings Run yala_gen. pl to generate localized files Use Yala UI to refine English strings 15 Yahoo! Confidential

What Have Been Automated? § § § § Daily Vendor kits are generated at midnight daily. Translation vendors can download vendor kits and upload translations by themselves. Manual processing of vendor kits is eliminated. Intl PMs can use Yala toggles to configure their products by themselves. Out-of-date translations are reported on-the-fly. Up-to-date statistics, such as untranslated strings, localizable strings, template files, etc. are presented on your fingertips. Translations can be searched vertically and horizontally. Localize a product into additional languages require no extra engineering resource. Reduce the amount of email traffic to coordinate workflow. 16 Yahoo! Confidential

Yala Data Flow Diagram Templates + English Strings Developers Yet Another Localization Automation (Yala) M M Browse Translations Import/Export/Edit Translations M Translated Files (de, fr, kr, gb, . . . ) M M N 1. Vendor Kits 2. Out-of-date translations Browser Translators M Monitor points: Web pages generated by either cron jobs or recording Requires human intervention 17 Yahoo! Confidential

Implementation § § My. SQL database Yapache Server - PHP enabled - cgi-bin configured Perl/DBI, Perl/Digest: : MD 5, Perl/HTTP, Perl/LWP All translations stored in the database are in UTF-8 encoding. 18 Yahoo! Confidential

Daily Cron Jobs § § Generate l 10 n files and vendor kits Produce web pages to report - § Changes on template files Changes on l 10 n files Changes on English Strings Changes on Yala databases Daily vendor kits Statistics, such as number of files, total localized words, etc. Backup all databases 19 Yahoo! Confidential

Imports and Exports § § It is not desirable to edit a large amount of translations via web UI, therefore Yala provides import and export features. Yala supports three import/export formats, - Yala - XLIFF - CSV § Yala provides both CLI and Web UI to perform import and export operations. 20 Yahoo! Confidential

Example – Yala File Format #index: 3 #filename: data/strings. xml #language: us #project: candygram #id: a 7641 c 190511835 f 095 fcec 3 b 5 c 1 d 7 ac #origin: yala workbench #status: new #occurrence: 0 #comments: #source: string_yahoo_mail #english: Yahoo! Mail #l 10 n_string: Yahoo! Mail #end-of-record: 21 Yahoo! Confidential

Example – XLIFF Format § § Yala external files can be converted to xliff format and vice versa. Example <trans-unit id="8 e 0 ff 7094 c 468 e 823 aab 597562 a 67 ff 2" resname="string_unknown_response_error_dialog_title" approved="yes" extradata="data/strings. xml" extype="0"> <source><![CDATA[Unexpected Server Response]]></source> <target><![CDATA[Unexpected Server Response]]></target> <note><![CDATA[]]></note> </trans-unit> 22 Yahoo! Confidential

Yala -- CLI § Yala CLI is designed for - developers to generate localized files and to insert new English strings to Yala database. - buildmaster to pull latest translations from Yala database and to build most up-to-date product. § Based on the philosophy of “One URL Does It All”, all other users should not use CLI at all. 23 Yahoo! Confidential

Yala CLI (scripts) - yala_gen. pl yala_import. pl yala_export. pl yala_assign. pl (generate l 10 n files & vendor kits) - yala_leverage. pl (leverage from other translations) - rosetta 2 yala. pl - babel 2 yala. pl (convert Rosetta to yala) (import translations) (export translations) (submit a single English string) (convert Babel to yala) 24 Yahoo! Confidential

Yala Web UI Interface § One URL does it all. § http: //yala. mail. mud. yahoo. com/yala. php 25 Yahoo! Confidential

Yala – Admin Interface § Yala administrators use the admin interface - To create Yala database for new products To add new users To add new languages To execute on-demand cron jobs To modify inheritance chain To lock databases To modify ACL (Access Control List) 26 Yahoo! Confidential

One URL Does It All - Locate translations with regular expression search Edit translations Export daily vendor kits Export search results Import translations View localized files on-the-fly View statistics View import logs View/edit translations across languages Search modification history Study differences between languages Manage toggles More … 27 Yahoo! Confidential

Yala Answers These Questions § § § How many localizable strings/words for a product? How many un-translated strings/words at present time? How many translations have been modified (who, when, why)? How many templates are there in a product? How many specialized templates are there for a language? How many localizable strings are there in a template? How many out-of-date translations? What are they? How many new English strings have been introduced yesterday? What are the differences between ca and us translations? How many tw translations have been modified in last two weeks? More … Yahoo! Confidential 28

Future – Road Map § § § § Connect Yala database to communities. Allow communities to contribute and to improve translations. Provide Yala Web Services API Integrate Yala with TRADOS. Spread Yala databases over multiple machines. Establish Yala User’s Group. Localize Yala itself. More … 29 Yahoo! Confidential

Which Yahoo! Products Use Yala? - Candygram Mail - the new Yahoo! Mail Classic Mail Addressbook Calendar Photos OCP - RSS My Yahoo! Go Groups Profiles/360 Vespa Yahoo! for Good Messenger 30 Yahoo! Confidential

Where to Start? § Visit http: //yala. mail. mud. yahoo. com/yala. php. § Talk to us to migrate your current projects to Yala. Talk to us to setup your new projects to use Yala. § Mcarlson@yahoo-inc. com Michele Carlson Jaytong@yahoo-inc. com Jay Tong Niko 2@yahoo-inc. com Ko-Haw Nieh 31 Yahoo! Confidential

Demo § YALA Home page http: //yala. mud. yahoo. com/yala. php Search (regular expression search, export, view cross language translations) Search modification history Toggles Browse l 10 n files Import log Daily vendor kits Out-of-date translations Inconsistent translations 32 Yahoo! Confidential