Using OLIF The Open Lexicon Interchange Format Susan

Using OLIF, The Open Lexicon Interchange Format Susan Mc. Cormick OLIF 2 Consortium October 1, 2004

The OLIF Format n The Open Lexicon Interchange Format XML-compliant standard n Supports exchange of lexical and terminological data for language technology applications n Handles basic exchange as well as more complex applications such as MT lexicons n

The OLIF 2 Consortium n OLIF v. 2 was developed by the OLIF 2 Consortium, a group of language technology companies and organizations interested in issues of MT data/term data exchange Led by SAP n Members include Xerox, Microsoft, Trados, IBM, Systran, IAI, DFKI and Comprendium n

Developing OLIF v. 2 n Based on OLIF prototype Developed in EC-funded OTELO project – proposing standards for users of disparate language tools n Original purpose of OLIF was to facilitate terminology exchange for industrial users of MT n

Developing OLIF v. 2 n Version 2 adapted from OLIF prototype using input from Developers/users of 3+ MT systems n Developers/users of terminology management systems n Other language standards projects: n EAGLES n SALT n ISLE n MARTIF, TBX n

OLIF Version 2 Released as open standard in 2002 n XML-compliant n Covers 6 European languages n n English, German, French, Spanish, Danish, Portuguese Includes options for modeling administrative, morphological, syntactic and semantic data

Available to Users n n XML implementation of OLIF specification in a DTD Available from OLIF 2 Consortium web site: www. olif. net

The OLIF File Follows Terminology Markup Framework (TMF) structure: n Header n Body n Shared resources

The OLIF Entry Collection of monolingual data on a specified sense of a word or phrase n Optional links for cross-reference and transfer n Transfer is bilingual and unidirectional n Multiple transfers in multiple languages possible for single word sense

Key Data Categories n The OLIF entry is uniquely identified by 5 key data categories: Canonical form n Language n Part of speech n Subject field n Semantic reading n

Basic Well-Formed OLIF Entry <entry> <mono> <key. DC> <can. Form>table</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> </mono> </entry>

<entry> <mono> <key. DC> <can. Form>table</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> <mono. DC> <mono. Admin> <originator>Weber</originator> <admin. Status>ver</admin. Status> </mono. Admin> <mono. Morph> <inflection>like book, books</inflection> </mono. Morph> <mono. Syn> <syn. Type>cnt</syn. Type> <syn. Frame>[gencomp-opt]</syn. Frame> </mono. Syn> <mono. Sem> <sem. Type>inform</sem. Type> </mono. Sem> </mono. DC> </mono> </entry>

OLIF Entry with Cross-Reference <entry> <mono> <key. DC> <can. Form>table</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> </mono> <cross. Refer> <key. DC> <can. Form>row</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>69</sem. Reading> </key. DC> <cr. Link. Type>has-meronym</cr. Link. Type> </cross. Refer> </entry>

OLIF Entry with Transfer <entry> <mono> <key. DC> <can. Form>table</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> </mono> <transfer> <key. DC> <can. Form>Tabelle</can. Form> <language>de</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> </transfer> </entry>

Data Category Values n n Allowed values specified by OLIF Administrative, terminological, linguistic values based on n General industry standards n n MT/Terminology standards n n E. g. , allowed values for date derived from recommendations from ISO 8601: 1988 E. g. , suggested values for subject field adapted from EC Widely-recognized linguistic standards n E. g. , allowed values for gender based on longstanding gender description for European languages

User Extensions: The OLIF Data Category Registry n Users may declare and use their own values for certain data categories: n n n n n Subject field Semantic reading Morphological structure Part of speech Inflection Aspect Syntactic type Syntactic frame Semantic type Concept hierarchy

Organizing Based on Concept n n Users may link monolingual entries via a concept identifier These IDs can be used to organize entries as equivalent word senses associated with the same concepts rather than source word senses associated with transfers.

Entries Linked by Concept <entry Concept. User. Id= ” 0731 F 16 CCCD 2 D 3119 B 4 D”> <mono> <key. DC> <can. Form>table</can. Form> <language>en</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> <entry Concept. User. Id= </mono> ” 0731 F 16 CCCD 2 D 3119 B 4 D”> </entry> <mono> <key. DC> <can. Form>Tabelle</can. Form> <language>de</language> <pt. Of. Speech>noun</pt. Of. Speech> <subj. Field>general</subj. Field> <sem. Reading>86</sem. Reading> </key. DC> </mono> </entry>

What’s Available to the OLIF User? n On www. olif. net n n n n n Complete XML DTD for download Hyperlinked DTD for viewing Graphical view of structure of DTD Current specification for OLIF v. 2 Formalization of OLIF data categories Alphabetic list of XML elements and attributes Fixed and recommended values for elements and attributes Guidelines formulating canonical forms Sample OLIF entries

Using OLIF n Some applications: SAP has implemented an OLIF converter to exchange terminological data from its central termbase SAPterm n MT developers in OLIF 2 Consortium currently developing OLIF converters (Comprendium, Systran) n OLIF User Forum = 60+ members n

What’s New: XML Schema OLIF XSD offers n 40+ built-in data types n Allows creation of user-defined data types n Supports inheritance

What’s New: The OLIF API n n Based on OLIF XSD, Java classes created Supports: Converting. csv files to OLIF n Converting from XML format to OLIF n Creating OLIF documents from scratch n Modifying OLIF documents n

What to Expect this Year from OLIF n n n OLIF XSD and API are available to the user from www. olif. net OLIF web site upgraded, updated Requirements for modeling Japanese entries integrated

OLIF User Forum n Users of OLIF can access and post questions, messages and sample data from the OLIF group site: http: //groups. yahoo. com/group/olif. Consortium/