LREC 2006 workshop Genova 28 May 2006 EUfunded

  • Slides: 24
Download presentation
LREC 2006 workshop Genova 28 May 2006 EU-funded research initiatives: - Interactive Systems -

LREC 2006 workshop Genova 28 May 2006 EU-funded research initiatives: - Interactive Systems - New Interfaces - Language Technology Xavier Gros DG INFSO, European Commission, Luxembourg

Contents • What has been done so far? • Projects running so far under

Contents • What has been done so far? • Projects running so far under “FP 6” • 2 nd batch of proposals under negotiation • “FP 7” info • What is it? • By when? • What’s in it for you?

Our Mission E 1 Interfaces The challenge is: • To make interfaces as simple

Our Mission E 1 Interfaces The challenge is: • To make interfaces as simple as possible, and interaction as easy as possible • For citizens to be able to access, receive and use information in their own language • • • We support the research, development and integration of: Advanced technologies for interfaces & interaction Integrated multilingual information systems and services We focus on: • Natural interaction between humans and physical or virtual environments • Multilingual communication systems for unrestricted domains, including real-time understanding of spoken and gesture input

So what happened? • Calls: 1 st in late 2002, and 5 th in

So what happened? • Calls: 1 st in late 2002, and 5 th in mid-2005 • Proposals: 169 requesting a total funding of 804 M€ • High quality: 58 asking for a total funding of 296 M€ • Projects: 29 projects for a total funding of ~136 M€ (15 start 2006) • Participants: ~320 • Type: 7 IP’s, 4 No. E’s, 17 STRe. P’s, 1 SSA

Integrated Projects 1 st Call: • CHIL - Computers In the Human Interaction Loop

Integrated Projects 1 st Call: • CHIL - Computers In the Human Interaction Loop http: //chil. server. de/ • AMI - Augmented Multi-party Interaction http: //www. amiproject. org/ • TC-STAR - Technology and Corpora for Speech to Speech Translation http: //www. tc-star. org/ 5 th Call: • • Augmented Multi-party Interaction with Distance Access • • Conveying Affectiveness in Leading-edge Living Adaptive Systems Intelligent, Persistent, Personalised Multimodal Interfaces to the Internet Multimodal Interfaces for Capturing and Transferring of Skills

Networks of Excellence 1 st Call: • SIMILAR - Federating the communities of humancomputer

Networks of Excellence 1 st Call: • SIMILAR - Federating the communities of humancomputer interaction and signal processing http: //www. similar. cc • HUMAINE - Human-Machine Interaction Network on Emotion http: //emotion-research. net • ENACTIVE - Enactive Interfaces http: //www. enactivenetwork. org And an SSA called MWEB - Multimodal Web Interaction http: //www. w 3. org/2004/MWeb/

Targeted Research 1 st call • MATRIS - Markerless Real-time Tracking for Augmented Reality

Targeted Research 1 st call • MATRIS - Markerless Real-time Tracking for Augmented Reality Image Synthesis http: //www. ist-matris. org/ • T’n. D - Touch and Design http: //www. kaemart. it/touch-anddesign/ • TAI-CHI - Tangible Acoustic Interfaces for Computer. Human Interaction http: //www. taichi. cf. ac. uk/ • DIVINES - Diagnostic and Intrinsic Variabilities in Natural Speech http: //www. divines-project. org/ • TALK - Talk and Look: Tools for Ambient Linguistic Knowledge http: //www. talk-project. org/ • HIWIRE - Human Input That Works In Real Environments http: //www. cvsp. cs. ntua. gr/projects/bin/view/HIWIRE/Web Home

Targeted Research 5 th call Statistical Multilingual Analysis for Retrieval and Translation • New

Targeted Research 5 th call Statistical Multilingual Analysis for Retrieval and Translation • New and more effective statistical approaches to machine translation and cross-lingual textual information access Multi-modal and Multi-Sensor Zero-distraction Interface • Develop a mobile human-computer interface for eye-busy and hand-busy applications in zerodistraction mode Multimodal Interactions Analysis of Users within Controlled Environments • Techniques for analysing eye gaze/fixation, eye blink and body movement in human-controlled environment interaction 3 D and Speech Interfacing Technologies • 3 D information visualisation and manipulation based upon the fusion of holographic displays, 3 D hand tracking and natural speech processing Sound and Tangible Interfaces for Novel Product Shaping • Develop a new type of interface for natural hands-free interaction with virtual shapes Statistical and Hybrid Machine Translation between all European Language • Combining statistical techniques with linguistic knowledge in machine translation for all EU language pairs subject to negotiation

Targeted Research 5 th call Open. Interface • Open. Interface is an open source

Targeted Research 5 th call Open. Interface • Open. Interface is an open source platform for rapid development of multimodal interactive systems Question Answering Learning Technologies in a Multilingual and Multimodal Environment • Open domain question-answering (QA) as a new way to search, integrating multilingual open domain QA, context-aware QA, and machine learning technologies Distant-talking Interfaces for Control of Interactive TV • Develop hands-free voice interaction for use with a interactive-TV remote control Action Recognition and Tracking based on time-of-flight Sensors • Algorithms and prototype for a time-of-flight (To. F) camera that captures 3 D information and can track head and eye movement Spoken Language Understanding in Multilingual Communication Systems • New algorithms and tools for robust spoken language understanding in multilingual telephone services subject to negotiation

E 1 FP 6 call-1 portfolio IP ~73 million EURO MULTILINGUALITY TC-STAR HIWIRE DIVINES

E 1 FP 6 call-1 portfolio IP ~73 million EURO MULTILINGUALITY TC-STAR HIWIRE DIVINES No. E SSA TALK SPEECH TECHNOLOGIES PASCAL AMI CHIL TAI-CHI INTERACTION SIMILAR HUMAINE MULTIMODALITY – NEW INNOVATIVE INTERFACES T’n D MWEB ENACTIVE MATRIS STREP

IST call 5 Closed: 21 September 2005 Budget: 63 M€ Number of proposals: 101

IST call 5 Closed: 21 September 2005 Budget: 63 M€ Number of proposals: 101 Under negotiation: 15 new projects • 4 IP: • 11 STREP: 6 -10 MEUR 3 -5 years 2 -3 MEUR 2. 5 -3 years Projects launch: around September 2006 subject to negotiation

FP 7 (2007 -2013) 4 main Specific Programmes • Cooperation Implementation of ICT in

FP 7 (2007 -2013) 4 main Specific Programmes • Cooperation Implementation of ICT in the “Cooperation” part to gain leadership in key scientific and technological areas by supporting cooperation between universities, industry, research centres and public authorities across the European Union as well as the rest of the world. • Ideas to support the truly creative scientists, engineers and scholars, whose curiosity and thirst for knowledge are most likely to make the unpredictable and spectacular discoveries that can change the course of human understanding and open up new vistas for technological progress and solving enduring social and environmental problems. • People to strengthen, qualitatively and quantitatively, human resources in R&D in EU • Capacities enhance research and innovation capacity throughout Europe http: //cordis. europa. eu. int/ist/about/fp 7. htm

Specific Programme: ICT Collaborative Research – Activities 1 st activity: Technology pillars pushing the

Specific Programme: ICT Collaborative Research – Activities 1 st activity: Technology pillars pushing the limits of performance, usability, dependability, costefficiency 2 nd activity: Integration Environments integrating multi-technology sets that underlie new functionalities, services and applications 3 rd activity: Applications research providing the knowledge and the means to develop a wide range of ICT-based services and applications and supporting research at the frontiers of knowledge Nano-electronics, photonics and micro/ nano-systems Communication Networks Embedded systems, Computing and control FET Personal environments Home environments Societal challenges Robotic systems Content, creativity personal development Intelligent infrastructures Business and industry Software, Grids, Security and dependability Knowledge, cognitive and learning systems Simulation, visualisation Interaction and mixed Realities New perspectives in ICT Drawing on other science and technology disciplines Trust and confidence

Draft FP 7 Texts • FP 7 Proposal by Commission to Council & Parliament

Draft FP 7 Texts • FP 7 Proposal by Commission to Council & Parliament „BUILDING THE EUROPE OF KNOWLEDGE” • Information & Communication Technologies • ICT Technology Pillar • Simulation, visualisation, interaction and mixed realities: tools for innovative design and creativity in products, services and digital media, and for natural, language-enabled and context-rich interaction and communication. Specific Programmes • Co-operation – ICT Simulation, visualisation, interaction and mixed realities: tools for modelling, simulation, visualisation, interaction, virtual, augmented and mixed reality and their integration in end-to-end environments; tools for innovative design and for creativity in products, services and digital audiovisual media; more natural, intuitive and easy-to-use interfaces and new ways to interact with technology, machines, devices and other artefacts; multilingual and automatic machine translation systems.

“Cooperation” – Collaborative Research – Themes Nano, materials, production: 3467 m€ (11%) ICT: 9080

“Cooperation” – Collaborative Research – Themes Nano, materials, production: 3467 m€ (11%) ICT: 9080 m€ (28. 5%) Energy: 2236 m€ (7%) Environment: 1886 m€ (6%) Food, agri, biotech: 1936 m€ (6%) Health: 5984 m€ (18%) Transport: 4150 m€ (13%) Socio-econ research: 607 m€ (2%) Space and security: 2858 m€ (9%)

Preparing for FP 7 • Interactive Systems & New Interface Technologies • 30 -31

Preparing for FP 7 • Interactive Systems & New Interface Technologies • 30 -31 January 2006, Luxembourg • Industrial Workshop on Simulation, Visualisation, Interaction, and Mixed Realities • 14 -15 February 2006, Luxembourg • Machine Translation & Human Language Technologies • 23 February 2006, Luxembourg

Some pointers … Important issues highlighted in FP 7 consultations: • Individual interaction modalities

Some pointers … Important issues highlighted in FP 7 consultations: • Individual interaction modalities need to be made far more accurate and robust. One important application challenge is the lack of robust and cost-effective hands-free and eye-free interfaces (e. g. hand-busy and eye-busy situations). • People want interaction and information access in their own language, and we must understand how to develop interfaces that are capable of dealing more efficiently and cost-effectively with different languages. A specific challenge is machine-assisted translation. • A major research challenge involves the shift from explicit interaction to implicit human-computer interaction, where the computer adapts to the user, their language and context. In addition, we must develop ways to create adaptive interfaces that enable personalised interaction. The move towards personalised interaction requires us to address ethical and privacy issues. under discussion

Some pointers … Important issues highlighted in FP 7 consultations: • We must be

Some pointers … Important issues highlighted in FP 7 consultations: • We must be able to understand model the behaviour of users in the way they interact with machines, with services, with their surrounding environment and with each other. • We must demonstrate how effective multimodal interaction helps people and we must answer the question about the benefits that multimodal interaction brings to users. One way forward is to stress-test new interaction modalities in industrial prototypes. Standards and open source software platforms are an essential prerequisite for the rapid adoption of prototype applications. • We need common evaluation and benchmarking schemes, test and training data, tools, and metrics. Open competitions are an effective way to mobilise the research community. under discussion

Some pointers … • Weak integration into real applications • Language resources: lack of

Some pointers … • Weak integration into real applications • Language resources: lack of integration and sharing • Speech understanding still difficult • MT: lack of robustness, questionable quality • No evaluation in specific real-life situations

Perspectives • Resource sharing, integration of applications, collaboration across scientific domains • Fully automatic

Perspectives • Resource sharing, integration of applications, collaboration across scientific domains • Fully automatic end-to-end MT systems • Easy to use and robust MT systems • Text and speech recognition • Linguistic infrastructure

Perspective 1: Machine Translation • Fully automatic end-to-end systems and semi-automatic machine translation systems

Perspective 1: Machine Translation • Fully automatic end-to-end systems and semi-automatic machine translation systems (spoken and written languages) to support a human translator/interpreter. Automatic multilingual speech-recognition toolkits. Initial objective is “high quality translation” of (semantically) domain-limited but otherwise (syntactically) unrestricted languages.

Perspective 2: Robustness • Easy to use machine assisted translation tools and portable and

Perspective 2: Robustness • Easy to use machine assisted translation tools and portable and robust translation devices for text-totext and speech-to-speech translation across multiple languages. Effective affordable, reusable systems. Interfaces and systems for on-line translation of web pages.

Perspective 3: Resources • Shared databases corpora and free distribution among research non-profit institutes

Perspective 3: Resources • Shared databases corpora and free distribution among research non-profit institutes • Open source toolkits and modules

Further information General FP 6: http: //europa. eu. int/comm/research/fp 6/ IST: http: //www. cordis.

Further information General FP 6: http: //europa. eu. int/comm/research/fp 6/ IST: http: //www. cordis. lu/ist FP 7: http: //www. cordis. lu/ist/about/fp 7. htm Xavier Gros e-mail: xavier. gros@cec. eu. int tel: +352 4301 37193