Tele Morph Tele Turas Bandwidth determined Mobile Multi
Tele. Morph & Tele. Turas: Bandwidth determined Mobile Multi. Modal Presentation Student: Anthony J. Solon Supervisors: Prof. Paul Mc Kevitt Kevin Curran School of Computing and Intelligent Systems, Faculty of Informatics, University of Ulster, Magee.
Objectives of Research To develop a system, Tele. Morph, that dynamically morphs between output modalities depending on available network bandwidth: n Wireless system’s output presentation (unimodal/multimodal) depending on the network bandwidth available n Implement Tele. Turas, a tourist information guide for the city of Derry n n n Receive and interpret questions from the user Map questions to multimodal semantic representation Match multimodal representation to database to retrieve answer Map answers to multimodal semantic representation Query bandwidth status Generate multimodal presentation based on bandwidth data
Wireless Telecommunications Generations of Mobile networks: n 1 G - Analog voice service with no data services n 2 G - Circuit-based, digital networks, capable of data transmission speeds averaging around 9. 6 K bps n 2. 5 G (GPRS) - Technology upgrades to 2 G, boosting data transmission speeds to around 56 K bps. Allows packet based “always on” connectivity n 3 G (UMTS) - digital multimedia, different infrastructure required, data transmission speeds from 144 K-384 K-2 M bps Positioning Systems: n GPS n DGPS n GLONASS n GSM
Mobile Intelligent Multi. Media Systems Smart. Kom n Mobile, Public, Home/office n Saarbrucken, Germany n Combines speech, gesture and facial expressions on input & output n Integrated trip planning, Internet access, communication applications, personal organising Voice. Log n BBN technologies in Cambridge, Massachusettes n Views/diagrams of military vehicles and direct connection to support n Damage identified & ordering of parts using diagrams MUST n MUltimodal multilingual information Services for small mobile Terminals n EURESCOM, Heidelberg, Germany n Future multimodal and multilingual services on mobile networks
Intelligent Multi. Media Presentation Flexibly generate various presentations to meet individual requirements of: 1) users, 2) situations, 3) domains Fine-grained coordination of communication media and modalities Key research problems: n Semantic Representation n Fusion, integration & coordination n Synchronisation
n n n Semantic representation - represents semantics w Frame-based representations: - CHAMELEON - REA w XML-based representations: - Smart. Kom - MUST Fusion, integration & coordination of modalities w Integrating different media in a consistent and coherent manner w Multimedia coordination leads to effective integrated multiple media in output Synchronisation of modalities w Time threshold between modalities E. g. Input - “What building is this? ”, Output - “This is the Millenium forum” w Not synchronised => side effect is contradiction
Intelligent Multi. Media Presentation Systems Automatically generate coordinated intelligent multimedia presentations User-determined presentation COMET n COordinated Multimedia Explanation Testbed n Generates instructions for maintenance and repair of military radio receivertransmitters n Coordinates text and 3 D graphics of mechanical devices WIP n n Intelligent multimedia authoring system presents instructions for assembling/using/maintaining/repairing devices (e. g. espresso machines, lawn mowers, modems) IMPROVISE n Graphics generation system n constructive/parameterised graphics generation approaches n Uses an extensible formalism to represent a visual lexicon for graphics generation
Intelligent Multi. Media Interfaces & Agents Intelligent multimedia interfaces n Parse integrated input and generate coordinated output n CUBRICON w Calspan-UB Research center Intelligent CONversationalist w Air Force Command Control w Generates & recognises Speech; natural language text; displays graphics; interprets gestures made with a pointing device Intelligent multimedia agents n Embodied Conversational Agents n Natural human communication - speech, facial expressions, hand gestures, & body stance n COLLAGEN w COLLaborative AGENt w object-oriented Java middleware for building collaborative interface agents w MIT Media Laboratory work on embodied conversational agents
Project Proposal Research and implement a mobile intelligent multimedia presentation system called Tele. Morph Dynamically generates a multimedia presentation determined by the bandwidth available Tele. Turas tourist navigation aid providing testbed for Tele. Morph incorporating: n route planning, maps, spoken presentations, graphics of points of interest and animations n Output modalities used n Effectiveness of communication Tele. Turas examples: n “Where is the Millenium forum? ” n “Take me to the Guild. Hall” n “What buildings are of interest in this area? ” n “Is there a Chinese restaurant in this area? ”
Architecture of Tele. Morph
Comparison of Intelligent Multi. Media Systems
Comparison of Mobile Intelligent Multi. Media Systems
Prospective Tools Development language - J 2 ME (Java 2 Micro Edition) Speech input/output - Java Speech API – IBM’s implementation of JSAPI “speech for Java” n n n US & UK English, French, German, Italian, Spanish, and Japanese Java Speech API Markup Language (JSML) Java Speech API Grammar Format (JSGF) Positioning system - GPS (Global Positioning System) provides the accurate location information necessary for a LBS (Location Based Service) Graphics input/output - The User Interface (UI) defined in J 2 ME is logically composed of two sets of APIs: n Low-level UI API n High-level UI API
Project Schedule
Conclusion A Mobile Intelligent Multi. Modal presentation System called Tele. Morph will be developed Dynamically morphing between output modalities depending on available network bandwidth n Bandwidth and Device determined Mobile Multi. Modal presentation Tele. Turas will be used as a testbed for Tele. Morph Corpora of questions to test Tele. Turas (prospective users/tourists)
- Slides: 15