Browsing Recorded Meetings with Ferret Pierre Wellner Mike

Browsing Recorded Meetings with Ferret Pierre Wellner, Mike Flynn & Maël Guillemot IDIAP MLMI, Martigny, June 21 st, 2004

Outline • The browsing task • Observations of interest • Ferret Meeting browser • Features • Demo • Architecture • How to evaluate

Meeting 2. Analysis 1. Recording annotations media 3. Browsing Quickly find out what happened in meetings

The. Media Meeting Browsing Task The Browsing Task Find a maximum number of observations of interest in a minimum amount of time. But what are “observations of interest”?

Observations of Interest • May not match researcher interest • (e. g. gesture, emotion, speech, documents, etc. ) • Depends on specific meeting and individuals. • Initial pilot collection indicate: • Most are associated with a specific participant. • Most about spoken content. • Substantial agreement between observers.

Ferret Browser • Control media playback. • Display user-selected XML annotations. • Data “basket” allows loading of data from any web server. • Zooming (handles long and short meetings). • Uses normal IE browser (with Real. Player, SVG).

Browsable annotations Speaker segmentations (who is talking). Speech transcripts. • Captured screens. • Meeting actions (e. g. presentation, discussion) • Still to do / in progress • Whiteboard & paper strokes • Magnitudes, directions • Phoneme data?

Meeting selection mmm. idiap. ch

Selection of data sources demo

Ferret Architecture Media Processin g ggg XML Data Various Text Transcripts Real Server Real Player XMLto. SVG Servlets, Servlet CGI & JSP SVG Viewer Apache Tomcat Internet Explorer Server Client

Aims a good Is for it any good? BET We want a Browser Evaluation Test (BET) with the following properties: • • Performance, not only judgment. Independent of experimenter perception. Directly comparable numeric scores. Replicable.

BET overview recording system meeting participants corpus playback system media browser subjects observers test sampling answers observations scoring scores

Summary • The browsing task • Ferret • Features • Demo • Architecture • How to evaluate