Synchronized Multimedia Interface Language SMIL and W 3

Synchronized Multimedia Interface Language (SMIL) and W 3 C Multimedia E-Commerce Course November 14, 2002 Mike Christel

World Wide Web Consortium • W 3 C, founded in October 1994: http: //www. w 3. org/ • Purpose: develop common protocols that promote WWW’s evolution and ensure its interoperability, in part by providing: • A repository of information about the WWW for developers and users, especially specifications about the web • A sample code implementation to embody and promote standards • Various prototype and sample applications to demonstrate use of new technology © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 2 Carnegie Mellon

More on W 3 C • Directed by Tim Berners-Lee, inventor of the WWW • 452 members as of 11/11/2002 (514 members as of 11/2001, 481 members as of 11/2000; 370 Members as of 11/1999) • vendor-neutral forum • http: //www. w 3. org/Consortium/Member/List • Specifications must be formally approved by the membership via W 3 C Recommendation track © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 3 Carnegie Mellon

W 3 C Recommendation Track • Working Draft Work in progress, commitment to pursue work in area • Candidate Recommendation Proposed for implementation experience and feedback • Proposed Recommendation Sent to the Advisory Committee for review • Recommendation Consensus within W 3 C, appropriate for widespread deployment and promote W 3 C's mission (source: http: //www. w 3. org/Consortium/Process/) © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 4 Carnegie Mellon

Some W 3 C Specifications • User Interface Domain Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), Document Object Model (DOM), Scalable Vector Graphics (SVG), SMIL • Technology and Society Domain Platform for Privacy Preferences (P 3 P), Platform for Internet Content Selection (PICS), Resource Description Framework (RDF) • Architecture Domain Hypertext Transfer Protocol (HTTP), Extensible Markup Language (XML) . . . Follow links at http: //www. w 3. org/ for more details… © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 5 Carnegie Mellon

XML: A Quick Introduction • XML is a method for putting structured data in a text file extensible, support for internationalization/localization, platform-independent • XML makes use of tags in ‘<‘ ‘>’ but is not HTML meaning of tags like “<p>” and attributes left to the application that reads it • XML is text, not binary, but is not meant to be read XML has strict parsing rules; more verbose that binary but zip/gzip, cheap storage, etc. minimize that disadvantage • XML is license-free, platform-independent and wellsupported, a W 3 C standard since February 1998 © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 6 Carnegie Mellon

SMIL 1. 0: Synchronized Multimedia Integration Language • SMIL 1. 0 specification, June 15, 1998, available at: http: //www. w 3. org/TR/REC-smil/ • W 3 C SMIL references available at: http: //www. w 3. org/Audio. Video/ • Purpose: a new language for scheduling multimedia presentations where audio, video, text and graphics are combined in real-time • Result: an XML application enabling authors to specify what should be presented when XML implies: extensible, support for internationalization and localization, platform-independent, editable by hand, etc. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 7 Carnegie Mellon

SMIL 2. 0 a W 3 C Recommendation as of August 7, 2001, available at: http: //www. w 3. org/TR/smil 20/, with two design goals: • Define an XML-based language that allows authors to write interactive multimedia presentations • describe the temporal behavior of a presentation • associate hyperlinks with media objects • describe the layout of the presentation • Allow reusing of SMIL syntax and semantics in other XML-based languages, in particular those that need to represent timing and synchronization • SMIL 2. 0 components are used for integrating timing into XHTML and into SVG © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 8 Carnegie Mellon

SMIL Timeline Oct 1996: Workshop Mar 1997: W 3 C working group on synchronized multimedia Jun 1998: W 3 C Recommendation (SMIL 1. 0) Sep 1998: Timed Interactive Multimedia Extensions for HTML (HTML+TIME), W 3 C Note http: //www. w 3. org/TR/NOTE-HTMLplus. TIME Feb 1999: 2 nd Working Group on Synchronized Multimedia Modules (SYMM) formed http: //www. w 3. org/TR/NOTE-SYMM-modules Aug 1999: First working draft (SMIL “Boston”) released © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 9 Carnegie Mellon

SMIL Timeline, Continued Dec 1999: Chinese Translation of SMIL 1. 0 Jan 2000: Real. Slideshow 2. 0 from Real. Networks Jan 2000: Internet Explorer 5. 5 Preview by Microsoft (supports selected modules of SMIL “Boston” draft) Jan 2000: Apple Quick. Time 4. 1 now a SMIL 1. 0 player http: //www. w 3. org/TR/NOTE-HTMLplus. TIME Jun 2000: Fourth Working Draft of SMIL “Boston” released Oct 20, 2000: Last call for comments on Public Working Draft of SMIL 2. 0 © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 10 Carnegie Mellon

SMIL Timeline, Continued March 2001: New SMIL 2. 0 Public Working Draft June 2001: SMIL 2. 0 Proposed Recommendation August 2001: XHTML + SMIL Profile Working Draft August 2001: Microsoft Internet Explorer 6. 0 includes implementation of XHTML + SMIL Profile Working Draft August 2001: W 3 C Recommendation (SMIL 2. 0) Sept. 2001: SMIL Animation a W 3 C Recommendation Sept. 2001: Real. Networks releases Real. One Platform with full support for SMIL 2. 0 Jan. 2002: W 3 C Note for XHTML + SMIL Profile, http: //www. w 3. org/TR/XHTMLplus. SMIL/ © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 11 Carnegie Mellon

SYMM-WG Contributors Synchronized Multimedia Working Group contributing organizations: Compaq, CSELT, CWI, France Telecom, Gateway, Glocomm, IBM, INRIA, Intel, Macromedia, Microsoft, Netscape/AOL, NIST, Nokia, Oratrix, Panasonic, Philips, Real. Networks, WGBH, W 3 C See http: //www. w 3. org/Audio. Video/Activity © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 12 Carnegie Mellon

Example of Contributing Org. Agendas • WGBH a contributor to Synchronized Multimedia Working Group • WGBH partner in CPB/WGBH National Center for Accessible Media http: //ncam. wgbh. org/aboutncam. html • Mission: “expand access to present and future media for people with disabilities…” • Future directions include audio descriptions of content for the blind, via new functionality provided in SMIL 2. 0 http: //ncam. wgbh. org/webaccess/pivot/accesslist. html © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 13 Carnegie Mellon

SMIL 1. 0 Modules • Structure • Meta (“who”, “what”) • Layout (“where”) • Timing and Synchronization (“when”) • Media Elements (“what”) • Linking • Content Control © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 14 Carnegie Mellon

SMIL High Level Document Structure <smil> <head> <meta> <!-- … information about the document … --> </meta> <layout> <!-- … definitions used for the spatial layout … --> </layout> </head> <body> <!-- … objects, temporal relations, links … --> </body> </smil> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 15 Carnegie Mellon

SMIL 1. 0 Structure <smil> <head> </head> <body> </smil> These elements used in documents based upon traditional SMIL profiles. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 16 Carnegie Mellon

SMIL 1. 0 Meta <smil> <head> <meta … /> </head> </smil> The meta elements contain information describing the document, either to inform the human user or to assist some automation, e. g. , <meta name="title" content=”My Italy Trip"/> <meta name="copyright" content="© 1998 WGBH" /> <meta name="base” content="http: //test. edu/Italy/"/> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 17 Carnegie Mellon

SMIL 1. 0 Layout <smil> <head> <layout> <root-layout … /> <region id=“R 1” … /> <region id=“R 2” … /> </layout> </head> </smil> Includes the <layout>, <root-layout> and <region> elements, and related attributes. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 18 Carnegie Mellon

SMIL 1. 0 Layout Examples <root-layout height="300" width="450" backgroundcolor="#FFFFFF" title="Venezia!"/> <root-layout height="450" width="625" backgroundcolor="black"/> <region id="title" left="5" top="150" width="400" height="200" z-index="1"/> <region id="videoregion" top="0" left="0" height="240" width="352"/> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 19 Carnegie Mellon

SMIL 1. 0 Timing and Synchronization <smil> <body> <!-- … timing included here … --> </body> </smil> • Sequence and parallel timelines, via <seq> and <par> • Timing control properties, via “begin”, “end”, “repeat”, “dur” • “The accuracy of synchronization between the children in a parallel group is implementation-dependent” • “soft synchronization” vs. “hard synchronization” • how to realize hard synchronization © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 20 Carnegie Mellon

Hard vs. Soft Synchronization • Hard synchronization: player synchronizes the children in the "par" (parallel play) element to a common clock • Soft synchronization: each child of the "par" element has its own independent clock audio video Hard sync. case 1: stop audio, e. g. , wait to regain lip synchronization Hard sync. case 2: drop video frames, e. g. , keep running narrative audio video © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann Soft sync: audio unaffected 21 Carnegie Mellon

SMIL 2. 0 New Focus on Sync. Behavior <par> <animation src=". . . " />. . . <par id="speech" sync. Behavior="can. Slip" > <video src="speech. mpg" sync. Behavior="locked" /> <audio src="speech. au" sync. Behavior="locked" /> </par>. . . </par> The sync. Tolerance attribute is referenced when sync. Behavior is “locked”, to ignore a given amount of slew without forcing resynchronization The sync. Master attribute defines the element that acts as the playback "clock" for the time container © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 22 Carnegie Mellon

SMIL 1. 0 Media Elements <smil> <body> <!-- … media elements included here … --> </body> </smil> • Includes the media declaration elements <text>, <img>, <audio>, <video>, <textstream>, <animation>, and <ref> • all absolutely positioned elements (animation, img, ref, textstream and video) are contained within a single containing block defined by the root-layout element © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 23 Carnegie Mellon

SMIL 1. 0 Linking <smil> <body> <!-- … linking included here … --> </body> </smil> • Includes the <a> and <anchor> elements, e. g. , <a href="http: //www. w 3 c. org" > <text src="media/w 3 c. txt" region="w 3 c” begin="14. 05 s" dur="15. 95 s" /> </a> • Timing applied to HTML <a> and <area> tags could provide much or all of SMIL functionality; hence, linking modules under review © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 24 Carnegie Mellon

SMIL 1. 0 Content Control <smil> <head> <!-- … switch element can be used here … --> </head> <body> <!-- … switch element can be used here too … --> </body> </smil> Content control based on “<switch>” and test attributes: system-bitrate, system-captions, system-language, system-overdub-or-caption, system-required, system -screen-size, system-screen-depth © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 25 Carnegie Mellon

Advantages of Content Control • Different connections may have different speeds system-bitrate • Different workstations may have different capabilities system-language, system-overdub-or-caption, systemrequired, system-screen-size, system-screen-depth • Different users may have different needs or media preferences system-captions, system-language, system-overdub-or -caption © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 26 Carnegie Mellon

SMIL 1. 0 Example with <switch> <smil> <head> <switch> <meta name="title" content="CNN Skim, CC via Real. Text" system-captions="on"/> <meta name="title" content="CNN Skim, CC off by player request"/> </switch> <meta name="author" content="Mike Christel (using CNN footage)"/> <meta name="copyright" content="©Cable News Network" /> <switch> <layout system-captions="on"> <root-layout background-color="black" height="340" width="352" title="CNN Skim, CC via Real. Text"/> <region id="videoregion" background-color="black" top="0" left="0" height="240" width="352"/> <region id="textregion" top="245" left="10" height="90" width="330"/> </layout> <root-layout background-color="black" height="240" width="352" title="CNN Skim, CC off by player request"/> <region id="videoregion 1" background-color="black" top="0" left="0" height="240" width="352"/> </layout> </switch> </head> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 27 Carnegie Mellon

SMIL 1. 0 <switch> Example, Continued <body> <par> <!-- VIDEO --> <switch> <video src="cwv 0 t 4_skm. mpg" region="videoregion" system-captions="on"/> <video src="cwv 0 t 4_skm. mpg" region="videoregion 1"/> </switch> <!-- CAPTION --> <textstream src="CNNskim. rt" region="textregion" system-captions="on" dur="13 s" /> </par> </body> </smil> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 28 Carnegie Mellon

SMIL 1. 0 Modules, Revisited • Structure • Meta (“who”, “what”) • Layout (“where”) • Timing and Synchronization (“when”) • Media Elements (“what”) • Linking • Content Control SMIL 1. 0 ~40 pages, while SMIL 2. 0 ~400 pages… © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 29 Carnegie Mellon

SMIL 2. 0 Modules • Animation Module • Content Control Modules • Layout Modules • Linking Modules • Media Object Modules • Metainformation Modules • Structure Module • Timing and Synchronization Module • Transition Effects Module (see http: //www. w 3. org/TR/smil 20/ for SMIL 2. 0 details) © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 30 Carnegie Mellon

SMIL Enhancements • Layout: “media layout can be described using the either SMIL basic layout syntax or CSS 2 syntax (note that these are not functionally identical)” • Timing and Sync. : soft/hard synchronization performance • Transitions: wipes, fades, dissolves, etc. • Animation: motion paths, rendering filters like position, color, size, rotation • Accessibility (closed captioning, audio descriptions, localization) http: //ncam. wgbh. org © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 31 Carnegie Mellon

SMIL Status Today • SMIL 1. 0 as W 3 C Recommendation, with players: Real. Networks G 2 (http: //www. realnetworks. com/) Helio SOJA written in Java (http: //www. helio. org/) GRi. NS (http: //www. oratrix. com/GRi. NS/index. html) Apple Quick. Time 4. 1 (http: //www. apple. com/quicktime/authoring/qtsmil. html) • SMIL 2. 0 as W 3 C Recommendation, with players: Real. Networks Real. One Player, Oratrix GRi. NS HTML+TIME in Microsoft IE SMIL conference (Nov. 2002) and book (http: //www. smilbook. com/tools/index. html) • SYMM Interest Group work continues for maintaining SMIL specification… Follow latest status via http: //www. w 3. org/Audio. Video/ © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 32 Carnegie Mellon

SMIL Resources • Validator for SMIL 1. 0 DTD, SMIL 2. 0 DTD http: //www. cwi. nl/~media/symm/validator/ • Vendor notes/articles on SMIL Real. Networks, http: //service. real. com/help/library/guides/production 8/htmfiles/ smil. htm Microsoft, http: //msdn. microsoft. com/library/default. asp? url=/library/en-us/dntime/htmltime. asp Apple, http: //www. apple. com/quicktime/authoring/qtsmil. html • Other articles/examples http: //www. webreview. com/2000/05_05/designers/05_05_00_1. shtml http: //www. webtechniques. com/archives/1998/09/bouthillier/ • PRIMARY REFERENCE: http: //www. w 3. org/Audio. Video/ © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 33 Carnegie Mellon

For More Details. . . • Just. SMIL at http: //smw. internet. com/smil/tutor/ • Check URLs referenced here • View other online SMIL tutorials • Use products that are newly released or soon to be released, e. g. , • Real. Networks SMIL 2. 0 support via Real. One Platform; SMIL 1. 0 support via Real. Slideshow, Real. Text, etc. • GRi. NS support for SMIL 2. 0 at http: //www. oratrix. com/GRi. NS • Apple Quick. Time SMIL support • Microsoft IE 5. 5 and 6 browser support for SMIL via HTML+TIME http: //msdn. microsoft. com/workshop/Author/behaviors/htmltime. asp © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 34 Carnegie Mellon

From W 3 C SMIL Recommendation to other W 3 C Recommendations • XSLT – Extensible Stylesheet Language Transformations • XPath – XML Path Language • XML Schema • SVG – Scalable Vector Graphics • Context will be Informedia Digital Video Library • Supported by NSF DLI-2 and NSDL Programs • Images in later slides © 1999 -2001, Cable News Network © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 35 Carnegie Mellon

Context: Informedia Digital Video Library © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 36 Carnegie Mellon

Sample XML for Informedia Results Set <? xml version="1. 0" ? > <IDVLResults xmlns: im="x-schema: idv. Res. Schema. xml"> <im: doc> <im: id>55195</im: id><im: pos>0</im: pos><im: shot>1667499</im: shot> <im: d_yr>1999</im: d_yr><im: d_mo>12</im: d_mo><im: d_day>2</im: d_day> <im: score>100</im: score><im: dur>29930</im: dur><im: mmss>0: 29</im: mmss> <im: title>The Maker of Cherry Garcia and Rainforest Crunch Ice Cream Could Soon be Scooped Up; Ice Cream Maker Didn't Name Potential</im: title> </im: doc><im: doc> <im: id>54770</im: id><im: pos>1</im: pos><im: shot>2111578</im: shot> <im: d_yr>1999</im: d_yr><im: d_mo>11</im: d_mo><im: d_day>19</im: d_day> <im: score>70</im: score><im: dur>196480</im: dur><im: mmss>3: 16</im: mmss> <im: title>So, While Ice Cream Parlor Closed, in Suite Next Door to where Tim and Heidi Manipulate Bones, Tim and Heidi now also Fill Jars </im: title> </im: doc></IDVLResults> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 37 Carnegie Mellon

Adding Style to XML • XML only text-based, but Informedia presentation is visually rich; also ideally tailorable • Extensible Stylesheet Language (XSL) and XSL Transformations (XSLT) available (XSL as working draft, XSLT as W 3 C Recommendation Nov. 16, 1999, http: //www. w 3. org/TR/xslt) • XSLT: a language for transforming XML documents into other XML documents • XPATH: a language for addressing parts of an XML document, designed to be used by both XSLT and XPointer © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 38 Carnegie Mellon

XSLT for Informedia Results Display <xsl: template match="IDVLResults"><xsl: for-each select="im: doc"> <xsl: sort select="im: score" order="descending" data-type="number" /> <span class="result. Stamp" tipshown="false" id="R{im: pos}"> <xsl: attribute name="tip"> <xsl: value-of select="im: title" />, <xsl: value-of select="im: mmss" />, <xsl: value-of select="im: d_mo" />-<xsl: value-of select="im: d_day" />-<xsl: value-of select="im: d_yr" /></xsl: attribute> <img id="Stamp_{im: pos}" src="graphics/Gstamp. gif" alt="" orgsrc="graphics/Gstamp. gif" oversrc="graphics/Gltstamp. gif" width="112" height="91" /> <xsl: variable name="n. Score" select="0. 8*im: score" /> <xsl: variable name="n. Score. Top" select="85 - $n. Score" /> <xsl: variable name="s. Score. Ht" select="format. Number($n. Score, '#. ')" /> <img id="Th_{im: pos}" src="graphics/red. gif" alt="" onmouseover="stamp. Change. Over(); " onmouseout="stamp. Change. Out(); "> <xsl: attribute name="style">position: absolute; left: 9; width: 4; top: <xsl: value-of select="format. Number($n. Score. Top, '#. ')"/>; height: <xsl: value-of select="$s. Score. Ht"/>; </xsl: attribute></img> <img id="I_{im: pos}" src="http: //test. edu/idv. Query/Get. Shot. asp? {im: shot}" style="position: absolute; left: 23; top: 9" alt="" width="80" height="55" onmouseover="stamp. Change. Over(); " onmouseout="stamp. Change. Out(); " /> </span></xsl: for-each></xsl: template> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 39 Carnegie Mellon

XSLT for Informedia, larger font, 1 of 2 <xsl: template match="IDVLResults"> <xsl: for-each select="im: doc"> <xsl: sort select="im: score" order="descending" data-type="number" /> <span class="result. Stamp" tipshown="false" id="R{im: pos}"> <xsl: attribute name="tip"> <xsl: value-of select="im: title" />, <xsl: value-of select="im: mmss" />, <xsl: value-of select="im: d_mo" />-<xsl: value-of select="im: d_day" /><xsl: value-of select="im: d_yr" /></xsl: attribute> <img id="Stamp_{im: pos}" src="graphics/Gstamp. gif" alt="" orgsrc="graphics/Gstamp. gif" oversrc="graphics/Gltstamp. gif" width="112" height="91" /> <xsl: variable name="n. Score" select="0. 8*im: score" /> <xsl: variable name="n. Score. Top" select="85 - $n. Score" /> <xsl: variable name="s. Score. Ht" select="format. Number($n. Score, '#. ')" /> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 40 Carnegie Mellon

XSLT for Informedia, larger font, 2 of 2 <img id="Th_{im: pos}" src="graphics/red. gif" alt="" onmouseover="stamp. Change. Over(); " onmouseout="stamp. Change. Out(); "> <xsl: attribute name="style">position: absolute; left: 9; width: 4; top: <xsl: value-of select="format. Number($n. Score. Top, '#. ')"/>; height: <xsl: value-of select="$s. Score. Ht"/>; </xsl: attribute> </img> <img id="I_{im: pos}" src="http: //test. edu/idv. Query/Get. Shot. asp? {im: shot}" style="position: absolute; left: 23; top: 9" alt="" width="80" height="55" onmouseover="stamp. Change. Over(); " onmouseout="stamp. Change. Out(); " /> </span></xsl: for-each></xsl: template> © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 41 Carnegie Mellon

XML Namespaces and Schemas • Used for validation and data typing • Informedia example: • XML documents begin with: <IDVLResults xmlns: im="xschema: idv. Res. Schema. xml"> • XSL documents can then do: <xsl: sort select="im: score" order="descending" data-type="number" /> • So, Informedia web presentation consists of: • XML data, validated and typed via XML schema, converted via XSLT to XHTML for browser presentation © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 42 Carnegie Mellon

Scalable Vector Graphics (SVG) • World Wide Web Consortium (W 3 C) Recommendation as of Sept. 4, 2001 (http: //www. w 3. org/Graphics/SVG) • A language for describing two-dimensional graphics in XML • Vector graphic shapes • Images • Text • Drawings can be dynamic and interactive • SVG provides an ideal framework for presenting interactive summarizations into a multimedia information repository © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 45 Carnegie Mellon

One Style of Web Presentation Architecture Informedia Database DB Request Results Web Server XML Conversion and Validation Query/Browse Processor Request XML XSL Processor HTML Thin XML (e. g. , Dublin Core) Web Browser © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann XSL Style Sheet 46 XHTML Basic Carnegie Mellon

Flexibility, Greater Interactivity through Client -Side Processing Query/Browse Processor DB Request XML Conversion and Validation Results Informedia Database Web Server Request XML XSL Processor XSL Style Sheet Web Browser © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann HTML View 1 47 … HTML View n Script to add to SVG DOM SVG View s Carnegie Mellon

SVG Map for Entering Geographic Queries • SVG map can be panned, zoomed within client • Can select specific countries and states/provinces, or perform a region-based search, as shown here: © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 48 Carnegie Mellon

XML Video Results Set as Thumbnail Grid © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 49 Carnegie Mellon

Same Results Set Drawn as SVG VIBE Plot

Same Results Set Drawn as SVG Map • Date filter illustrates one way to make SVG interactive © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 51 Carnegie Mellon

Overview-Zoom-Details Support via SVG • Overviews like world maps or Visualization by Example (VIBE) plot can naturally be zoomed to smaller areas which can then be rendered larger and in greater detail, since SVG supports vector-based drawing: © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 52 Carnegie Mellon

The Need for Visualization Strategies • As digital video library grows, so do result sets • As automated processing techniques improve, e. g. , speech and image processing, more metadata with which to build interfaces into video • Need overview capability to deal with greater volume • Prior work offered many solutions: • Visualization By Example (VIBE) for matching entity relationships • Scatter plots for low dimensionality relationships, e. g. , timelines • Dynamic query sliders for direct manipulation of plots • Colored maps for geographic relationships • SVG can deliver such interactive visualizations © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 53 Carnegie Mellon

Another Example, using Documentary Data © Copyright 2002 Michael G. Christel and Alex Hauptmann

Filtering through VIBE SVG Interaction Move anchor words, zoom into 8 documents for “(water AND management) AND (irrigation OR dam)” through mouse interaction. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann

Manipulating Filtered Data with SVG Map Select “North Dakota” and see remaining 2 video clips, so that grid of 100 thumbnails for result set is now reduced to an easily navigable grid of 2 clips: © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 56 Carnegie Mellon

Benefits of SVG • Quick generation • Typically much smaller than raster pixel-based image • Clean rendering at different scaling factors (vector-based) • Efficient manipulation (e. g. , XPath navigation of SVG tree allows colors and sizes to be changed for specific elements) • Standard ways for zooming into and out of regions of interest • W 3 C Recommendation, well integrated with other W 3 C Recommendations (XML, XPath, XSLT, SMIL Animation) © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 57 Carnegie Mellon

What’s Next… Better Integrating Imagery © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 58 Carnegie Mellon

Future SVG VIBE with More Detail As document count drops when zooming into an area, add representative thumbnails for those remaining documents © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 59 Carnegie Mellon