Different Levels of Interaction Passive only visualization Reactive

Different Levels of Interaction : -Passive : only visualization -Reactive : limited inetraction ( e. g. , scroll pane functionality). -Proactive : choose a path or make selections (e. g. button) -Reciprocal: corresponds to user authoring of information

Interaction Models : -Navigational: choose to dicide where to go next -Design: user can modify the visual style of the presentation (e. g. colors) -Movie: User can control the global time (e. g. VCR capabilities)

• Tradionally multimedia presentations did not have that much logic: -Virtual visit to a museum, DVD menus …… • Real – time interactive systems: -Virtual Reality worlds, games • Application logic needs of a programming language (if, case, goto, …. . ) -Compiled languages C, C++ -Virtual Machine: Java -WWW, MPEG-7, Director, Scripting

Video Description Video => Analysis -> Description -> Applications Scheme of audiovisual applications • Dublin core: the semantic indexing schema for video content description. • MPEG-7: the future standard tools will enable to define the semantic schemas for description of the audiovisual information. *Our video modeling for composing multimedia document.

Methodology • Specification of a modeling for the description of video content: – Multi-level structuration, – temporal and spatial relations, – actions interactive on the video elements. • Specification in XML • Experimentation in Video

Multi-level Structuration Raw video Video <!--XML schema for the description of Video. Content--> Occ. 1 Occ. 2 Occ. 3 Occ. 4 Structure • Video Content <!ELEMENT Video. Content – Structural Description (Meta. Info, Media. Info, Summary, Structure, Semantic, Thesaurus)> Irene Video Content Description Semantic Nabil Semantic Researcher Thesaurus – Semantic Description <!ELEMENT Structure (Sequence+, Relation? )> – Thesaurus <!ELEMENT Semantic (Video. Object*, Event. Semantics*)> <!ELEMENT Thesaurus (Reference. Dictionary*, User. Dictionary*)>

Video Structure Description • Motivation: for composition, the basis is to have the Structure description level. • Semantic and Thesaurus are more necessary for retrieval applications or as a support for structuration level. *First step is Structure description

High Level Description Video Structure • Video structure Structure --> – Shots <!--XML schema for the description high level Sequences <!ELEMENT Structure (Sequence+, Relation? )> <!ELEMENT Sequence (Scene+, Relation? )> – Scenes – Sequences (Shot+, Event*, Relation? )> Scenes <!ELEMENT Scene <!ELEMENT Shots Video (Transition? , Event*, Occurrence*, Background? , Relation? )>

Shot Content Description • Shot Content <!-- XML Shot Description --> Semantic – Transition (Transition? , Event*, Occurrence*, <!ELEMENT Shot – Event Index Background? , Relation? ) > <!ELEMENT Transition EMPTY > – Spatial. Layout EMPTY> <!ELEMENT Event Trans. Event Spatial. Layout Camera. Work – Occurrence Background – Background <!ELEMENT Background (Region+)> * Reference <!ELEMENT Occurrence (Region+, Trajectory? , Occurrence*) > <!ELEMENT Spatial. Layout (2 DBString. DS+) > – Camerawork <!ELEMENT Camera. Work (Camera. Motion? ) >

Occurrence Content Description • Occurrence Content <!ELEMENT Occurrence (Trajectory*, Region+, – Trajectory <!-- XML Occurrence description --> Occurrence*)> – Regions Trajectory Region Occurrence • Contour <!ELEMENT Region (Contour+, Color*, Texture*, • Color Centroid, Region*)> Contour Color <!ELEMENT Trajectory …> <!ELEMENT Contour … > Texture Centroid Region • Texture <!ELEMENT Color … > <!ELEMENT Texture … > • Regions <!ELEMENT Centroid … > – Occurrences <!ELEMENT Region … >

Experimentation of the model in Madeus - Video. Madeus

Madeus Architecture Editor/Presentation Tools EXECUTION View TIME LINE View HIERARCHICAL View VIDEO STRUCTURED View . . . MODEL MANAGEMENT PARSERS LOGIC STRUCTURATION TEMPORAL SAVE Madeus document STRUCTURATION EVENT MANAGEMENT SPATIAL MADEUS STRUCTURATION OUTILS JAVA Xerces JMF • To extend Madeus to Video. Madeus, video content description is handled both in composition and in presentation parts.

Madeus Document Model <Madeus> • Madeus Document Structured document organized according to the <Content> … </Content> Content dimensions: Logical, temporal, spatial. Actor Temporal Madeus Document Logical Spatial Content <Actor>. . . </Actor> Internal Document <Temporal>. . . </Temporal> <Spatial>. . . </Spatial> </Madeus> • Actor Content that describes the content information of the Temporal document • Spatial Actor that defines how this basic information in the content part is used in the document (style information, link, etc. )

Relations • Temporal relations (Allen extension) – meets, starts, equals, during, overlaps, parmin, etc. • Spatial relations – left_align, right_align, center_v, center_h, <Temporal> … <Relations> top_align, bottom_align, etc. <start Interval 1= « a » Interval 2= « b » /> <meet Interval 1= « b » Interval 2= « d » /> … <Relations> d </Temporal> <Spatial> … <Relations> <left-align Region 1= « b » Region 2= « d » /> … <Relations> </Spatial>

Overview of Video. Madeus Editing and Presentation Tools Execution View Video edition View " Element Management Structure View " Semantic View " Thesaurus View Synchronization Management Behavior Management Hyperlink "Follow-up "Erase "Display, etc. . . " Edit "Play "Search Temporal " " Spatial " Requested descriptions Modified description Data Management XML Description of video content Modify Parser Index on video Video Internal Structure (MODEL)

Video. Madeus document <Madeus> <Content> . . . <Video. Content. DS> . . . <Scene ID = « My. Scene » . . . >. . . </Scene> </Video. Content. DS> </Content> <Actor>. . . <Video. Element ID= «Scene. Video» Content = «My. Scene » . . . >. . . </Video. Element> </Actor> <Temporal> <Interval ID=“Scence. Int” Actor=“Scene. Video” Duration=“. . . ” … /> <Relations>. . . </Relations> </Temporal> <Spatial> <Region ID=“Scence. Reg” Actor=“Scene. Video” Height =“ 288” Width=“ 352” … /> <Relations>. . . </Relations> </Spatial> </Madeus>