Mpeg4 Overview Gerhard Roth 1 Overview Much more

Mpeg-4 Overview Gerhard Roth 1

Overview • Much more general than all previous mpegs – standard finished in the last two years • standardized ways to support: – coding • representing units of audio, visual, audiovisual content – composition • describing how these objects are composed together to produce compound media objects – multiplex • multiplex and synchronize data associated with media objects – interaction • interacting with user 2

System overview • a number of decoders for each type of media • a composition process which puts the decoded parts together • composition ideas are similar to VRML – – scene graph of objects composition information is defined in the links hierarchical structure, not necessarily static composition stream has extra information for mp 4 • spatial relationships • temporal relationships 3

Receiver/Decoder architecture 4

Multiplex synchronization • timing and synchronization of media • three layers – specific information for timing and synchronization of the coded media (synchronization layer) – multiplex streams with different characteristics (flexible multiplex layer) – add the multiplex stream to a particular network (transport multiplex layer) 5

Multiplex/synchronization layer 6

Intellectual property protection • ipmp = intellectual property management and protection • interface that permits mp 4 to host one or more ipmp systems • ipmp framework has – normative interface that permits mpeg-4 terminal to host one or more IPMP systems – non normative component that provides intellectual property management 7

Video • Tools for efficient compression of • • • images, video, 2 d textures time varying geometry (animated meshes) efficient random access extended manipulation functionality content based coding and scalability error robustness and resilience • based on video object – entity in scene that user can manipulate and access • video object planes – instances of video objects at a given time 8

Video 9

Media integration of text and graphics • 3 d graphics – like VRML including script nodes • face animation – face definition parameters (FDPs) – face animation parameters (FAPs) • 2 d mesh encoding – dynamic mesh, a temporal sequence of 2 d triangular meshes • texture coding via wavelets 10

Face definition parameters • a number of feature points on a neutral face 11

Face animation parameters • a number of face • animation points 12

2 d mesh decoding • mesh vertices points can move in 2 d 13

2 d mesh of fish 14

2 d mesh augmentation of fish – the letters are synthetically overlaid on the video object and move in synchronization with it 15

Conclusion • a superset of old VRML and Mpeg-1 and Mpeg-2 • very general, many concepts – can mix and match many different compression algorithms for different objects – handles everything (audio, video, 2 d-3 d, faces, mesh…) – is a first attempt at integrating 2 d and 3 d worlds • Div. X is a software application that uses the MPEG -4 standard to compress digital video • Div. X is done with the open source community – much better compression than ordinary DVDs 16

URLs 17