9 Standardization MPEG7 Multimedia Content Description Interface n

9. Standardization: MPEG-7 “Multimedia Content Description Interface” n Standard for describing multimedia content (metadata). n Goal: Efficient searching, browsing and filtering of audiovisual material: still images, graphics, 3 D models, audio, speech, video, multimedia presentations. Feature extraction Standard description Search engine Scope of MPEG-7 n n n Feature extraction and the search engine are not included in the standard. Descriptions are based on XML. Overview: http: //www. chiariglione. org/mpeg/standards/mpeg-7. htm MMDB-9 J. Teuhola 2012 194

MPEG-7 concepts n n n Feature: Distinctive characteristic of a given MM object Descriptor: Defines the syntax and semantics of feature representation; instantiation = descriptor value Description scheme: Structure and semantics between components (which can be descriptors or description schemes) Description definition language (DDL): Allows the creation of new descriptors and description schemes Description instance: Description scheme + set of descriptor values that describe the data. Descriptions have coded representations. Description Definition Language Definition Descriptors structuring Definition Description schemes MMDB-9 Tags Descriptions instantiation J. Teuhola 2012 195

MPEG-7: example descriptors n Visual: ¨ Basic structures and layout (2 D, 3 D, time) ¨ Color (color space, dominant color, quantization, layout, …) ¨ Texture (edge histogram, homogenous texture, …) ¨ Shape (region-based shape, contour-based shape, …) ¨ Motion (camera motion, motion activity, motion trajectory, …) ¨ Localization (region locator, spatio-temporal locator) ¨ Face recognition n Audio: ¨ Basic (low-level) features of audio signals (spectrum etc. ) ¨ High-level description tools, e. g. sound recognition and indexing, instrumental timbre, spoken content, audio signature, melodic description MMDB-9 J. Teuhola 2012 196

MPEG-7: potential application areas n n n n n Digital libraries (retrieval from archives of text, images, speech); Education (finding teaching material, preparing virtual courses) Journalism (searching archives by voice, face, etc. ) Broadcast indusry (audiovisual archives) Entertainment business (video-on-demand, games) Culture (contents of museums, art galleries); Police investigations (surveillance, face recognition) Geographical information systems (spatial databases, cartography, natural resources management) Medicine (patient information; telemedicine) MMDB-9 J. Teuhola 2012 197