An Assessment of 3 DTV Technologies Levent Onural

An Assessment of 3 DTV Technologies Levent Onural Bilkent Un. Thomas Sikora Tech. Univ. Of Berlin Aljoscha Smolic Fraunhofer Inst. -HHI M. Reha Civanlar Koç Univ. Jörn Ostermann Univ. Of Hanover John Watson Univ. Of Aberdeen NAB-2006 - Las Vegas - 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 1

Presentation Overview • Introduction to EC funded 3 DTV project • Introduction and Historical Overview • Technical Overview of Integrated 3 DTV • Capture • Representation • Transmission • Signal Processing • Display • Applications • Conclusion 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 2

EC Funded 3 DTV Project • 19 Partners from seven European countries • Universities, Research Institutions, SMEs • About 200 contributors • All technical aspects of 3 DTV are covered • A four year project (2004 -2008) • “Network of Excellence” 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 3

EC Funded 3 DTV Project • Partners: • Bilkent Univ – Turkey (Coordinator) • BIAS – Germany • CLOSPI-BAS – Bulgaria • De Montfort Univ – UK • Fraunhofer Inst. HHI – Germany • Fog. Screen – Finland • Un. Of Ilmenau - Germany 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 4

EC Funded 3 DTV Project • Partners (Cont’d): • Tampere Un. Of Tech – Finland • ITI-CERTH – Greece • Koç Univ. – Turkey • METU – Turkey • Momentum – Turkey • Max Planck Inst. – Germany • U. West Bohemia Plzen – Czech Rep 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 5

EC Funded 3 DTV Project • Partners (Cont’d): • U. Of Hanover – Germany • Tech. Univ. Berlin – Germany • U. Of Tuebingen - Germany • U. Of Aberdeen - UK • Yogurt Tech. – Turkey 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 6

26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 7

3 DTV Technical Overview 3 D Scene Capture Transmission Representation Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 8

3 DTV Historical Overview • “Realistic” ghost-like illusions have always been a goal • 2 D (still or motion) picture technologies are well developed and well accepted • 3 D counterparts are almost as old as 2 D • Stereoscopic 3 D pictures – 1838 (Wheatsone) • Popular by 1844 in USA and Europe • 2 D Photography - 1839 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 9

3 DTV Historical Overview • 3 D counterparts are almost as old as 2 D (continued) • 2 D Movies – 1867 (Lincoln) • 3 D stereoscopic cinema – early 1900 • 2 D TV – 1920 (Belin and Baird) • Stereoscopic 3 DTV – 1920 (Baird) • Stereoscopic 3 D cinema was popular by 1950 s • Stereoscopic 3 DTV broadcast – 1953 • First commercial 3 DTV broadcast – 1980 s 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 10

Observations • Stereoscopic 3 DTV is quite an old concept • Almost as old as its 2 D counterpart • 2 D has flourished • 3 D “failed” to be popular • Stereoscopic 3 D is simpler • Based on human visual system and perception • Usually creates “mismatches” of various 3 D cues • Current research on stereoscopic 3 DTV is mostly targeted to overcome such problems 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 11

An Overview of Candidate 3 DTV Techniques • Stereoscopic 3 DTV • Simplest, based on “perception” • Viewer discomfort • Autostereoscopic • Head-tracking stereoscopic • Holographic 3 DTV • Basic principle -1948, first holograms – 1960 • Based on physics : duplication of light field • Digital holographic techniques • Holographic cameras • Experimental holographic motion pictures – 1989 • Still at basic reseach phase 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 12

An Overview of Candidate 3 DTV Techniques • Integral Imaging Based 3 DTV • A principle known since 1908 • Based on “capturing a scene from different angles” • And then reverse projection from corresponding angles • Lenslet arrays • Variants can be classified under “holographic techniques” 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 13

Past Decade • NHK – Stereoscopic 3 D-HDTV (Since 1991) • 2002 FIFA World Cup broadcast in Stereo 3 D-HDTV (ETRI-Korea) • European ATTEST project (2002 -2004) • Backward compatible to DVB • Uses “depth data” (efficiently compressed by H. 264/AVC) • Matches MPEG 4 with minor modifications • MERL projection-based system (2004) 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 14

3 DTV Project -Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 15

3 DTV Project - Technical Overview - Capture • • • Multi-camera Single Camera Holographic Human Face and Body Specific Pattern Projection Other 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 16

3 DTV Project -Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 17

3 DTV Project – Technical Overview - Scene Representation • Point Representation Based • Surface Representation Based • Volume Representation Based • Human Face and Body Specific Representations • Object Specific Representations • Cloth and Garment Representation • Rendering Issues • Formats and Standardization Issues • DXF, OBJ, VRML, LWO, 3 DS, 3 DIF, U 3 D, Web 3 D, X 3 D, MPEG-4 -AFX • Texture Representation and Rendering 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 18

3 DTV Project -Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 19

3 DTV Project – Technical Overview-3 D Coding • 3 D Coding Techniques: • Holographic Signal Compression • Multiview Video Coding • Object-based coding • 3 D Mesh compression • Dynamic Texture Coding • Multiple Description Schemes • Interactivity Issues • Lightfield Compression • Point-cloud Compression • MPEG 4 -AFX, BIFS • Interactivity Issues • Compound Bit-stream Syntax • Security and Rights Management 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 20

3 DTV – Standardization Activities • Current Standardization Activities in 3 DTV • MPEG 4 -3 DAV • Multiview Coding 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 21

3 DTV Project -Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 22

3 DTV Project–Technical Overview - Transport Techniques for 3 DTV – Media Transport over IP networks • Qo. S Controls – Protocols • • Media Synchronization RTP DCCP H. 264 over RTP – Loss Concealment Techniques – Channel Coding Issues – Mobile 3 DTV Applications 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 23

3 DTV Project Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 24

Review Mtg – Technical Overview - Signal Processing Issues • Modeling of Diffraction from a Systems Point of View • Forward and Inverse Problems in Holographic 3 DTV • Signal Processing Problems in Different 3 DTV Techniques – – – Holographic 3 DTV Stereoscopic TV Integral Imaging Hybrid Techniques SAR techniques 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 25

3 DTV Project -Technical Overview 3 D Scene Capture Representation Transmission Signal Conversion Coding Display Its Replica 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 26

3 DTV Project – Technical Overview – Display Techniques – Holographic Displays • SLMs • DMDs • Acousto-optic Modulators – Stereoscopic Displays – Autostereoscopic Displays – Volumetric – Pseudo-volumetric – Head-mount displays 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 27

3 DTV Review Mtg – Technical Overview - Display Techniques • Autostereoscopic Displays (cont’d): – Pseudo-volumetric Fog. Screen 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 28

3 DTV Review Mtg – Technical Overview - Display Techniques 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 29

26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 30

3 DTV Technologies – Other Applications • Education, Training and Simulation • Scientific Simulation and Visualization • Medicine and Dentistry • Cultural Heritage • Technical Applications • Market Applications • Telepresence • Entertainment and Game Industry • Air Traffic Control • Military • Security • Communications • Computers 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 31

26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 32

26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 33

26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 34

Thank you. . Levent Onural, Thomas Sikora, Jörn Ostermann, Aljoscha Smolic, M. Reha Civanlar and John Watson www. 3 dtv-research. org 26 April 2006 Levent Onural --- www. 3 dtv-research. org © Copyright 2006 35