Video and Image Processing At Purdue Edward J

Video and Image Processing At Purdue Edward J. Delp Video and Image Processing Laboratory (VIPER) School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana, USA email: ace@ecn. purdue. edu http: //www. ece. purdue. edu/~ace Edward J. Delp Intel December 3, 1999 Slide 1

Acknowledgements • Students – Eduardo Asbun – Dan Hintz – Paul Salama – Ke Shen – Martha Saenz – Eugene Lin – Ray Wolfgang – Greg Cook – Sheng Liu Edward J. Delp Intel December 3, 1999 Slide 2

Intel T 4 E Project • Purdue awarded $6. 2 million in August 1997 for equipment – this is one of many strong relationships between Intel and Purdue • Has had a very significant impact on how we do research! THANKS! http: //www. cs. purdue. edu/homes/jtk/intel/ Edward J. Delp Intel December 3, 1999 Slide 3

Image and Video Processing at Purdue has a rich history 60 year history in image and video processing. Edward J. Delp Intel December 3, 1999 Slide 4

VIPER Research Projects • Scalable Video and Color Image Compression – still image compression (CEZW) – high and low bit rate video compression (SAMCo. W) – wireless video • Error Concealment • Content Addressable Video Databases (Vi. BE) – Scene Change Detection and Identification – Pseudo-Semantic Scene Labeling • Multimedia Security: Digital Watermarking Edward J. Delp Intel December 3, 1999 Slide 5

VIPER Research Projects • Multicast Video • Analysis of Mammograms • Embedded Image and Video Processing Edward J. Delp Intel December 3, 1999 Slide 6

Other Purdue Projects • Electronic Imaging - Jan Allebach and Charles Bouman – half-tone printing – compound document compression – image databases • Remote Sensing - David Langrebe • Medical Imaging - Charles Bouman, Peter Doerschuk, Thomas Talavage, Edward Delp – computed imagng – functional MRI – x-ray crystallography – breast imaging Edward J. Delp Intel December 3, 1999 Slide 7

Analysis of Mammograms Density 1 Edward J. Delp Density 2 Intel Density 3 December 3, 1999 Density 4 Slide 8

Detection Results A 12. 4 mm lesion detected at the second coarsest resolution Automatic Detection Edward J. Delp Intel Ground Truth December 3, 1999 Slide 9

Detection Results A 6. 6 mm lesion detected at the finest resolution Automatic Detection Edward J. Delp Intel Ground Truth December 3, 1999 Slide 10

Vi. BE: A New Paradigm for Video Database Browsing and Search • Vi. BE has four components – scene change detection and identification – hierarchical shot representation – pseudo-semantic shot labeling – active browsing based on relevance feedback • Vi. BE provides an extensible framework that will scale as the video data grows in size and applications increase in complexity Edward J. Delp Intel December 3, 1999 Slide 11

Video Analysis: Overview Proc. Closed-caption information Compressed video sequence Audio data Image data Data Extraction (DC frames) Proc. MPEG-related data (MVs, AC coeffs, etc. ) Shot Labeling Shot attributes Captions Edward J. Delp Proc. Intel Shot trees Intrashot Clustering December 3, 1999 Shot Transition Detection and Identification Transition locations and types Slide 12

Navigation via the Similarity Pyramid Zoom in Zoom out Edward J. Delp Intel December 3, 1999 Slide 13

Browser Interface Similarity Pyramid Edward J. Delp Intel Control Panel Relevance Set December 3, 1999 Slide 14

Video Over IP: Unicast Edward J. Delp Intel December 3, 1999 Slide 15

Video Over IP: Multicast Edward J. Delp Intel December 3, 1999 Slide 16

Video Over IP • • Currently multicasting 3 streams Multicast experiments with Europe Multicast HDTV over Internet 2 Issues: – what is the backward information? – which video compression technique? – how is network control connected to the server/encoder? Edward J. Delp Intel December 3, 1999 Slide 17

Why is Digital Watermarking Important? • Scenario – an owner places digital images on a network server and wants to detect the redistribution of altered versions • Goals – verify the owner of a digital image – detect forgeries of an original image – identify illegal copies of the image – prevent unauthorized distribution Edward J. Delp Intel December 3, 1999 Slide 18

Why is Watermarking Important? Edward J. Delp Intel December 3, 1999 Slide 19

Why is Watermarking Important? Edward J. Delp Intel December 3, 1999 Slide 20

Why Watermarking is Important? Edward J. Delp Intel December 3, 1999 Slide 21

Why is Watermarking Important? Edward J. Delp Intel December 3, 1999 Slide 22

VW 2 D Watermarked Image Edward J. Delp Intel December 3, 1999 Slide 23

Image Adaptive Watermarks (DCT) Edward J. Delp Intel December 3, 1999 Slide 24

Scalable Image and Video Compression • Problem: desire to have a compression technique that allows decompression to be linked to the application – databases, wireless transmission, Internet imaging – will support both high and low data rate modes • Other desired properties: – error concealment – will support the protection of intellectual property rights (watermarking) Edward J. Delp Intel December 3, 1999 Slide 25

Rate Scalable Image and Video Coding • Applications – Internet streaming – Image and video database search - browsing – Video servers – Teleconferencing and Telemedicine – Wireless Networks Edward J. Delp Intel December 3, 1999 Slide 26

Scalability • Picture Coding Symposium(April 1999) - panel on “The Future of Video Compression, ” importance of scalability: – rate scalability (Internet and wireless) – temporal scalability (Internet and wireless) – spatial scalability (databases - MPEG-7) – content scalability (MPEG-4) (Computational Scalability - implementation issues) Edward J. Delp Intel December 3, 1999 Slide 27

Scalability “Author and Compress once - decompress on any platform feed by any data pipe” Edward J. Delp Intel December 3, 1999 Slide 28

Scalability: Compression Standards • Scalability in JPEG – progressive mode – JPEG 2000 • Scalability in MPEG-2 – scalability is layered • Scalability in MPEG-4 – layered – “content” – fine grain scalability (fgs) Edward J. Delp Intel December 3, 1999 Slide 29

Color Embedded Zero-Tree Wavelet (CEZW) • Developed new technique known as Color Embedded Zero -Tree Wavelet (CEZW) • Modified EZW with trees connecting all color components – can be extended to other color spaces Edward J. Delp Intel December 3, 1999 Slide 30

Spatial Orientation Trees EZW Edward J. Delp SPIHT Intel December 3, 1999 Slide 31

New Spatial Orientation Tree (CEZW) Edward J. Delp Intel December 3, 1999 Slide 32

Color Image Compression Original CEZW JPEG Edward J. Delp SPIHT Intel December 3, 1999 Slide 33

Coding Artifacts CEZW Original JPEG SPIHT Edward J. Delp Intel December 3, 1999 Slide 34

Comparison JPEG 0. 25 bits/pixel Edward J. Delp Intel CEZW 0. 25 bits/pixel December 3, 1999 Slide 35

Color Compression - Experiments • Objectives: – Evaluate scalable color image compression techniques – Color Transformations – Spatial Orientation Trees and Coding Schemes – Embedded Coding • Embedded Zerotree Wavelet: Shapiro (Dec’ 93) • Set Partitioning in Hierarchical Trees: Said & Pearlman (Jun’ 96) • Color Embedded Zerotree Wavelets: Shen & Delp (Oct ‘ 97) M. Saenz, P. Salama, K. Shen and E. J. Delp, "An Evaluation of Color Embedded Wavelet Image Compression Techniques, " VCIP 1999 Edward J. Delp Intel December 3, 1999 Slide 36

Metrics Edward J. Delp Intel December 3, 1999 Slide 37

SAMCo. W • New scalable video compression technique - Scalable Adaptive Motion COompensated Wavelet compression • Features of SAMCo. W: – use wavelets on entire frame and for prediction error frames – uses adaptive motion compensation to reduce error propagation – CEZW is used for the wavelet coder on both the intracoded frames and prediction error frames Edward J. Delp Intel December 3, 1999 Slide 38

Generalized Hybrid Codec Edward J. Delp Intel December 3, 1999 Slide 39

Adaptive Motion Compensation Edward J. Delp Intel December 3, 1999 Slide 40

SAMCo. W Enhancements • • B frames (ICIP 98) unrestricted motion vectors (ICIP 98) half-pixel motion searches (ICIP 98) detailed study of PEF (ICIP 99 and VLBW 99) – denoising techniques • bit allocation and rate control (ICIP 99) Edward J. Delp Intel December 3, 1999 Slide 41

Error Concealment • In data networks, channel errors or congestion cause cell or packet loss • When compressed video is transmitted, cell loss causes macroblocks and motion vectors to be removed from compressed data streams • Goal of error concealment: Exploit redundant information in a sequence to recover missing data Edward J. Delp Intel December 3, 1999 Slide 42

Error Concealment Original frame Edward J. Delp Intel Damaged frame December 3, 1999 Slide 43

Approaches for Error Concealment • Two approaches for error concealment: – Active concealment: Use of error control coding techniques and retransmission • unequal error protection – Passive concealment: The video stream is postprocessed to reconstruct missing data • Passive concealment is necessary: – where active concealment cannot be used due to compliance with video transmission standards – when active concealment fails Edward J. Delp Intel December 3, 1999 Slide 44

Error Concealment Edward J. Delp Intel December 3, 1999 Slide 45

Error Concealment Edward J. Delp Intel December 3, 1999 Slide 46

Future Work • • • Video Streaming (wired and wireless) Color Compression experiments (JPEG 2000) Video databases Vi. BE Video watermarking Internet 2 and multicasting scalable video Error concealment in embedded codecs Edward J. Delp Intel December 3, 1999 Slide 47

How I Spent My Summer Edward J. Delp Intel December 3, 1999 Slide 48