CS 414 Multimedia Systems Design Lecture 6 Digital

CS 414 – Multimedia Systems Design Lecture 6 – Digital Video and Introduction to Compression Klara Nahrstedt Spring 2014 CS 414 - Spring 2014

Administrative MP 1 is posted n See Class website and compass n MP 1 lecture will be on February 7 (Friday) in class. Please, read the MP 1 before attending the class n CS 414 - Spring 2014

Today Introduced Concepts Analog and Digital Television n Need for compression and compression algorithms classification n Basic Coding Concepts n ¨ Fixed-length coding and variable-length coding ¨ Compression Ratio ¨ Entropy CS 414 - Spring 2014

Television History (Analog) n 1927, Hoover made a speech in Washington while viewers in NY could see, hear him n AT&T Bell Labs had the first “television” ¨ 18 fps, 2 x 3 inch screen, 2500 pixels

Analog Television Concepts n n n Production (capture) ¨ 2 D ¨ structured formats Representation and Transmission ¨ popular formats include NTSC, PAL, SECAM Re-construction ¨ scanning ¨ display issues (refresh rates, temporal resolution) ¨ relies on principles of human visual system CS 414 - Spring 2014

Color Space: YUV n PAL video standard ¨Y is luminance ¨ UV are chrominance n Y YUV from RGB Y =. 299 R +. 587 G +. 114 B U = 0. 492 (B - Y) V = 0. 877 (R - Y) U-V plane at Y=0. 5 U V CS 414 - Spring 2014 Source: wikipedia

YIQ (NTSC) n YIQ from RGB Y =. 299 R +. 587 G +. 114 B I =. 74 (R - Y) -. 27 (B - Y) Q = 0. 48 (R - Y) + 0. 41 (B Y) CS 414 - Spring 2014 YIQ with Y=0. 5 Source: wikipedia

Video Representations CS 414 - Spring 2014

TV History CS 414 - Spring 2014

HDTV (Digital) n Resolutions: ¨ 1920 x 1080 (1080 p) – Standard HD (HDTV) ¨ 2160 p, … ¨ 4096 x 2304 (4096 p) – 4 K High HD n Frame rate: ¨ HDTV - 50 or 60 frames per second ¨ HDTV – 120 fps CS 414 - Spring 2014

HDTV n Interlaced (i) and/or progressive (p) formats ¨ Conventional TVs – use interlaced formats ¨ Computer displays (LCDs) – use progressive scanning MPEG-2 compressed streams n In Europe (Germany) – MPEG-4 compressed streams n CS 414 - Spring 2014

Aspect Ratio and Refresh Rate n Aspect ratio Conventional TV is 4: 3 (1. 33) ¨ HDTV is 16: 9 (2. 11) ¨ Cinema uses 1. 85: 1 or 2. 35: 1 ¨ n Frame Rate ¨ NTSC is 60 Hz interlaced (actually 59. 94 Hz) ¨ PAL/SECAM is 50 Hz interlaced ¨ Cinema is 24 Hz noninterlaced CS 414 - Spring 2014 Source: wikipedia

Digital Video and TV n n Bit rate: amount of information stored per unit time (second) of a recording Color Coding: YCr. Cb of YUV that scales and shifts the chrominance values into range 0. . 1 Y ¨ Subset Y = 0. 299 R + 0. 587 G +. 114 B Cr = ((B-Y)/2) + 0. 5 Cb = ((R-Y)/1. 6) + 0. 5 CS 414 - Spring 2014 Cr Cb

Digital Video and TV n Color space compression ¨ YUV 444 n 24 bits per pixel ¨ YUV 422 n 16 bits/pixel ¨ YUV 411 n 12 bits/pixel CS 414 - Spring 2014

Digital Video and TV n DVD video ¨ Since 1997 ¨ Resolution and frame rate 704 x 480 at 29. 97 fps n 704 x 576 at 25 fps n ¨ Bitrate: 9. 8 Mbps CS 414 - Spring 2014

Digital Video and TV n Blu-ray video since 2006 ¨ Resolution and frame rate ¨ 1920 i (@59. 94 fps) – interlaced n 1920 p (@24 fps) – progressive n …. n ¨ Bitrate : 40 Mbps CS 414 - Spring 2014

3 DTV Refresh rate no less than 120 Hz n Synchronized shutter glasses to enable different views for different eyes n CS 414 - Spring 2014

Today Introduced Concepts Analog and Digital Television n Need for compression and compression algorithms classification n Basic Coding Concepts n ¨ Fixed-length coding and variable-length coding ¨ Compression Ratio ¨ Entropy CS 414 - Spring 2014

Reading n Media Coding and Content Processing, Steinmetz, Nahrstedt, Prentice Hall, 2002 ¨ Data n Compression – chapter 7 Basic coding concepts – Sections 7. 1 -7. 4 and lecture notes CS 414 - Spring 2014

Integrating Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2014 A/V Playback

Need for Compression n Uncompressed audio 8 KHz, 8 bit n 8 K per second ¨ 30 M per hour ¨ n 7. 37 Mbytes per second ¨ 26. 5 Gbytes per hour ¨ 44. 1 KHz, 16 bit n 88. 2 K per second ¨ 317. 5 M per hour ¨ n Uncompressed video 640 x 480 resolution, 8 bit color, 24 fps 640 x 480 resolution, 24 bit (3 bytes) color, 30 fps 27. 6 Mbytes per second ¨ 99. 5 Gbytes per hour ¨ 100 Gbyte disk holds 315 hours of CD quality music n 100 Gbyte disk holds 1 hour of high quality video CS 414 - Spring 2014

Broad Classification n Entropy Coding (statistical) ¨ lossless; independent of data characteristics ¨ e. g. RLE, Huffman, LZW, Arithmetic coding n Source Coding ¨ lossy; may consider semantics of the data ¨ depends on characteristics of the data ¨ e. g. DCT, DPCM, ADPCM, color model transform n Hybrid Coding (used by most multimedia systems) ¨ combine entropy with source encoding ¨ e. g. , JPEG-2000, H. 264, MPEG-2, MPEG-4, MPEG-7 CS 414 - Spring 2014

Data Compression n Branch of information theory ¨ minimize amount of information to be transmitted n Transform a sequence of characters into a new string of bits ¨ same information content ¨ length as short as possible CS 414 - Spring 2014

Concepts n Coding (the code) maps source messages from alphabet (A) into code words (B) n Source message (symbol) is basic unit into which a string is partitioned ¨ can n be a single letter or a string of letters EXAMPLE: aa bbb cccc ddddd eeeeee fffffffgggg ¨ A = {a, b, c, d, e, f, g, space} ¨B = {0, 1} CS 414 - Spring 2014

Today Introduced Concepts Analog and Digital Television n Need for compression and compression algorithms classification n Basic Coding Concepts n ¨ Fixed-length coding and variable-length coding ¨ Compression Ratio ¨ Entropy CS 414 - Spring 2014

Taxonomy of Codes n Block-block ¨ source msgs and code words of fixed length; e. g. , ASCII n Block-variable ¨ source message fixed, code words variable; e. g. , Huffman coding n Variable-block ¨ source n variable, code word fixed; e. g. , RLE Variable-variable ¨ source variable, code words variable; e. g. , Arithmetic CS 414 - Spring 2014

Example of Block-Block n n Coding “aa bbb cccc ddddd eeeeee fffffffgggg” Requires 120 bits Symbol Code word a 000 b 001 c 010 d 011 e 100 f 101 g 110 space 111

Example of Variable-Variable n n Coding “aa bbb cccc ddddd eeeeee fffffffgggg” Requires 30 bits ¨ don’t forget the spaces Symbol Code word aa 0 bbb 1 cccc 10 ddddd 11 eeeeee 100 fffffff 101 gggg 110 space 111

Concepts (cont. ) n A code is ¨ distinct if each code word can be distinguished from every other (mapping is one-to-one) ¨ uniquely decodable if every code word is identifiable when immersed in a sequence of code words n e. g. , with previous table, message 11 could be defined as either ddddd or bbbbbb CS 414 - Spring 2014

Static Codes n Mapping is fixed before transmission ¨ message represented by same codeword every time it appears in message (ensemble) ¨ Huffman coding is an example n Better for independent sequences ¨ probabilities of symbol occurrences must be known in advance; CS 414 - Spring 2014

Dynamic Codes n Mapping changes over time ¨ also n referred to as adaptive coding Attempts to exploit locality of reference ¨ periodic, frequent occurrences of messages ¨ dynamic Huffman is an example n Hybrids? ¨ build set of codes, select based on input CS 414 - Spring 2014

Traditional Evaluation Criteria n Algorithm complexity ¨ running n time Amount of compression ¨ redundancy ¨ compression n ratio How to measure? CS 414 - Spring 2014

Measure of Information Consider symbols si and the probability of occurrence of each symbol p(si) n In case of fixed-length coding , smallest number of bits per symbol needed is n L ≥ log 2(N) bits per symbol ¨ Example: Message with 5 symbols need 3 bits (L ≥ log 25) ¨ CS 414 - Spring 2014

Variable-Length Coding. Entropy What is the minimum number of bits per symbol? n Answer: Shannon’s result – theoretical minimum average number of bits per code word is known as Entropy (H) n CS 414 - Spring 2014

Entropy Example n Alphabet = {A, B} ¨ p(A) n = 0. 4; p(B) = 0. 6 Compute Entropy (H) ¨ -0. 4*log 2 0. 4 + -0. 6*log 2 0. 6 =. 97 bits CS 414 - Spring 2014

Summary n Symmetric compression ¨ requires same time for encoding and decoding ¨ used for live mode applications (teleconference) n Asymmetric compression ¨ performed once when enough time is available ¨ decompression performed frequently, must be fast ¨ used for retrieval mode applications (e. g. , an interactive CD-ROM) CS 414 - Spring 2014