Systems Support for Continuous Media OOI WEI TSANG

Systems Support for Continuous Media OOI WEI TSANG NUS. SOC. CS 5248 Ooi Wei Tsang

Scope of this class n Systems Support: n Application n Middleware n Operating System n Continuous Media: n Video, audio, animation NUS. SOC. CS 5248 OOI Wei Tsang

Local Media Applications n Watch Movie n Listen to Music n Video Editing : (Much Less Interesting …) NUS. SOC. CS 5248 OOI Wei Tsang

Network Media Applications n Live Webcast n Pre-recorded Webcast n Video Conferencing n Video on Demand NUS. SOC. CS 5248 OOI Wei Tsang

Categories Live Prerecorded NUS. SOC. CS 5248 OOI Wei Tsang Interactive Non. Interactive Video Conferencing NUSLive, NUSCast Lecture/Video on Demand

Streaming Media Requires n Reliable networks n Low latency n Bounded latency n Plenty of bandwidth Not in today’s Internet. NUS. SOC. CS 5248 OOI Wei Tsang

Overview Encoder Decoder Middlebox Receiver Sender Network NUS. SOC. CS 5248 OOI Wei Tsang

We will NOT n Design new codec n Study media processing n CS 6212/CS 5240 n Attempt to improve the Internet n EE 5910 NUS. SOC. CS 5248 OOI Wei Tsang

Summary of Topics Covered NUS. SOC. CS 5248 Ooi Wei Tsang

Sample of Topics Covered Encoder Decoder Middlebox Receiver Sender Network n Review of Media Compression n Network Communication Model NUS. SOC. CS 5248 OOI Wei Tsang

Sample of Topics Covered Encoder Decoder Middlebox Receiver Sender Network n Adaptation n Video on Demand NUS. SOC. CS 5248 OOI Wei Tsang

Sample of Topics Covered Encoder Decoder Middlebox Receiver Sender Network n Buffering n Audio/Video Synchronization n Packet Loss Recovery NUS. SOC. CS 5248 OOI Wei Tsang

Sample of Topics Covered Encoder Decoder Middlebox Receiver Sender Network n Caching n Application-Level Multicast NUS. SOC. CS 5248 OOI Wei Tsang

Lecture Philosophy n Cover the breadth, rather than depth n Classic papers, rather than recent papers NUS. SOC. CS 5248 OOI Wei Tsang

Class Organization NUS. SOC. CS 5248 Ooi Wei Tsang

Goals n Learn about existing techniques and systems n Learn about current research efforts n Learn about how to do research NUS. SOC. CS 5248 OOI Wei Tsang

Philosophy n Research oriented n Students are expected to be mature and independent n What you learned is more important than grade NUS. SOC. CS 5248 OOI Wei Tsang

Please don’t ask. . n “Is this equation important? ” n “Is this equation examinable? ” n “Do I have to memorize this header format? ” n “Post answer to quiz/survey/review” NUS. SOC. CS 5248 OOI Wei Tsang

Please do ask. . n “What is the effect of changing k in the equation? ” n “Why did the designer added this bit in the header? ” NUS. SOC. CS 5248 OOI Wei Tsang

Academic Honesty n No copying among students n No copying from published work ZERO TOLERENCE to Plagiarism NUS. SOC. CS 5248 OOI Wei Tsang

Discussion? n Strongly encouraged. . n but n must acknowledge all contributions n write up solutions independently NUS. SOC. CS 5248 OOI Wei Tsang

Gilligan’s Island Rule n Discuss with friends n Throw notes away n Go watch “Gilligan’s Island” (or “Phua Chu Kang”, or “Super Sunday”) n Write down your solution NUS. SOC. CS 5248 OOI Wei Tsang

Continuous Assessments NUS. SOC. CS 5248 Ooi Wei Tsang

Continuous Assessments n Project (50%) n Paper Survey (20%) n Paper Review (15%) n One of n Programming Assignment n Random Quizzes n Class Participation n Mid-term NUS. SOC. CS 5248 OOI Wei Tsang

Research Project n Define a problem n Propose a solution n Implement the solution n Evaluate the solution NUS. SOC. CS 5248 OOI Wei Tsang

Types of Project n New problem + solution n Existing problem + new solution n Existing problem + existing solutions + comparisons + new insights n See website for more. . NUS. SOC. CS 5248 OOI Wei Tsang

Variations n Combine with your own research n subjected to your advisor’s approval n Combine with another class project n subjected to the other instructor’s approval n must be more significant NUS. SOC. CS 5248 OOI Wei Tsang

Research Project n Deliverables: n 10 -12 page paper n poster presentation n demo (if applicable) n In a group of 1 to n students NUS. SOC. CS 5248 OOI Wei Tsang

Survey Paper n In-depth study of a topic n 4 -10 papers expected n Suggested list of topics available (can be the same topic as your project!) NUS. SOC. CS 5248 OOI Wei Tsang

Survey Paper n summarize recent results n organize, generalize, categorize n add understanding NUS. SOC. CS 5248 OOI Wei Tsang

Survey Paper n Deliverables: n 10 -12 page survey paper n Individual work NUS. SOC. CS 5248 OOI Wei Tsang

Paper Reviews n Read, summarize, and criticize 10 papers n One is assigned almost weekly n Deliverables: n One page report per paper NUS. SOC. CS 5248 OOI Wei Tsang

Expected Workload n Read 3 -4 papers per week n Some hours of coding per week NUS. SOC. CS 5248 OOI Wei Tsang

Office Hours n Saturday n 11 am – 12 noon n SOC 1 04 -20 n or email for appointment n or just pop in NUS. SOC. CS 5248 OOI Wei Tsang

Web Sites n Main Website n http: //www. comp. nus. edu. sg/~cs 5248 n IVLE Website NUS. SOC. CS 5248 OOI Wei Tsang

Questions NUS. SOC. CS 5248 Ooi Wei Tsang

Review of Media Compression NUS. SOC. CS 5248 Ooi Wei Tsang

You are Here Encoder Decoder Middlebox Receiver Sender Network NUS. SOC. CS 5248 OOI Wei Tsang

Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = NUS. SOC. CS 5248 OOI Wei Tsang

Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208. 6 GB NUS. SOC. CS 5248 OOI Wei Tsang

JPEG Compression NUS. SOC. CS 5248 Ooi Wei Tsang

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 1: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 3. 5: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 17: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 27: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 72: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang 192: 1

Compression Ratio Quality Raw TIFF Size 1153 KB Ratio 1: 1 Zipped TIFF 982 KB 1. 2: 1 Q=100 331 KB 3. 5: 1 Q=70 67 KB 17: 1 Q=40 43 KB 27: 1 Q=10 16 KB 72: 1 Q=1 6 KB 192: 1 NUS. SOC. CS 5248 OOI Wei Tsang

Magic of JPEG n Throw away information we cannot see n Color information n “Low-frequency signals” n Rearrange data for good compression n Use standard compression NUS. SOC. CS 5248 OOI Wei Tsang

Throw away color information Y V NUS. SOC. CS 5248 OOI Wei Tsang U

Color Sub-sampling NUS. SOC. CS 5248 OOI Wei Tsang 4: 4: 4 4: 2: 0 4: 2: 2 4: 1: 1

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang

Original Image (1153 KB) NUS. SOC. CS 5248 OOI Wei Tsang

NUS. SOC. CS 5248 OOI Wei Tsang

Discrete Cosine Transform n DCT demo applet and source code is available on the web. NUS. SOC. CS 5248 OOI Wei Tsang

Quantization 5 8 8 8 -54 -10 -4 -2 8 8 8 16 32 242 65 23 13 6 3 2 1 -1 -2 NUS. SOC. CS 5248 OOI Wei Tsang 5 / 8 8 16 32 64 30 8 = 2 0 -6 -1 0 0 0 0 0

Differential Coding 30 8 2 0 25 3 1 0 27 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 30 8 2 0 -5 3 1 0 2 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 NUS. SOC. CS 5248 OOI Wei Tsang

Zig-zag ordering 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 NUS. SOC. CS 5248 OOI Wei Tsang

Run-Length Encoding 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1 NUS. SOC. CS 5248 OOI Wei Tsang

Idea: Motion JPEG n Compress every frame in a video as JPEG n DVD-quality video = 208. 6 GB n Reduction ratio = 27: 1 n Final size = 7. 7 GB NUS. SOC. CS 5248 OOI Wei Tsang

Video Compression NUS. SOC. CS 5248 Ooi Wei Tsang

NUS. SOC. CS 5248 OOI Wei Tsang

Temporal Redundancy NUS. SOC. CS 5248 OOI Wei Tsang

Motion Estimation NUS. SOC. CS 5248 OOI Wei Tsang

Bi-directional Prediction NUS. SOC. CS 5248 OOI Wei Tsang

H. 261 P-Frame I-Frame NUS. SOC. CS 5248 OOI Wei Tsang

MPEG-1 B-Frame NUS. SOC. CS 5248 OOI Wei Tsang

MPEG Frame Pattern n Example display sequence: n IBBPBBP … n Example encoding sequence: n IPBBPBB NUS. SOC. CS 5248 OOI Wei Tsang

Compression Ratio Frame Typical Ratio I 10: 1 P 20: 1 B 50: 1 NUS. SOC. CS 5248 OOI Wei Tsang

Sequence sequence header: • width • height • frame rate • bit rate • : NUS. SOC. CS 5248 OOI Wei Tsang

GOP: Group of Picture gop header: • time • : NUS. SOC. CS 5248 OOI Wei Tsang

Picture pic header: • number • type (I, P, B) • : NUS. SOC. CS 5248 OOI Wei Tsang

Picture NUS. SOC. CS 5248 OOI Wei Tsang

Slice NUS. SOC. CS 5248 OOI Wei Tsang

Macroblock NUS. SOC. CS 5248 OOI Wei Tsang

Block 1 Macroblock = NUS. SOC. CS 5248 OOI Wei Tsang Y Y U Y Y V

MPEG in daily life MPEG Standards Bit-rate Usage MPEG-1 1. 5 Mbps VCD 10 -40 Mbps DVD, SVCD, HDTV Scalable Quick. Time, Div. X; ) MPEG-2 MPEG-4 NUS. SOC. CS 5248 OOI Wei Tsang

Impact on Systems Design n How to package data into packets? n How to deal with packet loss? n How to deal with bursty traffic? n How to predict decoding time? : : NUS. SOC. CS 5248 OOI Wei Tsang