Media Compression NUS SOC CS 5248 Ooi Wei

Media Compression NUS. SOC. CS 5248 Ooi Wei Tsang

You are Here Encoder Decoder Middlebox Receiver Sender Network NUS. SOC. CS 5248 Ooi Wei Tsang

Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = NUS. SOC. CS 5248 Ooi Wei Tsang

Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208. 6 GB NUS. SOC. CS 5248 Ooi Wei Tsang

JPEG Compression NUS. SOC. CS 5248 Ooi Wei Tsang

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 1: 1

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 3. 5: 1

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 17: 1

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 27: 1

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 72: 1

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 192: 1

Compression Ratio Quality Size Ratio Raw TIFF 1153 KB 1: 1 Zipped TIFF 982 KB 1. 2: 1 Q=100 331 KB 3. 5: 1 Q=70 67 KB 17: 1 Q=40 43 KB 27: 1 Q=10 16 KB 72: 1 Q=1 6 KB 192: 1 NUS. SOC. CS 5248 Ooi Wei Tsang

Magic of JPEG n Throw away information we cannot see n Color information n “High frequency signals” n Rearrange data for good compression n Use standard compression NUS. SOC. CS 5248 Ooi Wei Tsang

Discard color information Y V NUS. SOC. CS 5248 Ooi Wei Tsang U

Color Sub-sampling NUS. SOC. CS 5248 Ooi Wei Tsang 4: 4: 4 4: 2: 0 4: 2: 2 4: 1: 1

4: 2: 2 Sub-sampling Y V NUS. SOC. CS 5248 Ooi Wei Tsang U

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang 4: 2: 0

Original Image (1153 KB) NUS. SOC. CS 5248 Ooi Wei Tsang “ 4: 1: 0”

NUS. SOC. CS 5248 Ooi Wei Tsang

Discrete Cosine Transform Demo NUS. SOC. CS 5248 Ooi Wei Tsang

Quantization DC 242 65 23 5 8 8 8 -54 -10 -4 -2 8 8 8 16 13 6 3 5 8 8 16 32 2 1 -1 -2 / 8 8 16 32 64 Quantization Table NUS. SOC. CS 5248 Ooi Wei Tsang 30 8 = 2 0 -6 -1 0 0 0 0 0 AC

Differential Coding 30 8 0 25 3 1 0 27 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 30 8 2 0 -5 3 1 0 2 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 NUS. SOC. CS 5248 Ooi Wei Tsang 2

Zig-zag ordering 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 NUS. SOC. CS 5248 Ooi Wei Tsang

Run-Length Encoding 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4 NUS. SOC. CS 5248 Ooi Wei Tsang

Idea: Motion JPEG n Compress every frame in a video as JPEG n DVD-quality video = 208. 6 GB n Reduction ratio = 27: 1 n Final size = 7. 7 GB NUS. SOC. CS 5248 Ooi Wei Tsang

Video Compression NUS. SOC. CS 5248 Ooi Wei Tsang

NUS. SOC. CS 5248 Ooi Wei Tsang

Temporal Redundancy NUS. SOC. CS 5248 Ooi Wei Tsang

Motion Estimation NUS. SOC. CS 5248 Ooi Wei Tsang

Bi-directional Prediction NUS. SOC. CS 5248 Ooi Wei Tsang

Watch for Motion Vectors Demo NUS. SOC. CS 5248 Ooi Wei Tsang

H. 261 P-Frame I-Frame NUS. SOC. CS 5248 Ooi Wei Tsang

MPEG-1 B-Frame NUS. SOC. CS 5248 Ooi Wei Tsang

MPEG Frame Pattern n Example display sequence: n IBBPBBP … n Example encoding sequence: n IPBBPBB NUS. SOC. CS 5248 Ooi Wei Tsang

Compression Ratio Frame Typical Ratio I 10: 1 P 20: 1 B 50: 1 NUS. SOC. CS 5248 Ooi Wei Tsang

Sequence sequence header: • width • height • frame rate • bit rate • : NUS. SOC. CS 5248 Ooi Wei Tsang

GOP: Group of Picture gop header: • time • : NUS. SOC. CS 5248 Ooi Wei Tsang

Picture pic header: • number • type (I, P, B) • : NUS. SOC. CS 5248 Ooi Wei Tsang

Picture NUS. SOC. CS 5248 Ooi Wei Tsang

Slice NUS. SOC. CS 5248 Ooi Wei Tsang

Macroblock NUS. SOC. CS 5248 Ooi Wei Tsang

Block 1 Macroblock = NUS. SOC. CS 5248 Ooi Wei Tsang Y Y U Y Y V

For I-Frame n Every macroblock is encoded independently (“I-macroblock”) NUS. SOC. CS 5248 Ooi Wei Tsang

For P-Frame n Every macroblock is either n I-macroblock n a motion vector + error terms wrt a prev I/P-frame (“P-macroblock”) NUS. SOC. CS 5248 Ooi Wei Tsang

For B-Frame n Every macroblock is either n I-macroblock n P-macroblock n a motion vector + error terms wrt a future I/P-frame n 2 motion vectors + error terms wrt a prev/future I/P-frame NUS. SOC. CS 5248 Ooi Wei Tsang

Review: MPEG structure n Sequence n GOP n Picture n Slice n Macroblock n Block NUS. SOC. CS 5248 Ooi Wei Tsang

MPEG Decoding (I-Frame) 101000101 Entropy Decoding Dequantize IDCT NUS. SOC. CS 5248 Ooi Wei Tsang

MPEG Decoding (P-Frame) 101000101 Entropy Decoding Dequantize IDCT Prev Frame NUS. SOC. CS 5248 Ooi Wei Tsang +

MPEG Decoding (B-Frame) 101000101 Entropy Decoding Future Frame Dequantize IDCT AVG Prev Frame NUS. SOC. CS 5248 Ooi Wei Tsang +

There are more. . n Half-pel Motion Prediction n Skipped Macroblock n etc. NUS. SOC. CS 5248 Ooi Wei Tsang

MPEG in daily life MPEG Standards Bit-rate Usage MPEG-1 1. 5 Mbps VCD MPEG-2 10 -40 Mbps DVD, SVCD, HDTV MPEG-4 Scalable Quick. Time, Div. X; ) NUS. SOC. CS 5248 Ooi Wei Tsang

The Future: H. 264 n Same quality at half the rate n How: n Variable block size n Multiple reference frame n. . . NUS. SOC. CS 5248 Ooi Wei Tsang

Hands-On n Download source code, compile and play with n ffmpeg n mpeg_stat n Try different MPEG-1 encoding parameter NUS. SOC. CS 5248 Ooi Wei Tsang

Impact on Systems Design n n How How : : NUS. SOC. CS 5248 Ooi Wei Tsang to to package data into packets? deal with packet loss? deal with bursty traffic? predict decoding time?