Media Compression NUS SOC CS 5248 2007 Roger
- Slides: 62
Media Compression NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
You are Here Encoder Decoder Middlebox Receiver Sender Network NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Why compress? n “Bandwidth Not Enough” n “Disk Space Not Enough” n Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208. 6 GB NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Optical Disc Formats (1) n CD: ~650 MB n DVD: n 4. 7 (4. 38) GB (single layer) n 8. 5 (7. 92) GB (dual layer) n Single and dual sided (up to 18 GB) n 1 X max. read speed: ~10 Mb/s n Video codec: MPEG-2 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
JPEG Compression NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 1: 1
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 3. 5: 1
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 17: 1
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 27: 1
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 72: 1
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 192: 1
Compression Ratio Quality Size Ratio Raw TIFF 1153 KB 1: 1 Zipped TIFF 982 KB 1. 2: 1 Q=100 331 KB 3. 5: 1 Q=70 67 KB 17: 1 Q=40 43 KB 27: 1 Q=10 16 KB 72: 1 Q=1 6 KB 192: 1 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Magic of JPEG n Throw away information we cannot see n Color information n “High frequency signals” n Rearrange data for good compression n Use standard compression NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Discard color information Y V NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) U
Color Sub-sampling NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 4: 4: 4 4: 2: 0 4: 2: 2 4: 1: 1
4: 2: 2 Sub-sampling Y V NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) U
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 4: 2: 0
Original Image (1153 KB) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) “ 4: 1: 0”
NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Discrete Cosine Transform Demo NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Quantization DC 242 65 23 5 8 8 8 -54 -10 -4 -2 8 8 8 16 13 6 3 5 8 8 16 32 2 1 -1 -2 / 8 8 16 32 64 Quantization Table NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) 30 8 = 2 0 -6 -1 0 0 0 0 0 AC
Differential Coding 30 8 2 0 25 3 1 0 27 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 30 8 2 0 -5 3 1 0 2 3 1 0 6 -1 0 0 2 1 0 0 0 4 0 1 0 0 0 0 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Zig-zag ordering 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Run-Length Encoding 27 3 1 0 2 1 0 0 4 0 1 0 0 0 27, 3, 2, 4, 1, 1, 0, 0, 0, 1, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Idea: Motion JPEG n Compress every frame in a video as JPEG n DVD-quality video = 208. 6 GB n Reduction ratio = 27: 1 n Final size = 7. 7 GB NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Video Compression NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Temporal Redundancy NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Motion Estimation NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Bi-directional Prediction NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Watch for Motion Vectors Demo NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
H. 261 P-Frame I-Frame NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG-1 B-Frame NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG Frame Pattern n Example display sequence: n IBBPBBP … n Example encoding sequence: n IPBBPBB NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Compression Ratio Frame Typical Ratio I 10: 1 P 20: 1 B 50: 1 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Sequence sequence header: • width • height • frame rate • bit rate • : NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
GOP: Group of Picture gop header: • time • : NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Picture pic header: • number • type (I, P, B) • : NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Picture NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Slice NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Macroblock NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Block 1 Macroblock = NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) Y Y U Y Y V
Structure Summary NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
For I-Frame n Every macroblock is encoded independently (“I-macroblock”) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
For P-Frame n Every macroblock is either n I-macroblock n a motion vector + error terms wrt a prev I/P-frame (“P-macroblock”) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
For B-Frame n Every macroblock is either n I-macroblock n P-macroblock n a motion vector + error terms wrt a future I/P-frame n 2 motion vectors + error terms wrt a prev/future I/P-frame NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG-1/2 File Formats n (Packetized) Elementary streams, ES & PES n Program streams PS (reliable mediums, e. g. , DVD) n Transport streams TS (for lossy mediums, e. g. , on-air broadcast) Video Source MPEG-2 Elementary Encoder Packetizer MPEG encoded streams Audio Source MPEG-2 Elementary Encoder Data Source NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) PES: *. m 2 v PES: *. m 2 a Packetizer Systems Layer MUX TS: *. ts *. m 2 t *. mpg Transport Stream Packetizer Flow chart © Manish Karir
Review: MPEG structure n ES, PS, TS n Sequence n GOP n Picture n Slice n Macroblock n Block NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG Decoding (I-Frame) 101000101 Entropy Decoding Dequantize IDCT NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG Decoding (P-Frame) 101000101 Entropy Decoding Dequantize IDCT Prev Frame NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) +
MPEG Decoding (B-Frame) 101000101 Entropy Decoding Future Frame Dequantize IDCT AVG Prev Frame NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang) +
There is more. . n Half-pel Motion Prediction n Skipped Macroblock n etc. NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
MPEG in Daily Life MPEG Standards Bit-rate Usage MPEG-1 1. 5 Mbps VCD MPEG-2 3 -45 Mbps DVD, SVCD, HDTV MPEG-4 Scalable Quick. Time, Div. X, AVCHD NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Camcorders in Daily Life n Different formats used n DV 25 (Mini. DV, DVCAM, DVCPRO) n Capacity: 1 hour ~ 13 GB n Speed: 25 Mb/s (user data) n Color sampling: 4: 1: 1 n Compression ratio: ~10: 1 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Codec Comparison n “M-JPEG” (e. g. , DV) versus “MPEG” Compression Technique “M-JPEG” (I-frames only) “MPEG” (Temporal compression) Compression ratio Low (10: 1 to 30: 1) High (>100: 1) Editing (frame-accurate) Easy Difficult Encoding/decoding complexity Symmetric Asymmetric Processing latency Low to Medium High Multi-generation loss Medium High n No “perfect” codec -> application dependent NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
High-Definition n Standard by ATSC n 18 different sub-formats n 720 p and 1080 i are the most interesting n 1280 x 720 x 60 p, 1920 x 1080 x 60 i (30 p) n 1. 4 Gb/s raw bandwidth n 10 – 20 Mb/s compressed (distribution, broadcast) n 100 – 135 Mb/s compressed (pro tapes: DVCPROHD, HDCAM; for editing) NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Consumer HD n HDV: MPEG-2 n 19 (720 p) / 25 Mb/s (1080 i) n Tape format n http: //www. hdv-info. org n AVCHD: H. 264 n 5 to 20 Mb/s n Hard disk format n http: //www. avchd-info. org/ NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Optical Disc Formats (2) n HD DVD n Capacity: 15 GB and 30 GB n 1 X speed: 36 Mb/s n Video codec: VC-1, H. 264, MPEG-2 n Blu-ray n Capacity: 25 GB and 50 GB n 1 X speed: 36 Mb/s n Video codec: VC-1, H. 264, MPEG-2 NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Recent Codec: H. 264 n Same quality at half the rate n Encoding complexity: ~4 X n How: n Variable block size motion compensation n Multiple reference frames n Deblocking filter n. . . n Also called MPEG-4 Part 10 or AVC NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Hands-On n Download source code, compile and play with ffmpeg n mpeg_stat n Video ‘Surfing_short. m 2 t’ from course web site (98 MB, HDV, transport stream) n n Try different MPEG-1/2 encoding parameter NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)
Impact on Systems Design n n How How : : to to package data into packets? deal with packet loss? deal with bursty traffic? predict decoding time? NUS. SOC. CS 5248 -2007 Roger Zimmermann (based on slides by Ooi Wei Tsang)