An Introduction to H 264AVC and 3 D

An Introduction to H. 264/AVC and 3 D Video Coding

Outline n Video Coding Concepts basic concept review ¨ image coding structure ¨ video coding structure ¨ n H. 264/AVC Introduction history ¨ performance comparison ¨ n H. 264/AVC Coding Tools ¨ ¨ ¨ n inter prediction intra prediction transform & quantization de-blocking filter entropy coding 3 D Video Coding 3 D video format ¨ multiview video coding ¨ n Summary with Q&A

Video Coding Concept -basic concept review -image coding structure -video coding structure

The Scope of Image and Video Coding Standardization n Only the Syntax and Decoder are standardized:

Images and Video

Needs for Video Compression n n Without compression ¨ Visual telephony (e. g. CIF @ 15 frames/s): n 325 (pels) x 288 (lines) x 15 (farmes/s) x 1. 5 bytes = 18. 25 Mbit/s ¨ Digital TV (ITU-T 601 4: 2: 0 @30 frames/s): n 720 (pels) x 480 (lines) x 30 (farmes/s) x 1. 5 bytes = 124. 4 Mbit/s ¨ HDTV (e. g. 1280 x 720 pels 4: 2: 0 @ 60 frames/s): Compression results in lower bit rates ¨ Lower transmission and storage cost

RGB vs. YCb. Cr [1/3]

RGB vs. YCb. Cr [2/3]

RGB vs. YCb. Cr [3/3]

Common YCb. Cr Formats

Subjective View

Block Based Coding [1/2]

Block Based Coding [2/2]

Group of Picture (GOP)

Video Coding Concept -basic concept review -image coding structure -video coding structure

Image Coding Structure

Transform

Quantization S: 0 1 2 3 4 5 6 7 (3 bits) Quantization: Quantization step-size Q=2: S/2 Quantization Levels (Q): Inverse quantization (x 2): Quantization error: 0 0 1 1 2 2 3 3 (2 bits) 00224466 0101 Quantization step-size Q=4: S/4 Quantization Levels (Q): Inverse quantization (x 4): Quantization error: 0 0 1 1 (2 bits) 00004444 0123

Effect of DCT + Quantization

Entropy coding

Video Coding Concept -basic concept review -image coding structure -video coding structure

Temporal Redundancy [1/2] n The amount of data to be coded can be reduced significantly

Standard Video Encoder

Block Based Motion Compensation [1/2]

Algorithms for Motion Estimation n Full Search ¨ Guarantee find the global minimum SAD ¨ high computational complexity n Fast Search ¨ Local minimum SAD ¨ Low computational complexity ¨ Reduce candidate blocks ¨ Reduce matching pixels in candidate blocks

Diamond Search

Video coding structure

H. 264/AVC Introduction -History -Performance comparison

History

Joint Video Team

MPEG-2 Has Hit A Wall

MPEG-4 in Comparison

H. 26 L Provides Focus

MPEG-4 “Adopts” H. 264

State of the Art Standards n MPEG-2 ¨ DVD, n DVT, since 1994 MPEG-4 ¨ DVR, Digital Still Camera, since 1999 ¨ ~1. 5 x coding gain over MPEG-2 (ASP) n MPEG-4 part 10, AVC (H. 264) ¨ Mobile video, DVB-H, Blu-ray Disc and etc. ¨ 2~3 x coding gain over MPEG-2

AVC Profiles

coding tools and profiles

H. 264/AVC Introduction -History -Performance comparison

Compare to Other Standard n Fair comparisons of H. 26 L(TML-8. 0) versus H. 263 v 3, MPEG-2, and MPEG-4 TML-8. 0 at half of the bit rate as MPEG-4 for the same visual fidelity ¨ Source from VCEG-N 18. doc (Soptember, 2001) ¨ n Objective evaluation Average improvement of TML-8. 0 over MPEG-2 (VM-5) of 5. 8 d. B PSNR (peak gain 7. 2 d. B) for equal bandwidths ¨ TML-8. 0 average gain of 3. 1 d. B relative to H. 263++ (High-Latency Profile) for equal bandwidths (up to 5. 2 d. B) ¨ Gain of 2. 2 d. B over MPEG-4 (Advanced Simple Profile) for equal bandwidths (max. 3. 6 d. B) ¨

Test Sets n “Streaming” Test: Four QCIF sequences coded at 10 Hz and 15 Hz (Foreman, Container, News, Tempete) ¨ Four CIF sequences coded at 15 Hz and 30 Hz (Bus, Flower, Garden, Mobile and Calendar, and Tempete) ¨ With B frame ¨ n “Real-Time Conversation” Test: Four QCIF sequences encoded at 10 Hz and 15 Hz (Akiyo, Foreman, Mother and Daughter, and Silent Voice) ¨ Four CIF sequences encoded at 15 Hz and 30 Hz (Carphone, Foreman, Paris, and Sean) ¨ Without B frames ¨

Objective evaluation [1/2]

Objective evaluation [2/2]

Subjective evaluation n Example: Sequence Mobile, frame 40

Perceptual Test of H. 264/AVC High Profile

Objective Performance of H. 264/AVC High Profile

Intra mode performance [1/2] Average gain of H. 264 to JPEG: 5. 2 d. B (luma) ¨ Average gain of H. 264 to JPEG 2000: 1. 12 d. B (luma) ¨ Average gain of Motion JPEG 2000 to H. 264: 1. 42 d. B (chroma) ¨ The smaller the bit rate, the higher the gain of H. 264 ¨

Intra mode performance [2/2]

Intra mode performance [chroma]

Intra mode performance [FRExt] n a set of 8 photographic monochrome test images ¨ n with resolutions from 512 x 512 up to 2048 x 3072 samples Average gain of H. 264/AVC HP to JPEG 2000: 0. 5 d. B ¨ over the entire test image set and all bit-rates

JPEG 2000 vs. H. 264 Intra

H. 264/AVC Coding Tools -Inter prediction -Intra prediction -Transform and Quantization -De-blocking Filter -Entropy Coding

Basic Coding Structure

Standard Tools Comparison

Motion Compensation

Macro Block Partitions

Example – Frame 1

Example – Frame 2

Example – Residual [no MC]

Example – Residual [16 x 16]

Example – Residual [8 x 8]

Example – Residual [4 x 4]

Example of Variable Block Sizes n Large block means Less bits for MVs ¨ More bits on residuals ¨ n Small block means More bits for MVs ¨ Less bits on residuals ¨

Summary n Key Features Enhances motion compensation ¨ Small blocks for transform coding ¨ De-blocking filter ¨ n 50% bit rate saving against any other standards, by Better prediction ¨ More computation ¨ More memory ¨ n Video coding layer is still based on hybrid video coding algorithm, buy with important differences