Hierarchical Prediction Structures in H 264AVC KaiChao Yang

Hierarchical Prediction Structures in H. 264/AVC Kai-Chao Yang 1

Outline Analysis of Hierarchical B Pictures and MCTF ICME 2006 Multiple Description Video Coding using Hierarchical B Pictures ICME 2007 Rate-Distortion Optimization for Fast Hierarchical Picture Transcoding ISCAS 2006 All Related Researches 2

Analysis of Hierarchical B Pictures and MCTF Heiko Schwarz, Detlev Marpe, and Thomas Wiegand ICME 2006 3

Hierarchical B-Pictures (1/2) Key pictures GOP Hierarchica l prediction IDR GOP Hierarchica l prediction I/P Hierarchical prediction structures Dyadic structure … … 4 … … … Non-dyadic structure … I/P

Hierarchical B-Pictures (2/2) Coding delay Minimum coding delay = hierarchy levels – 1 Coding order 0 5 4 3 2 Memory requirement Maximum decoded picture buffer (DPB): 16 Reference picture buffering type Sliding window Adaptive memory control Frame buffer Short-term Long-term frames 0 1 2 … … N-2 N-1 N Memory 5 o o o management control operation (MMCO) 0: End MMCO loop 1: mark a Short-term frame as “Unused” 2: mark a Long-term frame as “Unused” 3: assign a Long-term index to a frame 4: specify the maximum Long-term frame index 5: reset 1 N e w O l d replace Thomas Wiegand, “Joint Committee Draft (CD), ” Joint Video Team, JVT-C 167, 6 -10 May, Minimum DPB size =2002 hierarchy levels

Coding Efficiency of Hierarchical BPictures QPk = QPk-1 + (k=1 ? 4: 1) High spatial detail and slow regular motion Problem : PSNR fluctuations 6 Fast and complex motion

Visual Quality Comparison of visual quality Finer detailed regions of the background using larger GOP sizes. IBBP 7 GOP 16

MCTF Versus Hierarchical B-Pictures Drawbacks of MCTF Open-loop encoder control Significant cost in update stage 8

Multiple Description Video Coding using Hierarchical B Pictures Minglei Liu and Ce Zhu ICME 2007 9

Concept of Multiple Description Coding Multiple bit-streams are generated from one source signal and transmitted over separate channels Source signal MDC encoder S 1 Channel 1 S 2 Channel 2 Decoder 1 Decoded signal from S 1 Decoder 2 Decoded signal from S 1 and S 2 Decoder 3 Decoded signal from S 2 MDC decoder 10

The proposed architecture for MDC GOP size = 8 Two output streams (S 1, S 2) are generated GOP S 1 i i+8 GOP S 2 i+1 i+9 Combination … … 11 i i+1 i+2 i+3 i+4 i+5 i+6 i+7 i+8 i+9

Coding Efficiency (1/2) Improvement of coding efficiency Increasing QP values for higher layers Transmitting MVs only for higher layers Skipping frames at higher layers 12

Coding Efficiency (2/2) Max. QP = 51 for highest level Side distortion 13 Central distortion

Rate-Distortion Optimization for Fast Hierarchical Picture Transcoding Huifeng Shen, Xiaoyan Sun, Feng Wu, and Shipeng Li ISCAS 2006 14

Rate Reduction Transcoding (1/3) Cascaded pixel-domain transcoding structure Fully decoding the original signal, and then re-encoding it 15 A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Rate Reduction Transcoding (2/3) Open-loop transcoding in coded domain Partially decoding the original signal and re-quantizing DCT coefficients drift 16 A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Rate Reduction Transcoding (3/3) Closed-loop transcoding with drift compensation Partially decoding the original signal, and then compensating the re-quantized drift data 17 A. Vetro, C. Christopoulos, and H. Sun, "Video transcoding architectures and techniques: an overview", IEEE Signal processing magazine, March 2003.

Hierarchical B Pictures Transcoding Open-loop transcoding method can be used Motion information is unchanged; DCT coefficients are truncated, re-quantized, or partially discarded Drift inside a GOP will not propagate to other GOPs However, motions are more important in hierarchical B-pictures structure At low bit-rate, most bits are spent on motion information Proposed RDO model – combination of texture RDO and motion RDO 18

Traditional Rate-Distortion Model RD model S = (S 1, …, Sk) denotes k MBs I = (I 1, …, Ik) denotes k coding parameters of S Fully decoding and re-encoding is needed! 19

Proposed Rate-Distortion Model (1/4) Proposed RD model Claim Rtexture: rate spent in coding quantized DCT coefficients Rmotion: rate spent in coding MB modes, block modes, and Dtexture: MVs distortion caused by downscaled texture with unchanged MVs Dmotion: distortion caused by motion adjustment relative to the unchanged motion case 20

Proposed Rate-Distortion Model (2/4) Texture RDO model To minimize the RD function, Let 21 N. Kamaci, Y. Altunbasak, and R. M. Mersereau, "Frame bit allocation for the H. 264/AVC video coder via Cauchy-density-based rate and distortion models", IEEE Trans. on CSVT, Vol 15, No. 8, Aug. 2005. 2. 54 -5. 35

Proposed Rate-Distortion Model (3/4) Motion RDO model Rmotion can be easily computed, but Dmotion is unknow Dmotion can be approximated by mv mean-square error 22 A. Secker and D. Taubman, "Highly scalable video compression with scalable motion coding", IEEE Trans. on Image Processing, Vol. 13, No. 8, August 2004.

Proposed Rate-Distortion Model (4/4) Motion adjustment Original Adjustment 23 … …

Simulation results 24

All related researches Rate control optimization Bit allocation Trade-off between coding efficiency and delay Multi-view Temporal scalable coding in SVC Elimination of PSNR fluctuation? More efficient hierarchical structures? 25