JVTO 026 Invocation of the Inverse MCTF Process

JVT-O 026 Invocation of the Inverse MCTF Process Panasonic Singapore Laboratories Men. Huang Lee 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Introduction MCTF structure not fixed Switch dynamically between Haar filter and 5/3 transform filter Multiple reference frames possible Inter-partition prediction/update steps removed for low delay mode operation In SVC WD 1. 0, not clear when inverse MCTF is invoked In JSVM S/W, inverse MCTF invoked at start of next GOP 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Problems Significant time delay to complete inverse MCTF of whole GOP Periodic complexity peaks to perform inverse MCTF of whole GOP Large memory requirement to buffer all pictures in a GOP until inverse MCTF process is invoked Low delay mode operation not possible 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Proposed Solution Define inverse MCTF process such that inverse update/prediction steps are distributed across all pictures of a partition, as soon as the step can be performed When to invoke inverse MCTF? How to determine when an inverse update/prediction steps can be performed? What to do during inverse MCTF? 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

When to invoke inverse MCTF? After all slices of the current picture, pic, is decoded from the bitstream Bitstream-decoded picture decoded from video stream but before it undergoes the inverse MCTF process 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

How to determine when a step can be performed? Define, initialize, use & update a set of variables for each picture, pic: coding order 1 0 2 4 6 display order 0 prediction 1 2 3 4 5 6 7 8 3 = 3 cur. Temporal. Level H H H update L 5 L 7 3 temporal level H 2 L L cur. Temporal. Level 2 = 2 prediction H H update L 1 cur. Temporal. Level L 1 = 1 prediction H update 0 cur. Temporal. Level =0 L 0 GOP border Variable Definition/Purpose cur. Temporal. Level § Specifies the current temporal level of picture pic in the MCTF structure, as illustrated. § Initialized based on Temporal. Level derived from syntax element decodability_dependency_information. § Increment by 1 after every inverse update step or inverse prediction step. § If a reference picture (L picture) of a residual picture (H picture) have same value of cur. Temporal. Level as H picture, the L picture needs to be inverse updated before being used by H picture as reference picture for inverse prediction. 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

How to determine when a step can be performed? Variable Definition/Purpose max. Active. Prd. L 0 max. Active. Prd. L 1 § Specify the total number of pictures in reference picture lists (Ref. Pic. List 0 and Ref. Pict. List 1) of pic required for the inverse prediction step of pic, if pic is a residual picture. § Set equal to 1 plus the maximum value of the syntax element num_ref_idx_l. X_active_minus 1 of all slices inside the picture pic, if pic is a residual picture. Set to 0, if otherwise. 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

How to determine when a step can be performed? temporal level 2 L 1 1 L L H L L pic L L L H L L (max. L 0, max. L 1) = (1, 1) (cur. L 0, cur. L 1) = (0, (1, 0) 0) 1) Variable Definition/Purpose cur. Active. Prd. L 0 cur. Active. Prd. L 1 § Specify the current number of available pictures in reference picture lists (Ref. Pic. List 0 and Ref. Pict. List 1) of pic, if pic is a residual picture. § Initialized to 0. § Available picture refers to a reference picture that has been inverse updated and ready to be used as reference picture for the inverse prediction of residual picture pic. § A reference picture is determined to be available if its cur. Temporal. Level is equal to 1+cur. Temporal. Level of residual picture pic. § Increment by 1 when a reference picture is determined to be available. § When cur. Active. Prd. L 0 = max. Active. Prd. L 0 and cur. Active. Prd. L 1 = max. Active. Prd. L 1, all pictures in reference picture lists of pic are available and pic is ready for inverse prediction. 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

How to determine when a step can be performed? Variable Definition/Purpose max. Active. Upd. L 0[] max. Active. Upd. L 1[] § 2 arrays to specify the total number of pictures in update picture lists (upd. Pic. List 0 and upd. Pic. List 1) of pic required for the inverse update steps of pic at different temporal levels, upd. Level. § Set equal to the maximum value of the syntax element num_ref_idx_update_l. X_active[upd. Level] of all slices inside the picture pic. 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

How to determine when a step can be performed? L L L H L L L (max. L 0, max. L 1) (0, 1) 1) (cur. L 0, cur. L 1) (0, 0) L L L H L (1, 1) (0, 0) (1, 1) Definition/Purpose cur. Active. Upd. L 0[] cur. Active. Upd. L 1[] § 2 arrays to specify the current number of available pictures in update picture lists (upd. Pic. List 0 and upd. Pic. List 1) of pic at different temporal levels, upd. Level. § Initialized to 0. § Available picture refers to a residual picture that has been bitstream-decoded and ready to be used as update reference picture for the inverse update of a picture pic. § After a residual picture pic is bitstream-decoded, it is determined to be available for upd. Pic. List. Y of reference picture ref. Pic in Ref. Pict. List. X of pic, where X = (0, 1) and Y = 1 -X. § cur. Active. Upd. LY[cur. Temporal. Level] of ref. Pic is incremented by 1 when an update reference picture is determined to be available. § When cur. Active. Upd. L 0[upd. Level] = max. Active. Upd. L 0[upd. Level] and cur. Active. Upd. L 1[upd. Level] = max. Active. Upd. L 1[upd. Level], all pictures in update picture lists of pic are available and pic is ready for inverse update at temporal level upd. Level. L H H L L Variable L (1, 0) (1, (0, 0) 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

What to do during inverse MCTF? L L H H L (max. Active. Prd. L 0, max. Active. Prd. L 1) (cur. Active. Prd. L 0, cur. Active. Prd. L 1) L L (1, 2) (2, 1) (0, 0) (1, 1) 2) (0, (1, 0) (2, 1) L L L H (max. Active. Upd. L 0, max. Active. Upd. L 1) (cur. Active. Upd. L 0, cur. Active. Upd. L 1) L L H L L (0, 2) (1, 1) (1, 0) (0, 1) (0, 0) 1) 2) L L (2, 0) (0, (1, 0) (2, 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

What to do during inverse MCTF? Start L picture, upd. Pic 1 Determine if upd. Pic is ready for inverse update 2 Check for inverse update for all pictures upd. Pic in reference picture lists of prd. Pic 1 Ready? H picture, prd. Pic Determine if prd. Pic is ready for inverse prediction Yes Perform inverse update for upd. Pic No Check for inverse prediction for all pictures prd. Pic in update picture lists of upd. Pic Done Ready? 2 Yes No Perform inverse prediction for prd. Pic Done 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Illustration coding order display order 0 1 0 2 4 6 5 7 3 1 2 3 4 5 6 7 8 temporal level 3 prediction H H 2 update L L 2 prediction H H 1 update L L 1 prediction H 0 update L GOP border coding order display order 0 0 GOP border 1 0 2 4 6 5 7 3 1 2 3 4 5 6 7 8 temporal level 3 prediction H H 2 update L L 2 prediction H H 1 update L L 1 prediction H 0 update L GOP border 0 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Proposed Modification to SVC WD “S. 8 Decoding process” - added description of when to invoke inverse MCTF process “S. 8. 8 Inverse motion-compensated filtering process” - replaced “S. 8. 8. 3 Check for inverse prediction process” added “S. 8. 8. 4 Check for inverse update process” added 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic

Conclusion Inverse update/prediction steps distributed across all pictures of a partition, as soon as the step can be performed Propose when to invoke inverse MCTF Propose how to determine when an inverse update/prediction steps can be performed, via a set of variables Propose what to do during inverse MCTF Reduce complexity, spreading periodic complexity peaks over time Reduce time delay, support low delay mode operation Reduce memory requirement No addition of syntax element No effect on coding efficiency 72 nd MPEG Meeting, Busan, Korea, Apr 05 Men. Huang Lee, PSL Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic ideas for life Panasonic