MPEG1 Video Part 1 Ketan MayerPatel CS 294

MPEG-1 Video (Part 1) Ketan Mayer-Patel CS 294 -9 : : Fall 2003

Encoding Techniques • • • Subsampling Transform Coding Run-length Encoding Predictive Encoding Entropy Encoding Quantization CS 294 -9 : : Fall 2003

Bitstream Organization Picture Data Seq. Header Width Height Frame Rate Buffer Control Seq. End Code GOP Header Time Code Picture Header Temporal Ref Picture Type Motion Vector Parameters All headers begin with 23 zeroes followed by 9 bits that indicate header type. Encoding process will never produce 23 zeroes. CS 294 -9 : : Fall 2003

Frame Types • 3 Frame Types: I, P, B I : All information for frame present. P: Predictively encoded from previous I or P. B: Predictively encoded from previous I or P and next I or P. I B B P B B I CS 294 -9 : : Fall 2003

Frame Order • Predictive relationships create an obvious problem: B-frames depend on the future. • Obvious solution: send the frames out of order. I B B P 1 2 3 4 1 3 4 2 B B P B B I 5 6 7 8 9 10 11 12 13 14 15 16 6 7 5 9 10 8 12 13 11 15 16 14 CS 294 -9 : : Fall 2003

Source Input • Before we describe how I-frames are encoded, we should describe our input. • 3 planes of Y, U, V – 8 bits per pixel. – Y range [0, 255]. – U and V range [-128, 127] • Planes are all of the same size. • Pixels colocated between frames. CS 294 -9 : : Fall 2003

Chrominance Subsampling • • • First step: downsize chrominance. 4: 2: 0 (with chrominance samples centered) Requires bilinear interpolation. U and V biased by 128 to put in range [0, 255] Compression Ratio: 2: 1 Wow, doing well already. CS 294 -9 : : Fall 2003

Subsampling In General • Severe loss of data. • Exploits imperceptibility of data loss. – In this case: human not as sensitive to color. • What if we were using images as input to feature extractor? – Depending on what was being extracted, subsampling might not be such a good idea. • Compression gain is directly related to subsampling factors. CS 294 -9 : : Fall 2003

Macroblocks • Y is cut into 8 x 8 tiled pixel regions. • U and V cut into 8 x 8 tiled pixel regions. • Macroblock defined as 4 Y tiles that form a 16 x 16 pixel region and associated U and V tiles. • Macroblocks organized in row order fashion from top to bottom. • Compression gain: none. CS 294 -9 : : Fall 2003

Discrete Cosine Transform • Each tile (aka block) in a macroblock is transformed with a 2 D DCT. • DCT is an orthonormal, separable, frequency basis much like a Fourier transform. • 1 -D case: 8 pixel values are transformed into 8 DCT coefficients. • 2 -D case: apply 1 -D transform to all of the rows and then apply 1 -D transform to all of the columns. CS 294 -9 : : Fall 2003

DCT Basis Functions DC AC CS 294 -9 : : Fall 2003

DCT Properties • 8 -bit pixel values produces 12 -bit signed coefficient values. • Fast algorithms exist for computation. – 13 multiplies and 29 additions – Fixed point integer math. • Good perceptual properties. – Losing higher freq. results in a bit of blurring. – Ringing fairly minimal. CS 294 -9 : : Fall 2003

Transform Coding Properties • No loss of data – Except for numerical errors • No compression either. • Used to rearrange the data into a form to make another coding technique more effective. CS 294 -9 : : Fall 2003

Coefficient Quantization • Each block is now 64 coefficients instead of 64 pixel values. • Each coefficient quantized independently. – Allows larger quantization factors to be used with higher frequency coefficients. • Quantization is controlled by two parameters: – Quantization table. • Set in picture header or system header. • Two different tables, one each for intra and non-intra blocks. – Quantization factor. • Can be set on a per macroblock basis. Used to scale the table. • Can take value from (2 -62) CS 294 -9 : : Fall 2003

Quantization Properties • Data loss relative to quantization step. • Compression in two ways: – Smaller range to represent. • In our case 12 bit signed values turn into 9 -bit signed values. – Creates runs of the same number. – In our case: runs of zeroes. CS 294 -9 : : Fall 2003

Run Length Encoding • High quantization step size for higher frequency components results in lots of zero coefficients. • Run Length Encoding provides better representation. – Convert 2 D matrix into 1 D ordering of coefficients. – Reorganize as (run, value) pairs. – Run specifies number of zeroes to insert in the ordering before value appears. – Special marker that indicates nothing left but zeroes. CS 294 -9 : : Fall 2003

Zig-Zag ordering • In order to group as many of the zeroes together, zig-zag ordering used. CS 294 -9 : : Fall 2003

RLE Properties • Compression related to avg. size of run. • No data loss. CS 294 -9 : : Fall 2003

DC Term Encoding • At this stage, each block in our macroblock is represented as a set of RLE’d DCT coefficients. • DC term is always coded even if it is zero. – Coded as difference between last DC term and current DC term. – Blocks are ordered within a macroblock. • Why code the difference? – Avg. pixel value of one block is likely to be correlated to nearby block. CS 294 -9 : : Fall 2003

DC Term Encoding Cont’d • Now DC term is expressed as difference from previous DC term (DC_DIFF) • Encoded as two parts: – Size of difference (i. e. , log(DC_DIFF)) – Size number of bits that provides the value. • Size is encoded as a Huffman code. CS 294 -9 : : Fall 2003

Differential Encoding • Useful when values being encoded are well correlated. • Distribution of differences is expected to not be uniform. • No compression per se, but increases the efficiency of entropy encoding techniques (i. e. , Huffman coding) CS 294 -9 : : Fall 2003

AC Term Encoding • AC terms are given as (run, value) pairs. • Encoded in one of two ways: – Huffman code for (run, abs(value)) followed by single bit for sign of value. – Special Huffman code indicating ESCAPE, followed by 6 bits for run and either 8 or 16 bits for value. • 6 bits for run simply encode 0 through 63 • First 8 bits of value put value at – 128 to 127. • If first 8 bits is -128, next 8 bits provide codes for – 128 through – 255 • If first 8 bits is 0, next 8 bits provide codes for 128 through 255. CS 294 -9 : : Fall 2003

Entropy Coding • Huffman codes are a form of entropy encoding. • Relies on uneven distribution of values to be encoded. • Length of code associated with values inversely related to weight in distribution. – The more likely the value is to occur, the small the code length relative to all the other codes. • No data loss. • Compression depends on distribution. CS 294 -9 : : Fall 2003

Stepping Back A Bit Picture Header Picture Data Luminance Blocks Row Major Scan of Encoded Macroblocks U Block V Block First Non-zero AC Coeff. (variable bit length) Last Non-zero AC Coeff. (variable bit length) Q Scale (5 bits) Macroblock Type (1 or 2 bits) Macroblock Address Increment (1 -bit) DC Bits (0 -8 bits) DC Size (2 -7 bits) CS 294 -9 : : Fall 2003 EOB (2 bits)

Slices • One last level of organization. • Macroblocks grouped into slices. – Typically, one row of macroblocks in one slice. – Other groupings also possible. • Slice starts with a slice header. – Contains qscale. and indicates row in which slice starts. • Decoder state is reset. – DC predictors for Y, U, and V set to 1024. – Prev. macroblock address set to address of first macroblock in slice row (may not be first macroblock in slice). CS 294 -9 : : Fall 2003

I-Frame Review • All macroblocks are intra-coded. • Blocks DCT’d and quantized to produce coefficients. • DC terms encoded differentially. • AC terms encoded with entropy codes associated with (run, value) pairs. – Escape code with fix length encoding for seldom used possibilities. • In general, compression ratio is 10: 1 to 20: 1 CS 294 -9 : : Fall 2003