Multimedia Image Compression 1 T SharonA Frank Coding

Multimedia Image Compression 1 T. Sharon-A. Frank

Coding Techniques – Hybrid 2 T. Sharon-A. Frank

Hybrid coding • Images: – JPEG • Video/Audio – MJPEG – MPEG (1, 2, 4) – Other codings – H. 26 x 3 T. Sharon-A. Frank

Image Compression • Image files may be too big for network transmission, even at low resolutions. • Use more sophisticated data representation or discard information to reduce data size. • Effectiveness of compression will depend on actual image data. • For any compression scheme, there will always be some data for which 'compressed' version is actually bigger than the original. 4

Compression Steps Picture Image Preparation Uncompressed Picture Processing Quantization Entropy Encoding Image Compressed 1. Preparation: analog to digital conversion. 2. Processing: transform data into a domain easier to compress. 3. Quantization: reduce precision at which the output is stored. 4. Entropy encoding: remove redundant information in the resulting data stream. 5 T. Sharon-A. Frank

JPEG Joint Photographic Experts Group Goals: • Support still images • True color/grayscale • Continuous-tone pictures • Usually lossy 6 T. Sharon-A. Frank Subjects: • Quality • Steps • Modes

JPEG Picture Quality • 0. 25 – 0. 5 bits/pixel Moderate to good quality. • 0. 5 – 0. 75 bits/pixel Good to very good quality. • 0. 75 – 1. 5 bits/pixel Excellent quality. • 1. 5 – 2. 0 bits/pixel Usually indistinguishable from the original. 7 T. Sharon-A. Frank

JPEG Compression • Lossy technique, well suited to photographs, images with fine detail and continuous tones. • Consider image as a spatially varying signal that can be analysed in the frequency domain. • Experimental fact: people do not perceive the effect of high frequencies in images very accurately. • Hence, high frequency information can be discarded without perceptible loss of quality. 8

Stage 1: Image Preparation Components Sample s Left Cn C 2 * * * * Y * * * X i C 1 1<= n <=255, usually 3 9 Top T. Sharon-A. Frank Bottom Line j Right

3 components having the same resolution X X A 1 A 2 B 1 B 2 Y C 1 C 2 Y An 10 X Y Bn T. Sharon-A. Frank Cn

3 components having different resolution X A 1 A 2 Y X/2 B 1 B 2 C 1 C 2 Y An 11 Y Bn/2 T. Sharon-A. Frank Cn/2

Changing Resolution • If image resolution < output device resolution, must interpolate extra pixels – Always leads to loss of quality. • If image resolution > output device resolution, must downsample (discard pixels): – Quality will often be better than that of an image at device resolution (uses more information). – Image sampled at a higher resolution that of intended output device is over sampled. 12

Block Preparation (a) RGB input data. (b) After block preparation. 13 T. Sharon-A. Frank

Non Interleaved Order of Data Units MCU = Minimum Coded Unit Left Top * * * * * * * Bottom 14 T. Sharon-A. Frank * * * * Right

Interleaved Data Units 1 2 3 * * * 1 2 0 0 5* 0 1 2 1 15 0 5 * * * 3 * * * * 4 * * 3 * * 0 0 2* 1 2 3 1 * * * * 4 * * T. Sharon-A. Frank * * 0 1 0 2 1 * * *

Stage 2: Image Processing • • Using DCT (Discrete Cosine Transform) Convert from Spatial to Frequency Domain Input : 8 x 8 matrix of 64 values Output: – 1 DC value (basic color of block = average color) – 63 AC values • low frequency coefficients, have high values • high frequency coefficients – represent sharp edges, have low values 16 T. Sharon-A. Frank

Discrete Cosine Transform (DCT) 17 • A Fourier-related transform similar to the Discrete Fourier Transform (DFT), but using only real numbers. • Often used in signal and image processing. • Maps values from spatial domain to frequency domain – image areas with low frequency (large blocks of single color) are compressed more efficiently. • Used in JPEG, MPEG, and DV Compression. T. Sharon-A. Frank

DCT Dynamics • Analyses a signal into its frequency components. • Takes array of pixel values, produces an array of coefficients of frequency components in the image. • Computationally expensive process – time proportional to square of number of pixels. • Usually applied to 8 x 8 blocks of pixels. 18

JPEG: DCT-Based Encoder 8 x 8 blocks FDCT Source Image Data 19 Quantizer Table Specification T. Sharon-A. Frank Entropy Encoder Compressed Image Data Table Specification

Preparation of DC Coefficients DCi-1 DCi Blocki-1 Blocki DIFF=DCi-1 20 T. Sharon-A. Frank

DCT Transformation (a) One block of the Y matrix. (b) The DCT T. Sharon-A. Frank coefficients. 21

Preparation of AC Coefficients DC AC 01 AC 07 Zig-Zag Sequence AC 70 22 AC 77 T. Sharon-A. Frank

Probability Distribution of DCT Coefficients 1. 0 Probability of being nonzero 0. 5 23 0. 0 77 8 16 24 T. Sharon-A. Frank 32 40 48 Zigzag index 56

Next Stages • Stage 3: Quantization Based on quantization table: – Quantization table: values from 1 to 255 – DCT coefficient/Table value – If value > 0, keep DCT coefficient else, don’t keep DCT coefficient • Stage 4: Entropy Encoding 24 T. Sharon-A. Frank

JPEG Quantization • Applying DCT does not reduce data size – Array of coefficients is same size as array of pixels. • Allows information about high frequency components to be identified and discarded. • Use fewer bits (distinguish fewer different values) for higher frequency components. • Number of levels for each frequency coefficient may be specified separately in a quantization matrix. 25

Computation of the quantized DCT coefficients 26 T. Sharon-A. Frank

Order in which the quantized values are transmitted 27 T. Sharon-A. Frank

JPEG Entropy Encoding • After quantization, there will be many zero coefficients. – Use RLE on zig-zag sequence (maximizes runs). • Use Huffman coding of other coefficients (best use of available bits). 28

Steps of JPEG Compression Picture Preparation Picture Processing Pixel Predictor Block, MCU FDCT Entropy Encoding Quantization Run-Length Huffman Arithmetic Steps taking into account the different JPEG modes 29 T. Sharon-A. Frank

JPEG Decompression • Expand runs of zeros and decompress Huffmanencoded coefficients to reconstruct array of frequency coefficients. • Use Inverse Discrete Cosine Transform to take data back from frequency to spatial domain. • Data discarded in quantization step of compression procedure cannot be recovered. • Reconstructed image is an approximation (usually very good) to the original image. 30

JPEG: DCT-Based Decoder Compressed Image Data Entropy Encoder Table Specification 31 Dequantizer Table Specification T. Sharon-A. Frank IDCT Reconstructe d Image Data

JPEG Modes of Operation • Lossless Encoding: the image is encoded to guarantee exact recovery of every source image sample value (low compression ratio). • Sequential Encoding: each image is encoded in a single left-to-right, top-to-bottom scan. • Progressive Encoding: the image is encoded in multiple scans for applications in which transmission time is long. • Hierarchical Encoding: the image is encoded at multiple resolutions. 32 T. Sharon-A. Frank

Sequential Presentation 33 T. Sharon-A. Frank

Progressive Presentation 34 T. Sharon-A. Frank

Types of image processing in DCT Image display 35 Sequential Progressive successive Progressive spectral Bits per sample Entropy coding 8 8 12 12 12 T. Sharon-A. Frank 12 Huffman coding Arithmetic coding Huffman coding Arithmetic coding

Hybrid Coding Examples • JPEG – image compression using a discrete cosine transform, then quantization, then Huffman coding. • JPEG 2000 – image compression using wavelets, then quantization, then entropy coding. • MP 3 – A part of the MPEG-1 standard for sound and music compression, using subbanding and MDCT, perceptual modeling, quantization, and Huffman coding. 36 T. Sharon-A. Frank