Image Processing and Computer Vision 9 1 Image

Image Processing and Computer Vision: 9 1

Image and Video Coding Compressing data to a smaller volume without losing (too much) information Image Processing and Computer Vision: 9

Why Code Data? To reduce storage volume n To reduce transmission time n n One colour image n n 760 by 580 pixels 3 channels, each 8 bits 1. 3 Mbyte Video data n n n Same resolution 25 frames per second 33 Mbyte/second Image Processing and Computer Vision: 9 3

Redundancy n Spatial n n Chromatic n n Correlation between colour channels Temporal n n Correlation between adjacent pixels Correlation between adjacent frames Perceptual n Unnoticed losses Image Processing and Computer Vision: 9 4

Example Image Processing and Computer Vision: 9 5

Redundancy - Consequences n Data exceeds information n n More data than content justifies Can lose data without losing information Image Processing and Computer Vision: 9 6

Compression Ratio Image Processing and Computer Vision: 9 7

Lossy vs Lossless Compression n Can lose data without losing information Lossily compressed images can look similar to the original Lossy compression has greater C Image Processing and Computer Vision: 9 8

Quality of Decoded Images n Measure differences between n Original Coded/decoded images Options n n n Maximum difference Average difference Subjective scales Image Processing and Computer Vision: 9 9

Example Image Processing and Computer Vision: 9 10

Subjective Quality Measurement Scales - Example 0 1 2 3 4 Unusable Annoying degradation Adequate images Barely perceptible degradation No observable degradation Image Processing and Computer Vision: 9 11

Difference coding n Adjacent pixels are similar n Difference is small n Uncompressed: n Compressed: Image Processing and Computer Vision: 9 12

Coding n n n Assign 4 bits to a difference Can code – 7, … , +8 Overflow? Use – 7 and +8 to show larger differences Code – 6, … , +7 directly Use overflow codes to indicate shift of codes Image Processing and Computer Vision: 9 13

Image Processing and Computer Vision: 9 14

Predictive Coding n If signals are well sampled n n Adjacent samples are correlated They have similar values Differences are small Can guess next sample from value of current Image Processing and Computer Vision: 9 15

n n n Constants are correlation coefficient and mean grey value Difference between real and predicted values are smaller Code as for difference coding Image Processing and Computer Vision: 9 16

Run Length Coding n Replace runs of equal brightness values by n (length of run, value) 12233444565 (1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5) n More use when few brightness values n e. g. fax Image Processing and Computer Vision: 9 17

Huffman Coding n n n Uses variable length codes Most frequently occurring grey value has shortest code Least frequently occurring values have longest codes Image Processing and Computer Vision: 9 18

Example Symbol Probability A B C D E F 0. 4 0. 3 0. 1 0. 06 0. 04 1 0. 4 0. 3 0. 1 2 3 0. 4 0. 3 0. 2 0. 1 0. 4 0. 3 Image Processing and Computer Vision: 9 4 0. 6 0. 4 19

Symbol Probability A B C D E F 0. 4 1 0. 3 00 0. 1 011 0. 1 0100 0. 06 01010 0. 04 01011 1 0. 4 0. 3 0. 1 2 1 00 011 0100 0101 Image Processing and Computer Vision: 9 0. 4 0. 3 0. 2 0. 1 3 1 00 011 0. 4 1 0. 3 00 0. 3 01 4 0. 6 0 0. 4 1 20

GIF n n n Applicable to images with 256 colours Replace sequences of bytes with shorter codes Most common sequences use shortest codes Image Processing and Computer Vision: 9 21

Wavelet coders n n Wavelet transform organises image content efficiently Can easily select data to be discarded Image Processing and Computer Vision: 9 22

Wavelet coders Image Processing and Computer Vision: 9 23

JPEG standard n n A subcommittee of ISO Optimised for a range of image subject matter Compression rates can be defined Quality inversely proportional to C Image Processing and Computer Vision: 9 24

Block Transform Coding original image decompose Sequence of 8 by 8 blocks - different planes treated separately (RGB, YUV etc. ) transform quantise entropy code Transformed blocks reduce redundancy and concentrate signal energy into a few coefficients discrete cosine transformation (DCT) Blocks with discarded information - goal is to smooth picture and discard information that will not be missed, e. g. high frequencies Image Processing and Computer Vision: 9 25

Block Transform Encoding DCT Zig-zag run length code Image Processing and Computer Vision: 9 quantise entropy code 010111000111…. . 26

Block Encoding Original image 139 144 149 153 144 151 153 156 150 155 160 163 159 161 162 160 DCT 1260 -1 -12 -5 -23 -17 -6 -3 -11 -9 -2 2 -7 -2 0 1 79 0 -2 -1 -1 -1 0 0 0 0 0 run length code 0 79 1 -2 0 -1 2 -1 0 0 Image Processing and Computer Vision: 9 Zig-zag Huffman code quantise 79 -2 -1 0 0 -1 -1 0 0 0 0 10011011100011…. 27

Block Transform Decoding DCT Zig-zag run length code Image Processing and Computer Vision: 9 quantise entropy code 010111000111…. . 28

Result of Coding and Decoding 139 144 149 153 144 151 153 156 150 155 160 163 159 161 162 160 144 146 149 152 148 150 152 154 155 156 157 158 160 161 162 Original block Reconstructed block -5 -4 -5 -1 -2 1 -1 0 0 1 3 1 1 2 5 -2 errors Image Processing and Computer Vision: 9 29

Discrete Cosine Transform 4 C(u)C(v) F[u, v] = n 2 with n-1 S S f(j, k) cos j=0 k=0 1 2 For w = 0 1 otherwise (2 j+1) u p 2 n cos (2 k+1) v p 2 n C(w) = Image Processing and Computer Vision: 9 30

Discussion n Where are lossy steps? n n Quatisation and subsampling before coding How is quantisation matrix chosen? n Its predefined by the standard after much experimentation Image Processing and Computer Vision: 9 31

Video coding n Many specific standards n n AVS, MOV, QT, … One universal standard n MPEG Image Processing and Computer Vision: 9 32

MPEG Standards n n Standard specifies audio, video and system layers MPEG-1: low data rates, poor quality: VHS quality at 1. 5 Mbits-1 MPEG-2: high quality hence high data rates: studio quality, 15 Mbits-1 MPEG-4: low data rates, small images, 64 kbits-1 Image Processing and Computer Vision: 9 33

MPEG-1 n Audio and video designed to work at CD ROM speeds: 1. 5 Mbits-1 n n n Video 1. 150 Mbits-1 Audio 0. 256 Mbits-1 System 0. 094 Mbits-1 Image Processing and Computer Vision: 9 34

MPEG-2 n n n Released in 1994 Aimed at digital TV, ATM. Additions for n n Interlaced video Scalable video coding Graceful degradation with noise Implementation of full standard impractical n Varying profiles/levels of conformity Image Processing and Computer Vision: 9 35

MPEG-4 n Coding specifically for multimedia objects Image Processing and Computer Vision: 9 36

Coding Algorithms n n n Frame sequence Motion compensation Frame coding Image Processing and Computer Vision: 9 37

Frame Sequence n I frames (Intraframes) n n P frames (Predicted frames) n n Coded independently of any other frame Derived from previous I frame by motion prediction B frames (Bidirectionally interpolated) n Interpolate motion compensated blocks between I and P frames Image Processing and Computer Vision: 9 38

Motion Compensation n Image is divided into macroblocks (16 x 16 pixels) Matching macroblocks are found by minimising differences Code differences and macroblock displacement Image Processing and Computer Vision: 9 39

Frame Coding n Use JPEG algorithms Image Processing and Computer Vision: 9 40

Summary n n n Why code data? Redundancy Assessment of compression Lossy vs. lossless compression Algorithms n JPEG, MPEG Image Processing and Computer Vision: 9 41

But what … is it good for? Engineer at the Advanced Computing Systems Division of IBM, commenting on the microchip in 1968 Image Processing and Computer Vision: 9 42