Basic Image Compression Concepts and MPEG Standard Presenter

Basic Image Compression Concepts and MPEG Standard Presenter: Pao-Yen Lin Research Advisor: Jian-Jiun Ding , Ph. D. Assistant professor Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Outlines • Introduction • Basic Concept of Image Compression • KLT and DCT • JPEG Standard • MPEG Standard 2

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Introduction • Constitution of Digital Camera 3

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Introduction • Downsampling of YCb. Cr data 4

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Introduction • Image Storage System 5

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Basic Concept of Image Compression • Lossy Coding • Lossless Coding 6

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Basic Concept of Image Compression • Why we can do image compression? 7

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Basic Concept of Image Compression • How can we do image compression? 1. Reduce pixel correlation 2. Enlarge the transformation region 3. Do transformation both on horizontal and vertical direction 4. Do transformation serially (Wavelet Transform) 5. Enlarge the quantization step size 6. Entropy coding 8

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Pixel Correlation Reduction 9

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Pixel Correlation Reduction 10

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Image Partition 11

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Karhunen-Loeve Transform (KLT) • Covariance Matrix 12

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Karhunen-Loeve Transform (KLT) • Covariance Matrix • : Covariance matrix of input image data • : Covariance matrix of output image data 13

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Karhunen-Loeve Transform (KLT) • : Covariance matrix of input image data 14

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Karhunen-Loeve Transform (KLT) • Properties 1. Optimal Orthogonal Transform with minimal MSE 2. Depends on image data • Drawbacks 1. Large computation time 2. Not unique 3. Additional data transmission 15

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Discrete Cosine Transform (DCT) • Definition 16

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Discrete Cosine Transform (DCT) • 8× 8 DCT basis 17

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Discrete Cosine Transform (DCT) • Properties 1. When correlation coefficient approaches to 1, it becomes Discrete Cosine Transform 2. Similar to KLT 3. Fast Algorithm 4. Excellent Energy Compaction 5. Real numbers 18

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Joint Photographic Experts Group Input image Pixel correlation reduction (DCT) Entropy coding Quantization Bit stream 19

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Baseline JPEG Encoder Huffman Table AC Color components (Y, Cb, or Cr) 8 8 DCT Zig-zag reordering Huffman coding JPEG bit-stream Quantizer Difference DC Encoding Quantization Table Huffman coding Huffman Table 20

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Quantization matrix 21

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Zig-zag scanning 22

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Entropy coding 1. DC coefficient Ø Differential coding 2. AC coefficient Ø Zero run length coding 23

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Zero Run Length Coding for AC Coefficient Example : 57, 45, 0, 0, 23, 0, -30, -16, 0, 0, 1, 0, 0, . . . , 0 (0, 57) ; (0, 45) ; (4, 23) ; (1, -30) ; (0, -16) ; (2, 1) ; EOB: (0, 0) Ø The JPEG Huffman coding makes the restriction that the number of previous 0's to be coded as a 4 -bit value 24

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 JPEG Standard • Differential Coding for DC Coefficient 25

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Basic Concept of Video Compression • Issues 1. Large amount of video data 2. Three dimensional data 26

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Basic Concept of Video Compression • 3 -D DCT • Intra-frame Coding Ø 2 -D DCT • Inter-frame Coding Ø Motion compensation 27

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Inter-frame Coding • Difference Coding • Motion Compensation 28

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 MPEG Block Diagram • Ref: http: //www. brogent. com 29

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Motion Compensation I B B P I 30

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Motion Compensation • Divide into Macroblocks (MBs) • Search in a specific range • Matching Criteria 31

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Bidirectional Motion Compensation 32

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Search Method • Full Search • Three Step Search • Diamond Search • Hierarchical Search 33

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Three Step Search 34

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Diamond Search 35

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Hierarchical Search 36

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Matching Criteria • Mean Square Error (MSE) • Mean Absolute Error (MAE) • Sum of Absolute Differences 37

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Conclusion 38

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Reference • R. C. Gonzalez and R. E. Woods, Digital Image Processing 2/E. Upper Saddle River, NJ: Prentice-Hall, 2002. • J. J. Ding and J. D. Huang, “Image Compression by Segmentation and Boundary Description, ” June, 2008. • 酒井善則．吉田俊之 共著，白執善 編譯，影像壓縮技術. 全華科技 圖書， 2004年 10月. • G. K. Wallace, 'The JPEG Still Picture Compression Standard', Communications of the ACM, Vol. 34, Issue 4, pp. 30 -44. 39

Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2021/9/8 Q&A 40