Discrete Cosine Transform l 1 D DCT expression

Discrete Cosine Transform l 1 D DCT: expression of a set of n samples, f(x) as a sum of n cosine basis functions 1 Ming-Yen Lin, IECS, FCU

* The cosine basis function for a set of 8 samples u=0, 438 u=4, -34 u=1, -55 u=5, -132 u=2, 45 u=6, 34 u=3, 52 u=7, 19 2 Ming-Yen Lin, IECS, FCU

u=0 u=4 u=1 u=5 u=2 u=6 u=3 u=7 3 Ming-Yen Lin, IECS, FCU

u=0 u=4 u=1 u=5 u=2 u=6 u=3 u=7 4 Ming-Yen Lin, IECS, FCU

2 D Discrete Cosine Transform l 2 D DCT: transfer spatial domain to frequency domain Ø DC: F(0, 0), average of f(x, y) Ø AC: other 63 coefficients 5 Ming-Yen Lin, IECS, FCU

The 64 (8 x 8) DCT basis functions Entropy coding vs. transform coding Ming-Yen Lin, IECS, FCU 6

JPEG l Joint Photographic Expert Group l Continuous-tone image compression standard (color, grayscale) l Joint: collaboration between CCITT(ITU-T), ISO, IEC (ISO JTC 1/SC 2/WG 10) l Official document: ISO/IEC 10918 -1, -2, -3 l History Ø 1987, selection process & narrow 12 proposal to 3 Ø 1988, select the DCT based method Ø 1988 -90, defining, documenting, simulating, testing, validating Ø 1991, draft Ø 1992, International standard Ming-Yen Lin, IECS, FCU 7

Goals of JPEG l be at or near the state of the art of compression methods l be applicable to practically any kind of continuous-tone digital image l have tractable computational complexity to make feasible software & hardware implementation l 4 modes of operations ØLossy sequential encoding (baseline mode) Øprogressive encoding (multiple scan) Ølossless encoding Øhierarchical encoding (multiple resolution) 8 Ming-Yen Lin, IECS, FCU

4 modes (illustration) Baseline sequential mode Expanded lossy DCT-based encoding (progressive) 9 Ming-Yen Lin, IECS, FCU

4 modes (illustration) (cont’d) Hierarchical encoding ３ ４ ７ 初始圖 ８ ２ ５ 原圖 (original image) １ 縮圖 (reduction) ６ 擴展圖 (expansion) ９ 預備編碼值 (coded result) 10 Ming-Yen Lin, IECS, FCU

Major Steps of DCT-based modes l Encoder 8 x 8 blocks DCT-Based Encoder FDCT Source Image Data Quantizer Entropy Encoder Compressed Image Data Table Specification 11 Ming-Yen Lin, IECS, FCU

Major Steps of DCT-based modes (cont. ) l Decoder DCT-Based Decoder Compressed Image Data Entropy Encoder Table Specification Quantizer Table Specification IDCT Reconstructed Image Data 12 Ming-Yen Lin, IECS, FCU

Major Steps: Baseline Sequential Encoding U V YUV DC coeff Ming-Yen Lin, IECS, FCU AC coeff 13

Color Model Transformation l Y = 0. 299 R +0. 587 G +0. 114 B l Cb = -0. 168 R -0. 331 G -0. 499 B l Cr = 0. 500 R -0. 419 G -0. 081 B 色相轉換 R Cr G B Cb Y 14 Ming-Yen Lin, IECS, FCU

Subsampling (1/3) 32 32 16 16 original 15 Ming-Yen Lin, IECS, FCU

4: 1: 1 Sampling (2/3) 16 Ming-Yen Lin, IECS, FCU

2: 1: 1 Sampling (3/3) 17 Ming-Yen Lin, IECS, FCU

Preprocessing l Components, planes: 1 C 255 Øin a C: different pixels (horizontal/vertical) allowed l Each pixel same depth 8, 12, 2 – 12 l component coding ØMinimum coded unit (MCU): 4 Y, 1 U, 1 V l Original sample value [0, 2 b-1] b bits used l 8 -bits per sample in baseline mode l Shift to [-2 b-1, 2 b-1 ], center at 0 l b=8, [-128, 127] l Allow for low-precision calculation in DCT 18 Ming-Yen Lin, IECS, FCU

Quantization l Quantization Øto achieve further compression by representing DCT coefficients with no greater precision than is necessary Ødiscard information which is not visually significant ØUniform quantization vs. Bit-rate control technique <e. g. > (110101)2= (53)10 quantization by 4 (1101)2 = (13)10 quantization by 8 (110)2 = (6)10 19 Ming-Yen Lin, IECS, FCU

Quantization (cont. ) l Adjust quantization step: tradeoff between compression ratio and image quality l Bit-rate control technique: Øallocate more bits to the coefficients with large variances Ødivision of DCT coefficients by corresponding quantizer step size Øprinciple source of lossiness in DCT-based encoder 20 Ming-Yen Lin, IECS, FCU

Quantization (Cont. ) l Human eye is more sensitive to low frequencies than to high frequencies Luminance Quantization Table -----------------------16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99 -----------------------Ming-Yen Lin, IECS, FCU Chrominance Quantization Table -----------------------17 18 24 47 99 99 18 21 26 66 99 99 24 26 56 99 99 99 47 66 99 99 99 99 99 99 99 99 99 99 -----------------------21

A Quantization example 22 Ming-Yen Lin, IECS, FCU

After Quan. , Entropy Coding 23 Ming-Yen Lin, IECS, FCU

Zig-Zag Scan l transfer quantized coefficients from 2 D to 1 D sequence l transfer (8 x 8) to a (1 x 64) vector l Order coefficients in the order of increasing frequency l group low frequency coefficients in top of vector l Most high coeff are zero, result in most zero at the end of scan l Leading to higher entropy and run-length encode efficiency 24 Ming-Yen Lin, IECS, FCU

Zig-Zag Scan 25 Ming-Yen Lin, IECS, FCU

Entropy Coding in JPEG l Additional lossless compression of DCT coefficients ØHuffman Coding (baseline sequential) ØArithmetic Coding n pro: 5~10% better compression n con: complexity l Steps Øconversion of quantized DCT coefficients to intermediate sequence of symbols Øassignment of variable length codes to symbols 26 Ming-Yen Lin, IECS, FCU

DPCM on DC Coefficients l DC component is large and varied, but often close to previous value. l DPCM (Differential Pulse Code Molulation) DC 3 DCi-1 5 (DC) DCi 2 DCi=DCi-1 27 Ming-Yen Lin, IECS, FCU

DPCM on DC Coefficients (cont. ) l Step 1: DCi (Size, Amplitude) l Step 2: Encoding Øsize: nindicating size of VLI of amplitude, n. VLC(huffman coding): input to encoder, application specific Øamplitude n. VLI (Variable Length Integer): hardwired into proposal 28 Ming-Yen Lin, IECS, FCU

Size & Amplitude ------------------SIZE Value ------------------1 -1, 1 (0, 1) 2 -3, -2, 2, 3 (00, 01, 10, 11) 3 -7. . -4, 4. . 7 (000, 001, 010, 011, 100, …. ) 4 -15. . -8, 8. . 15. . . 10 -1023. . -512, 512. . 1023 ------------------ l Categorize DC values into SIZE (number of bits needed to represent) and actual bits. 29 Ming-Yen Lin, IECS, FCU

RLC on AC Coefficients l 1 x 64 vector has lots of zeros in it l RLE: Run Length Encoding l Processing Steps Ø Step 1: AC (Run. Length, Size), (Amplitude) Ø Step 2 n Run. Length: VLC (Huffman coding) n Size: VLC n Amplitude: VLI 30 Ming-Yen Lin, IECS, FCU

Example: 31 Ming-Yen Lin, IECS, FCU

32

33

Lossless Mode l Predictive lossless coding (ratio 2 -3 : 1) 34 Ming-Yen Lin, IECS, FCU

Processing Steps of Lossless Mode Losseless Encoder Predictor Source Image Data Entropy Encoder Compressed Image Data Table Specification 35 Ming-Yen Lin, IECS, FCU

Predictors for Lossless coding Selection Value 0 1 2 3 4 5 6 7 Prediction no prediction A B C A+B-C A+(B-C)/2 B+(A-C)/2 (A+B)/2 Data unit in lossless Encoding is a single pixel C B A X Difference between X and predictor is entropy coded Ming-Yen Lin, IECS, FCU 36

Expanded lossy mode l Pixel depth of 12/ 8 bits l Huffman coding & Arithmetic coding l Progressive encoding & sequential encoding ØEach component encoded in multiple scans instead of single scan ØFirst scan rough, recognizable version for fast transmission, refined by succeeding scans ØNeed image sized buffer memory 37 Ming-Yen Lin, IECS, FCU

Progressive Mode l DCT-based encoding l Difference: DCT Coefficients of each image component is encoded in multiple scan MSB Successive Approximation LSB Zig-Zag Sequence MSB Spectral Selection LSB Zig-Zag Sequence Ming-Yen Lin, IECS, FCU MSB (LSB): Most (Least) 38 Significant Bit

Hierarchical Mode l Pyramid Coding at multiple resolution, each differ by factor 2 in either Hori/vertical T M O 39 Ming-Yen Lin, IECS, FCU

Hierarchical encoding (1) l Filter and down sample original image by desired multiples of 2 in each dimension >>T l Encode the reduced size image T using one of sequential, progressive, lossless encoding l Decode the reduced image and interpolate and up-sample by a factor of 2 as a prediction of M l Get difference between Prediction and M, encode the difference using one the 3 coding schemes l Repeat steps 3 – 4 until full resolution encoded 40 Ming-Yen Lin, IECS, FCU

Hierarchical encoding (2) l. Encoded data of T is stored first, then encoded differences next l. Useful when high resolution images be accessed by lower resolution device, no buffer to reconstruct full image then scale down l. In browsing mode, low resolution data transmitted, displayed for fast response 41 Ming-Yen Lin, IECS, FCU