ON BETWEENCOEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS

ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS 1 Nikolay Ponomarenko (*), Flavia Silvestri(**), Karen Egiazarian (***), Marco Carli (**), Jaakko Astola (***) and Vladimir Lukin (*) National Aerospace University, Kharkov, Ukraine (**) University of Rome "Roma TRE", Rome, Italy (***) Tampere University of Technology, Tampere, Finland Marco Carli VPQM 2006 26/01/2007

2 Outline 1. 2. 3. 4. 5. Introduction Proposed model of between-coefficient contrast masking of DCT basis functions Modification of PSNR using a new masking model MATLAB implementation of the proposed measure A set of test images for comparative analysis for taking into account the masking effect in quality metrics 6. Subjective experiment to test quality measures 7. Results of the experiment 8. Examples of quality assessment of test images 9. Example of use of the proposed model to masking noise on a real image 10. Summary and Conclusion Marco Carli VPQM 2006 26/01/2007

3 Introduction Human visual sensitivity varies as a function of several key image properties, such as: üLight level üSpatial frequency üColor Masking model can be used in : üImage and video compression üImage filtering üDigital watermarking üValidation of effectiveness of image processing methods üLocal image contrast üEccentricity üTemporal frequency Goal of the research: Efficient accounting for local image contrast using a model of betweencoefficient contrast masking of DCT basis functions Marco Carli Requirements to the model: Images compressed (filtered or processed) with accounting the model can be visualized in unknown illumination conditions, monitor brightness, distance to the monitor, viewing angle, etc. Thus such model should operate by only some averaged parameters of image visualization VPQM 2006 26/01/2007

4 Proposed model of between-coefficient contrast masking of DCT basis functions Let us denote a weighted energy of DCT coefficients of an image block 8 x 8 as Ew(X): (1) where Xij is a DCT coefficient with indices i, j, Cij is a correcting factor determined by the CSF. The DCT coefficients X and Y are visually undistinguished if Ew(X-Y) < max(Ew(X)/16, Ew(Y)/16), where Ew(X)/16 is a masking effect Em of DCT coefficients X (normalizing factor 16 has been selected experimentally). Reducing of the masking effect due to an edge presence in the analyzed image block: we propose to reduce a masking effect for a block D proportionally to the local variances V(. ) in blocks D 1, D 2, D 3, D 4 in comparison to the entire block: Em(D) = Ew(D)δ(D)/16, (2) where δ(D) = (V(D 1)+V(D 2)+V(D 3)+V(D 4))/4 V(D), V(D) is the variance of the pixel values in block D. Marco Carli VPQM 2006 26/01/2007

5 Proposed model of between-coefficient contrast masking of DCT basis functions Values of Cij have been obtained using the quantization table for the color component Y of JPEG (the values of quantization table JPEG have been normalized by 10 and squared) JPEG Quantization table of Y component Values of Cij ij 0 1 2 3 4 5 6 7 0 0 0. 8264 1. 0000 0. 3906 0. 1736 0. 0625 0. 0384 0. 0269 1 0. 6944 0. 5102 0. 2770 0. 1479 0. 0297 0. 0278 0. 0331 2 0. 5102 0. 5917 0. 3906 0. 1736 0. 0625 0. 0308 0. 0210 0. 0319 3 0. 5102 0. 3460 0. 2066 0. 1189 0. 0384 0. 0132 0. 0156 0. 0260 4 0. 3086 0. 2066 0. 0730 0. 0319 0. 0216 0. 0084 0. 0094 0. 0169 16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 5 0. 1736 0. 0816 0. 0331 0. 0244 0. 0152 0. 0092 0. 0078 0. 0118 72 92 95 98 112 100 103 99 6 0. 0416 0. 0244 0. 0164 0. 0132 0. 0094 0. 0068 0. 0069 0. 0098 7 0. 0193 0. 0118 0. 0111 0. 0104 0. 0080 0. 0100 0. 0094 0. 0102 Marco Carli VPQM 2006 26/01/2007

6 Modification of PSNR using a new masking model A basis of the proposed metric is a PSNR-HVS (Egiazarian K. , Astola J. , Ponomarenko N. , Lukin V. , Battisti F. , Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p). Flow-chart of PSNR-HVS-M calculation Reduction by value of contrast masking in accordance to the proposed model is carried out in the following manner. First, the maximal masking effect Emax is calculated as max(Em(Xe), Em(Xd)) where Xe and Xd are the DCT coefficients of a original image block and a distorted image block, respectively. Then, the visible difference between Xe and Xd is determined as: Marco Carli X∆ij = . where Enorm is VPQM 2006 26/01/2007

7 MATLAB implementation of the proposed measure The MATLAB implementation of PSNR-HVS-M is available on www. cs. tut. fi/~ponom/psnrhvsm. htm Marco Carli VPQM 2006 26/01/2007

A set of test images for comparative analysis for taking into account the masking effect in quality metrics 8 While creating an image test set we took into consideration the following: üSuch set should contain images with both spatially uncorrelated and correlated noise (the latter one is typical for images formed by digital cameras and is more visible for humans); üThe set should contain images with noise distributed spatially uniformly and with noise which is masked or unmasked (concentrated in regions with maximal and minimal masking properties, respectively); üThe set is to be maximally simple for visual comparison by humans (because of this in our set we used only three values of noise variance σ2 and a total number of distorted test images was 2 x 3 x 3 = 18 images). Original test images having a lot of different type regions with high masking effect Marco Carli VPQM 2006 26/01/2007

9 Subjective experiment to test quality measures Result of the experiment: the test image set ordered according to subjective visual quality. Number of observers: 155 (45 from Finland, 43 from Italy, 67 from Ukraine). Number of comparisons of visual appearance of test images: 8192 (on average 53 for each observer). 17” or 19” Monitor Resolution: 1152 x 864 pixels. Number of experiments carried out using CRT monitors: 128. Number of experiments carried out using LCD monitors: 27. Cross correlation factors Group of observers Spearman correlation Kendall correlation Finland – Italy 0. 996 0. 895 Finland – Ukraine 0. 996 0. 935 Italy - Ukraine 0. 997 0. 961 CRT - LCD 0. 998 0. 922 Marco Carli VPQM 2006 26/01/2007

10 Results of the experiment Spearman Kendall correlation Measure Reference PSNR-HVS-M This paper 0. 984 0. 948 PSNR-HVS Egiazarian K. , Astola J. , Ponomarenko N. , Lukin V. , Battisti F. , Carli M. “New fullreference quality metrics based on HVS”, CD-ROM Proceedings of the Second Intern. Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p 0. 895 0. 712 NQM Damera-Venkata N. , Kite T. , Geisler W. , Evans B. and Bovik A. "Image Quality Assessment Based on a Degradation Model", IEEE Trans. on Image Processing, Vol. 9, 2000, pp. 636 -650 0. 857 0. 673 Solomon J. A. , Watson A. B. , and Ahumada A. “Visibility of DCT basis functions: Effects of contrast masking”. Proc. of Data Compression Conf. , 1994, pp. 361 -370 http: //vision. arc. nasa. gov/dctune/ - DCTune 2. 0 page 0. 829 0. 712 Wang Z. , Bovik A. “A universal image quality index”, IEEE Signal Processing Letters, vol. 9, March, 2002, pp. 81– 84 0. 550 0. 438 0. 537 0. 359 DCTune UQI PSNR Peak Signal to Noise Ratio VQM Xiao F. “DCT-based Video Quality Evaluation”, Final Project for EE 392 J, 2000 0. 441 0. 281 SSIM Wang Z. , Bovik A. , Sheikh H. , Simoncelli E. “Image quality assessment: from error visibility to structural similarity”, IEEE Trans. on Image Proc. , vol. 13, 2004, pp. 600 -612 0. 406 0. 358 VIF Sheikh H. R. and Bovik A. C. , "Image Information and Visual Quality", IEEE Transactions on Image Processing, vol. 15, February, 2006, pp. 430 -444 0. 377 0. 255 PQS Miyahara, M. , Kotani, K. , Algazi, V. R. ”Objective picture quality scale (PQS) for image coding”, IEEE Transactions on Communications, vol. 46, issue 9, 1998, pp. 1215 -1226 0. 302 0. 242 Marco Carli VPQM 2006 26/01/2007

Examples of quality assessment of test images DCTune = 24. 9, PSNR-HVS-M = 33. 20 d. B PSNR-HVS-M says: “This is better!” Marco Carli 11 DCTune = 24. 5, PSNR-HVS-M = 29. 31 d. B DCTune says: “This is better!” VPQM 2006 26/01/2007

Examples of quality assessment of test images SSIM = 0. 80, PSNR-HVS-M = 25. 50 d. B SSIM says: “This is better!” Marco Carli 12 SSIM = 0. 79, PSNR-HVS-M = 31. 29 d. B PSNR-HVS-M says: “This is better!” VPQM 2006 26/01/2007

Example of use of the proposed model to masking noise on a real image Original test image Baboon Marco Carli 13 The image with masked noise, PSNR=26. 18 d. B, MSE=158, PSNR-HVS=34. 43 d. B, PSNR-HVS-M=51. 67 d. B VPQM 2006 26/01/2007

14 Summary and Conclusion Summary üA simple and efficient model of between-coefficient contrast masking of DCT basis functions is proposed; üA modification of PSNR that takes into account this masking model is proposed; üSubjective experiments on comparison of known quality metrics are carried out; Conclusions üThe proposed measure based on the designed masking model has demonstrated the best correspondence to the results of the subjective experiments. However for providing more reliable conclusions on efficiency of the proposed model it is necessary to carry out additional more extensive experiments and research. üThe proposed test set has allowed to demonstrate drawbacks of many well known metrics that do not fully or even badly correspond to human visual perception. Marco Carli VPQM 2006 26/01/2007