Patchbased Image Deconvolution via Joint Modeling of Sparse
Patch-based Image Deconvolution via Joint Modeling of Sparse Priors Chao Jia and Brian L. Evans The University of Texas at Austin 12 Sep 2011 1
Non-blind Image Deconvolution � Reconstruct natural image from blurred version � Camera shake; astronomy; biomedical image reconstruction � 2 D convolution matrix H and Gaussian additive noise vector n � Maximum a-posteriori (MAP) estimation for vector X Prior model for p(X) for natural images? [Elad 2007] � Optimization method? � 2
Analysis-based modeling [Krishnan 2009] Prior based on hyper-Laplacian distribution of the spatial derivative of natural images 3 Linear filtering to compute spatial derivative Fit (0. 5 -0. 8) and (normalization factor) to empirical data
Patch-based modeling � Sparse coding of patches � Spatial receptive fields of visual cortex [Olshausen 1997] � For 10 10 patches � Learn an overcomplete dictionary from natural images. � Application in image restoration � Denoising, superresolution [Yang 2010] � Localized algorithm: patches can overlap � Use this model in deconvolution? [Lee 2007] 4
Prior model in natural images � From local to global � Slow convergence (EM Algorithm) � Patches should not overlap (Why? ) 5 boundary artifacts
Joint modeling Take advantage of patch-based sparse representation while resolving the problems in? Combine analysis-based prior and synthesis-based prior Accelerate convergence Keep consistency on the boundary of adjacent patches Patch-based sparse coding Sparse spatial gradient Keep details and textures 6
Joint modeling � Discard the generative model � Prior probability sparsity of gradients sparsity of representation coefficients � After training, we fix the parameters for all images 7 compatibility term
MAP estimation using the joint model � Problem: likelihood prior � Iteratively updating w and X until convergence � w sub-problem small-scale L 1 regularized square loss minimization � X sub-problem Half-quadratic splitting [Krishnan 2009] 8
Experimental results Initialization: Wiener estimates / blurred images Dictionary: learned from Berkeley Segmentation database Patch size 12 12 Prior parameters: Runtime: (Matlab) 16 s with Intel Core 2 Duo CPU @2. 26 GHz Experiment settings: 9
Experimental results 2 ISNR comparison test 1 test 2 [Krishnan 2009] test 3 [Portilla 2009] test 4 proposed 0, 822 0, 872 0, 922 test 1 SSIM comparison test 2 [Krishnan 2009] test 3 [Portilla 2009] test 4 proposed 10 PASCAL Visual Object Classes Challenge (VOC) 2007 database
Experimental results 11
Experimental results [Krishnan 2009] Original image [Portilla 2009] keeps more brick textures Blurred image 12 Proposed
Experimental results Textures zoomed in Original image [Krishnan 2009] 13 [Portilla 2009] Proposed
Conclusions Global model for MAP estimation Joint model of image pixels and representation coefficients Sparsity of spatial derivative (analysis-based) Sparsity of representation of patches in overcomplete dictionary (synthesis-based) Iterative algorithm Able to solve general non-blind image deconvolution converges in a few iterations Matlab code for the proposed method is available at http: //users. ece. utexas. edu/~bevans/papers/2011/sparsity/ 14
References [Elad 2007] M. Elad, P. Milanfar and R. Rubinstein, “Analysis versus synthesis in signal priors”, Inverse Problems, vol. 23, 2007. [Krishnan 2009] D. Krishnan and R. Fergus, “Fast image deconvolution using hyper-Laplacian priors, ” Advances in Neural Information Processing Systems, vol. 22, pp. 1 -9, 2009. [Olshausen 1997] B. A. Olshausen and D. J. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V 1, ” Vision Research, vol. 37, no. 23, pp. 3311 -3325, 1997. [Portilla 2009] J. Portilla, “Image restoration through L 0 analysis-based sparse optimization in tight frames, ” in Proc. IEEE Int. Conf. on Image Processing, 2009, pp. 3909 -3912. [Yang 2010] J. yang, J. Wright, T. S. Huang and Y. Ma, “Image superresolution via sparse representation, ” IEEE Trans. on Image Processing, vol. 19, no. 11, pp. 2861 -2873, 2010. 15
Thank you! 16
w sub-problem patches do not overlap small-scale l 1 regularized square loss minimization 17
X sub-problem � Conjugate gradient iteratively reweighted least squares � Half-quadratic splitting [Krishnan 2009] auxiliary variable component-wise quartic function 18 No need to solve the equation
MAP estimation using the joint model blurred image; noise level; blurring kernel; initialization of recovered image finish X sub-problem Update the coefficient of patches (w sub-problem) X converges? Set α=α 0 α>αmax ? Update auxiliary variable Y (quartic equation) 19 α=kα Update image X (FFT)
Image Quality Assessment � Full reference metric � ISNR -- increment in PSNR (peak signal-to-noise ratio) � SSIM -- structural similarity [Wang 2004] 20
Prior model of natural images Analysis-based prior Fast convergence Over smooth the images Synthesis-based prior (patch-based sparse representation) 21 Dictionary well adapted to nature images Captures textures well Slow convergence Boundary artifacts
Computational complexity For each iteration: N is the total number of pixels in the image Average runtime comparison 22 [Krishnan 2009] [Portilla 2009] Proposed 2 s 15 s 16 s
- Slides: 22